Synopsis

The PAN Wikipedia vandalism corpus 2010 (PAN-WVC-10) is a corpus for the evaluation of automatic vandalism detectors for Wikipedia. For research purposes the corpus can be used free of charge.

This corpus is supplemented by the PAN-WVC-11, which features additional edits in English, Spanish and German. Both corpora should be used to get more representative results.

Download

To download the corpus use the following link:

If you use the dataset in your research, please send us a copy of your publication. We kindly ask you to refer to the corpus via [bib].

Research

As part of our research on automatic vandalism detection we have compiled a corpus of vandalism cases found in Wikipedia. The corpus compiles 32452 edits on 28468 Wikipedia articles, among which 2391 vandalism edits have been identified. To annotate the corpus we have used Amazon's Mechanical Turk; 753 workers have been recruited who cast more than 150000 votes on the edits, so that each edit was reviewed by at least 3 annotators. The achieved level of agreement was analyzed in order to label an edit as "regular" or "vandalism."

The corpus has been successfully employed in the 1st International Competition on Wikipedia Vandalism Detection, PAN'10, which was held in conjunction with the CLEF'10 conference. In the competition, 9 vandalism detectors have been evaluated based on the corpus. More details about the competition can be found in the overview paper.

Previous Corpus Versions. The first vandalism corpus was the Webis-WVC-07. However, that corpus is significantly smaller than the PAN-WVC-10 and its class distribution is not representative.

People

Students: Teresa Holfeld

Publications