The Webis-Editorials-18 corpus is a novel corpus with 1000 news editorials.
In the paper, "Challenge or Empower: Revisiting Argumentation Quality in a News Editorial Corpus", we introduce a new notion for news editorials quality based on whether an editorial brings readers of opposing beliefs closer together or rather increases the gap between them. For this reason, we create a Corpus that contains the quality assessments of 1000 news editorials, each annotated by three liberals and three conservatives. The annotators also reported free-text reasons for the effects they observed.
We recruited 24 native English speakers annotators, from the United States via upwork.com. In order to determine the political typology of the annotators, we use the PEW political typology quiz.
For more information, please check our paper.
To download the corpus use the following link: corpus-webis-editorial-quality-18 on Zenodo