Events

Conferences and Distinguished Workshops

Webis Symposium 2024
"1st International Webis Symposium", held at the Bauhaus-Universität Weimar, program co-chairs Matthias Hagen, Martin Potthast, and Henning Wachsmuth, and local chair Johannes Kiesel. Goal: reflect on the past 20 years of Webis, present current work, and discuss future directions.

WOWS 2024 @ ECIR 2024
"1st International Workshop on Open Web Search (WOWS)", held in conjunction with the 46th European Conference on Information Retrieval (ECIR 2024), co-organized by Sheikh Mastura Farzana, Maik Fröbe, Gijs Hendriksen, Michael Granitzer, Djoerd Hiemstra, Martin Potthast, Saber Zerhoudi. Goal: promote and discuss ideas and approaches to open up the web search ecosystem so that small research groups and young startups can leverage the web to foster an open and diverse search market.

SCAI 2024 @ CHIIR 2024
"8th Workshop on Search-Oriented Conversational Artificial Intelligence (SCAI)", held in conjunction with the 9th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2024), co-organized by Alexander Frummet, Andrea Papenmeier, Maik Fröbe, and Johannes Kiesel. Goal: exploring conversational AI for intelligent information access.

CLEF Labs 2022
Labs of the 13th Conference and Labs of the Evaluation Forum "CLEF", a continuation of the popular CLEF campaigns which have run since 2000, lab chairs Allan Hanbury and Martin Potthast. Goal: contribute to the systematic evaluation of information access systems, primarily through experimentation on shared tasks.

TMG 2022 @ KI 2022
"1st Workshop on Text Mining and Generation (TMG)", held in conjunction with the 45th German Conference on Artifical Intelligence (KI 2022), co-organized by Alexander Bondarenko and Shahbaz Syed. Goal: bridging the gap between text mining (knowledge extraction) and text generation.

ArgMining 2019 @ ACL 2019
"6th Workshop on Argument Mining", held in conjunction with the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), and chaired by Benno Stein and Henning Wachsmuth. Goal: foster future research and development in the area of computational argument mining. [proceedings]

PODA 2018
"Algorithmic Discourse Analysis", an interdisciplinary workshop chaired by Henning Schmidgen, Benno Stein, and Frank Simon-Ritz in 2018. Goal: bring together researchers and practitioners from the humanities, computer science, and information science to explore digital methods for the analysis of discourses.

CLEF Conference 2013
4th Conference and Labs of the Evaluation Forum "CLEF", a continuation of the popular CLEF campaigns which have run since 2000, chaired by Paolo Rosso and Benno Stein. Goal: contribute to the systematic evaluation of information access systems, primarily through experimentation on shared tasks.

Dagstuhl Seminars

Retrieval-Augmented Generation – The Future of Search? @ Schloss Dagstuhl 2025
Dagstuhl Seminar 25391 "Retrieval-Augmented Generation – The Future of Search?", to be organized by Matthias Hagen, Josiane Mothe, Smaranda Muresan, Martin Potthast, Benno Stein, and Min Zhang in 2025. Goal: investigate the expectations, the promises, the potential, and the limits of integrating retrieval-augmented generatoipn (RAG) in information retrieval (IR).

Towards a Unified Model of Scholarly Argumentation @ Schloss Dagstuhl 2022
Dagstuhl Seminar 22432 "Towards a Unified Model of Scholarly Argumentation", organized by Khalid Al-Khatib, Anita de Waard, Iryna Gurevych, and Yufang Hou in 2022. Goal: Lay the groundwork for a nascent, multidisciplinary community devoted to building and maintaining principles, tools, and models to identify key components in scholarly argumentation.

Framing in Communication @ Schloss Dagstuhl 2022
Dagstuhl Seminar 22131 "Framing in Communication: From Theories to Computation", organized by Katarzyna Budzynska, Chris Reed, Manfred Stede, and Benno Stein in 2022. Goal: carefully study and assess the various relevant disciplines (sociology, political science, psychology, communication science, and others) to develop (1) a common ground for the computational treatment of framing and (2) a roadmap for the automatic identification of framing in text and speech.

Conversational Search @ Schloss Dagstuhl 2019
Dagstuhl Seminar 19461 "Conversational Search", organized by Avishek Anand, Hideo Joho, Mark Sanderson, and Benno Stein in 2019. Goal: bring together leading researchers from relevant communities to understand and analyze the promising retrieval paradigm of conversational search and its future from different angles like interactivity, result presentation, clarification, user models, evaluation, and search behavior. [report]

Debating Technologies @ Schloss Dagstuhl 2015
Dagstuhl Seminar 15512 "Debating Technologies", organized by Iryna Gurevych, Eduard H. Hovy, Noam Slonim, and Benno Stein in 2015. Goal: set the grounds for a new interdisciplinary research community, interested in Debating Technologies, defined henceforth as computational technologies developed directly to enhance, support, and engage with human debating. [report]

Challenges in Document Mining @ Schloss Dagstuhl 2011
Dagstuhl Seminar 11171 "Challenges in Document Mining", organized by Hamish Cunningham, Oren Etzioni, Norbert Fuhr, and Benno Stein in 2011. Goal: bring together researchers from the main areas of document mining to present their views, understand where and how latest disciplinary achievements can be combined, and develop an integrative view on document mining. [report]

Invited Talks

Elements of the Information Nutrition Label: Understanding Trigger Warnings
Martin Potthast at the DLR Open Search Colloquium in July 2024. [slides]

Understanding to Stay in Control: Using, Assessing, and Mastering AI
Keynote of Benno Stein and Johannes Kiesel at the Jahrestagung des eTeach-Netzwerks Thüringen in Jena in November 2024. [slides]

The Evolution of Web Search
Invited talk at the Research Center for Information System Design (ITeG) in Kassel in November 2024. [slides]

Generative Search and Biases: Challenges and Opportunities of a New Information Ecology
Martin Potthast at the Webinar series: 9 Months, 9 Universities of the Arqus Plurilingual and Intercultural Hub in June 2024. [slides] [video]

TIRA for IR: Theory and Hands-On Tutorial
Maik Fröbe at the IR Lab in Augsburg in June 2024. [slides]

Perspectives on Evaluating Diverse Open Web Search Applications with TIREx
Maik Fröbe at the IR Lab in Padua in March 2024. [slides] [video]

Hackathon for The First Workshop on Open Web Search
Maik Fröbe at TU Dresden in April 2024. [summary]

Generative Search and Biases: Challenges and Opportunities of a New Information Ecology
Martin Potthast at the German National Library (DNB) professional event AI in Libraries: New Pathways with Large Language Models? in Frankfurt in December 2023. [slides] [video]

From Noise to Art: User-Controlled Image Generation Beyond Prompt Engineering
Niklas Deckers at the DFG-funded lecture series Feeds & Flows: Interdisciplinary Perspectives on Ephemeral Image Cultures in Weimar in November 2023. [slides]

Biases and Generative AI
Matthias Hagen, Martin Potthast, and Benno Stein at the BMBF DigitalGipfel in Jena in November 2023. [slides] [video]

Manipulating Embeddings of Stable Diffusion Prompts
Invited poster for the ScaDS.AI delegation at the All Hands Meeting of the German AI competence centers by Niklas Deckers, October 2023.

Open Web Search And its Use to Find Arguments Against Populist Claims
Johannes Kiesel at the Joint Research Centre of the European Commission (JRC) Disinformation Workshop in Brussels in September 2023. [slides]

Query Automation for Systematic Reviews
Harry Scells at the Glasgow IR Group Seminar in July 2023. [slides]

Cognitive Biases and Information Retrieval
Benno Stein at the DLR Open Search Colloquium in July 2023. [slides] [video]

Web Search: The Next Generation
Martin Potthast at the ScaDS.AI State Secretary Visit in June 2023. [slides]

Blinded Experiments with TIRA
Maik Fröbe at the Institute for Applied Computer Science in June 2023. [slides]

Open Web Search and Web Archive Analytics at Scale
Martin Potthast at the HPC Status Conference 2023. [slides]

Language, Language Models, and ChatGPT
Benno Stein at the Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft (TMWWDG) in March 2023.
Martin Potthast at the ARQUS Forum Digital Humanities and the Final Undimes Project Meeting in 2023. [slides] [video]

TIREx: The Information Retrieval Experiment Platform
Maik Fröbe at the Glasgow IR Group Seminar in March 2023. [slides]

Green Information Retrieval Research
Harry Scells at the University of Linz in January 2023. [slides]

IR and NLP Research in the Webis Group
Benno Stein at the Paderborn Data Science Colloquium in December 2022. [slides]

Clickbait Spoiling at SemEval-2023
Maik Fröbe at TU Dresden (18.10.2022), TH Köln (21.10.2022), Uni Regensburg (26.10.2022), and Uni Leipzig (07.11.2022). [slides]

The Treachery of Answers
Martin Potthast at the ScaDS.AI General Assembly in December 2022. [slides]

Algorithms for Authorship Analytics
Keynote of Benno Stein at the Lange Nacht der Wissenschaften 2022 in Weimar. [slides]

Finding Answers to Complex Medical Questions
Harry Scells at the Hybrid Network Meeting of the Alexander von Humboldt Foundation in November 2022. [slides]

Touché @ CLEF: Shared Task on Argument Retrieval
Alexander Bondarenko at the Search Engines Course at the University of Padova in 2022. [slides]

Perceived Limits in Information Retrieval
Keynote of Benno Stein at the CLEF 2022 Conference and Labs of the Evaluation Forum in Bologna. [slides]

A new Resource for Analyzing Collaborative Writing Styles and One-Sidedness
Erik Körner at the Scientific Authorship and Peer Review Workshop at the German Center for Higher Education Research and Science Studies in September 2022. [slides]

Webis-STEREO-21: Exploring Scientific Text Reuse at Scale
Victor Zimmermann at the Scientific Authorship and Peer Review Workshop at the German Center for Higher Education Research and Science Studies in September 2022. [slides]

A User Study on Clarifying Comparative Questions
Alexander Bondarenko at the SCAI 2022 Workshop on Search-Oriented Conversational AI in July 2022. [slides]

Exploring our Digital Past: Web Archive Analytics
Keynote of Martin Potthast at the ARQUS Forum Digital Humanities in June 2022. [slides] [video]

Investigating and Mitigating Topic Bias in Authorship Attribution
Niklas Deckers at the ScaDS.AI Colloquium in May 2022. [slides]

Web Archive Analytics
Martin Potthast at the ZIH Colloquium in April 2022. [slides]

Quantifying Luhmann: A Semi-Supervised Approach to Automatic Detection of Social Systems
Arno Simons and Martin Potthast at the Dagstuhl Seminar 22131 "Framing in Communication: From Theories to Computation" in March 2022. [slides]

Touché @ CLEF: Shared Task on Argument Retrieval
Alexander Bondarenko at the Search Engines Course at the University of Padova in 2021. [slides]

Practical Experiences and New Challenges in Web Crawling
Christopher Schröder and Martin Potthast at the DLR Open Search Colloquium in June 2021. [slides]

Algorithms for the Manipulation of Writing Style
Keynote of Martin Potthast at Joint 69th Student Conference on Linguistics and Computational Linguistics Students Conference 2021. [slides]

Authorship Analysis and Obfuscation
Matthias Hagen and Martin Potthast at "Imitatoren des Menschlichen", a talk series of the Institure Groupd of Sociology at Martin-Luther-Universität Halle-Wittenberg in February 2021. [slides]

Tackling Societal Challenges with Style Analysis
Martin Potthast at the Christmas Lecture 2020 of the Faculty of Mathematics and Computer Science at Leipzig University. [slides]

ACQuA: Answering Comparative Questions with Arguments
Alexander Bondarenko at the Digital Humanities and AI Seminar at Perm State University in December 2020. [slides]

Extracurricular Teaching

Using AI in Everyday Life at the University
Internal seminar at the Bauhaus-Universität Weimar by Benno Stein, Johannes Kiesel, and Marcel Gohsen, November 2024. [slides]

Wie lernen Computer?
Lecture at the Kinderuni Weimar by Benno Stein and Johannes Kiesel, November 2024. [slides]

Crash course on LLM Research and Paper Writing
Lab for high school students at Tag der Technik at University of Kassel hosted by Tim Hagen and Niklas Deckers, 2024.

Introduction to Chain-of-Thought Prompting
Talk at the Leipzig University Open Day by Niklas Deckers, 2024.

Generative Text-To-Image Models
Talk at a ScaDS.AI Living Lab event for alumni of the German Academic Scholarship Foundation by Niklas Deckers, 2023.

Solving Complex Problems Using Language Models
Interactive session on Chain-of-Thought Prompting at the Leipzig University Studium Universale "Creative AI Breakfast" by Niklas Deckers, 2023.

Prompting Text-To-Image Models and Chain-of-Thought Prompting
Two talks and two demonstrators at the Leipzig Lange Nacht der Wissenschaften by Niklas Deckers, 2023.

Exchange of Experiences and Discussion About the Use of Lightboards in Teaching
Talks at Leipzig University for the E-Learning Team and at the Tag der Lehre by Niklas Deckers, 2023. [slides]

Weights & Biases Event on Reproducibility, Hyperparameter Optimization and Evaluation
Tutorial by Ayush Thakur, ML engineer at Weights & Biases, co-hosted by Niklas Deckers, 2023. [slides]

Keynote about the current progress in Generative Models and ongoing research approaches
Talk at a ScaDS.AI Living Lab symposium with members of the German Academic Scholarship Foundation by Niklas Deckers, 2022.

Stable Diffusion: AI-generated images
Hands-on session at ScaDS.AI hosted by Niklas Deckers, 2022.

Introduction to Prompt Engineering: Strategies and Examples @ ScaDS.AI 2022
Research tutorial and seminar at the Center for Scalable Data Analytics and Artificial Intelligence, Leipzig, held by Niklas Deckers.

ML Demo.cratization Tour with Hugging Face 2022
Introduction to the Hugging Face ecosystem and tutorial on the creation of shareable machine learning apps, organized by Christopher Akiki, held by Omar Espejel from Hugging Face.

Argumentation Technology @ KI 2020
Tutorial "Argumentation Technology", organized by Philipp Cimiano, Henning Wachsmuth, Khalid Al-Kathib, and Benno Stein in 2020. Goal: outline the importance of argumentation for artificial intelligence while putting the focus on three important fields within computational argumentation: argumentation mining, argumentation retrieval, and argumentation synthesis.

Argumentation Technology @ KI 2019
Tutorial "Argumentation Technology", organized by Philipp Cimiano, Henning Wachsmuth, and Benno Stein in 2019. Goal: outline the importance of argumentation for artificial intelligence while putting the focus on three important fields within computational argumentation: argumentation mining, argumentation retrieval, and argumentation synthesis.

Stylometry and Paraphrasing @ Studienstiftung des deutschen Volkes 2017 - 2019
Research Training Group "Stylometry and Paraphrasing", organized by Matthias Hagen, Martin Potthast, Benno Stein, and Efstathios Stamatatos. Funded by the Studienstiftung des deutschen Volkes within the natural sciences and engineering program and organized as week-long retreats in Cologne (2017), Heidelberg (2018), Weimar (2018), and Springe (2019). Goal: theoretical and practical analysis of the state of the art in authorship analytics and development of new authorship verification approaches based on deep learning and phonetic analyses.

Setting the News Straight Through Technology @ Studienstiftung des deutschen Volkes 2018
Workshop "Fake News Detection Technology", organized by Matthias Hagen, Johannes Kiesel, and Martin Potthast in 2018. Goal: joint work on technology to detect clickbait and hyperpartisan news as well as retrieval technology to search police press releases and arguments.

Writing Style and Anonymity @ Studienstiftung des deutschen Volkes 2017
Workshop "Writing Style and Anonymity" (Schreibstil und Anonymität), organized by Martin Potthast in 2017. Goal: discuss current capabilities of technologies to manipulate writing style and possible repercussions on society of future applications.

NLP Approaches to Computational Argumentation @ ACL 2016
Tutorial "NLP Approaches to Computational Argumentation", organized by Noam Slonim, Iryna Gurevych, Chris Reed, and Benno Stein at the 54th Annual Meeting of the Association for Computational Linguistics (ACL 2016). Goal: review recent advances in the field of Computational Argumentation and outline the challenging research questions for the ACL community that naturally arise when trying to model human argumentation. [slides]

Summer Academy "Who Wrote the Web?" @ Studienstiftung des deutschen Volkes 2015
Workshop "Who Wrote the Web?", organized by Matthias Hagen and Martin Potthast in 2015. Goal: reproduce the top most influential approaches, and to share the resulting code base to foster future research and development in this area. [code] [paper]

Building a Research Group @ HLF 2014
HLF Workshop "Building a Successful Research Group", organized by Martin Potthast at the 2nd Heidelberg Laureate Forum in 2014. Goal: discuss with young researchers and laureates about the challenges and best practices of building a research group, and gain insights into the question of whether building a research group is better for a scientific career than working alone. [abstract] [slides]

Balance @ HLF 2013
HLF Workshop "Balance", organized by Matthias Hagen at the 1st Heidelberg Laureate Forum in 2013. Goal: discuss with young researchers and laureates about issues of maintaining a work-life balance as a successful scientist, and gain insights into the question of how to develop a research career and a growing family. [abstract] [slides]

WIQE 2010
PhD colloquium "Web Information and Quality Evaluation", organized by Paolo Rosso, Alberto Barrón-Cedeño, and Benno Stein 2010 in Valencia, Spain. Funded by a Marie Curie Action under the Seventh Framework Program. Goal: development of promising research directions related to robust methods for detecting low quality information, judging credibility and reliability, and separating contradicting facts or outdated information from valuable information assets.

Shared Tasks

Chat Analytics for Twitch @ ECML PKDD 2020
A shared task organized by Konstantin Kobe, Martin Potthast, Albin Zehe, Matti Wiegmann, Andreas Hotho, and Benno Stein. Goal: predict a user's subscription status regarding Twitch channels they are interacting with. Four teams participated in this task.

TL;DR Challenge @ INLG 2021
A shared task organized by Shahbaz Syed, Wei-Fan Chen, Matthias Hagen, Benno Stein, Henning Wachsmuth, and Martin Potthast. Goal: generate abstractive snippets of web pages given the query and the web content.

Same Side Stance Classification @ ArgMining 2019
A shared task organized by Yamen Ajjour, Khalid Al-Khatib, Philipp Cimiano, Roxanne El Baff, Basil Ell, Benno Stein, and Henning Wachsmuth. Goal: given two arguments on a certain topic, decide whether or not the two arguments have the same stance. Eleven teams participated in this task.

TL;DR Challenge @ INLG 2019
A shared task organized by Shahbaz Syed, Michael Völske, Nedim Lipka, Benno Stein, Hinrich Schütze, and Martin Potthast. Goal: generate short summaries of social media posts using abstractive summarization technology. The task was hosted on the Webis evaluation-as-a-service platform TIRA. [paper]

GameStory: E-Sport Match Summarization @ MediaEval 2019
A shared task organized by the Mathias Lux, Michael Riegler, Duc-Tien Dang-Nguyen, Marcus Larson, Martin Potthast, and Pål Halvorsen. Goal: synthesis of summaries of e-sports matches. Twenty-six teams participated in this task.

Argument Reasoning Comprehension @ SemEval 2018
A shared task organized by Ivan Habernal, Henning Wachsmuth, Iryna Gurevych, and Benno Stein, Webis. Goal: analyze and better understand certain hidden mechanisms of reasoning. Twenty-two teams participated in this task. [paper]

GameStory: E-sport Match Summarization @ MediaEval 2018
A shared task organized by the Mathias Lux, Michael Riegler, Duc-Tien Dang-Nguyen, Marcus Larson, Martin Potthast, and Pål Halvorsen. Goal: synthesis of summaries of e-sports matches. Three teams participated in this task. [paper]

Universal Dependencies @ CoNLL 2018
A shared task organized by the Charles University, Uppsala University, University of Turku, Google, and Leipzig University. Goal: learning syntactic dependency parsers, that take raw text as input, and that can work over many typologically different languages, even low-resource ones. Twenty-six teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA. [paper]

Knowledge Base Quality and Search @ WSDM 2017
Two shared tasks organized by Stefan Heindorf, Hannah Bast, and Martin Potthast. Goal: (1) detect vandalism in Wikidata, and (2) compute relevance scores for triples from type-like relations. Five teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA. [paper]

Universal Dependencies @ CoNLL 2017
A shared task organized by the Charles University, Uppsala University, University of Turku, Google, Bauhaus-Universität Weimar, Stanford University, Ohio State University, Bar-Ilan University, Open University of Israel, University of Cambridge, The Arctic University of Norway, Univerity of Oslo, Leipzig University, IBM Research, University of Tübingen, University of Hamburg, University of Geneva, University of Helsinki, University of Stuttgart, DFKI, and University of Saarland. Goal: learning syntactic dependency parsers, that take raw text as input, and that can work over many typologically different languages, even low-resource ones. Thirty-three teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA. [paper]

Clickbait Challenge @ Google DNI 2017
Two shared tasks organized by Martin Potthast, Tim Gollub, Matthias Hagen, and Benno Stein. Goal: Detect teaser messages in social media that manipulate readers to click a link, namely clickbait. Thirteen teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA. [paper]

Multilingual Shallow Discourse Parsing @ CoNLL 2016
A shared task organized by the Brandeis University, the National University of Singapore, Boulder Learning, Inc., and the University of Edinburgh. Goal: extracting discourse relations from newswire text in the form of a discourse connective (explicit or implicit) taking two arguments (which can be clauses, sentences, or multi-sentence segments). Twenty-three teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA.

Shallow Discourse Parsing @ CoNLL 2015
A shared task organized by the Brandeis University, the National University of Singapore, Boulder Language Technologies, and the University of Wisconsin-Milwaukee. Goal: extracting discourse relations from newswire text in the form of a discourse connective (explicit or implicit) taking two arguments (which can be clauses, sentences, or multi-sentence segments). Sixteen teams participated in this task. The task was hosted on the Webis evaluation-as-a-service platform TIRA.

SCAI QReCC Shared Task 2021
Conversational Question Answering (QA) is one of the core applications for retrieval-based chatbots. In conversational QA, the task is to answer a series of contextually-dependent questions like they may occur in natural human-to-human conversations. The task was hosted on the Webis evaluation-as-a-service platform TIRA.

SCAI QReCC Shared Task 2022
Conversational Question Answering (QA) is one of the core applications for retrieval-based chatbots. In conversational QA, the task is to answer a series of contextually-dependent questions like they may occur in natural human-to-human conversations. The task was hosted on the Webis evaluation-as-a-service platform TIRA.

PAN Research Network on Digital Text Forensics

PAN @ CLEF 2024
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2024) in Grenoble, France.
[style change detection] [multilingual text detoxification] [oppositional thinking analysis] [generative ai authorship verification]

PAN @ CLEF 2023
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2023) in Thessaloniki, Greece.
[authorship verification] [profiling irony and stereotype spreaders on twitter] [style change detection] [trigger detection]

PAN @ SemEval 2023
In conjunction with the 17th International Workshop on Semantic Evaluation (SemEval 2023).
[clickbait spoiling]

PAN @ CLEF 2022
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2022) in Bologna, Italy.
[authorship verification] [profiling irony and stereotype spreaders on twitter] [style change detection]

PAN @ CLEF 2021
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2021) in Bucharest, Romania.
[cross-domain authorship verification] [profiling hate speech spreaders on twitter] [style change detection]

PAN @ CLEF 2020
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2020) in Thessaloniki, Greece.
[celebrity profiling] [cross-domain authorship verification] [profiling fake news spreaders on twitter] [style change detection]

PAN @ CLEF 2019
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2019) in Lugano, Switzerland.
[bots and gender profiling] [celebrity profiling] [cross-domain authorship attribution] [style change detection]

PAN @ SemEval 2019
In conjunction with FactMata Ltd. and the 13th Workshop on Semantic Evaluation (SemEval 2019).
[hyperpartisan news detection]

PAN @ CLEF 2018
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2018) in Avignon, France.
[author identification] [author profiling] [author obfuscation] [obfuscation evaluation] [style change detection]

PAN @ CLEF 2017
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2017) in Dublin, Ireland.
[author profiling] [author masking] [author clustering] [obfuscation evaluation] [style change detection]

PAN @ CLEF 2016
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2016) in Évora, Portugal.
[author clustering] [author diarization] [author profiling] [author masking] [obfuscation evaluation]

PAN @ FIRE 2015
In conjunction with the Information Retrieval Society of India and the Forum for Information Retrieval Evaluation (FIRE 2015) in Gandhinagar, India.
[plagiarism detection] [source code reuse detection]

PAN @ CLEF 2015
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2015) in Toulouse, France.
[author profiling] [authorship verification] [source retrieval] [text alignment]

PAN @ FIRE 2014
In conjunction with the Information Retrieval Society of India and the Forum for Information Retrieval Evaluation (FIRE 2014) in Bagalore, India.
[source code reuse detection]

PAN @ CLEF 2014
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2014) in Sheffield, UK.
[authorship verification] [author profiling] [source retrieval] [text alignment]

PAN @ FIRE 2013
In conjunction with the Information Retrieval Society of India and the Forum for Information Retrieval Evaluation (FIRE 2013) in New Delhi, India.
[news story search]

PAN @ CLEF 2013
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2013) in Valencia, Spain.
[author verification] [author profiling] [source retrieval] [text alignment]

PAN @ FIRE 2012
In conjunction with the Information Retrieval Society of India and the Forum for Information Retrieval Evaluation (FIRE 2012) in Kolkata, India.
[news story search]

PAN @ CLEF 2012
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2012) in Rome, Italy.
[authorship attribution] [sexual predator identification] [source retrieval] [text alignment] [wikipedia quality flaw prediction]

PAN @ FIRE 2011
In conjunction with the Information Retrieval Society of India and the Forum for Information Retrieval Evaluation (FIRE 2011) in Bombay, India.
[plagiarism detection]

PAN @ CLEF 2011
In conjunction with the Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2011) in Amsterdam, Netherlands.
[authorship attribution] [external plagiarism detection] [intrinsic plagiarism detection] [wikipedia vandalism detection]

PAN @ CLEF 2010
In conjunction with the Conference on Multilingual and Multimodal Information Access Evaluation (CLEF 2010) in Padua, Italy.
[plagiarism detection] [wikipedia vandalism detection]

PAN @ SEPLN 2009
In conjunction with the 25th Annual Conference of the Spanish Society for Natural Language Processing (SEPLN 2009) in San Sebastian, Spain.
[external plagiarism detection] [intrinsic plagiarism detection] [proceedings]

PAN @ ECAI 2008
In conjunction with the 18th European Conference on Artificial Intelligence (ECAI 2008) in Patras, Greece. [proceedings]

PAN @ SIGIR 2007
In conjunction with the 30th Annual ACM SIGIR Conference (SIGIR 2007) in Amsterdam, Netherlands. [proceedings]

Touché Research Network on Computational Argumentation and Causality

Touché @ CLEF 2025
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2025) in Madrid, Spain.
[retrieval-augmented debating] [ideology and power identification in parliamentary debates] [image retrieval/generation for arguments] [advertisement in retrieval-augmented generation]

Touché @ CLEF 2024
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2024) in Grenoble, France.
[human value detection] [ideology and power identification in parliamentary debates] [image retrieval/generation for arguments]

Touché @ CLEF 2023
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2023) in Thessaloniki, Greece.
[argument retrieval for controversial questions] [evidence retrieval for causal questions] [image retrieval for arguments] [multilingual multi-target stance classification]

Touché @ SemEval 2023
In conjunction with the 17th International Workshop on Semantic Evaluation (SemEval 2023) in Toronto, Canada. 39 teams participated.
[human value detection]

Touché @ CLEF 2022
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2022) in Bologna, Italy. 23 teams participated.
[argument retrieval for comparative questions] [argument retrieval for controversial questions] [image retrieval for arguments]

Touché @ CLEF 2021
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2021) in Bucharest, Romania. 27 teams participated.
[argument retrieval for comparative questions] [argument retrieval for controversial questions]

Touché @ CLEF 2020
In conjunction with the Conference and Labs of the Evaluation Forum (CLEF 2020) in Thessaloniki, Greece. 17 teams participated.
[argument retrieval for comparative questions] [argument retrieval for controversial questions]

TIR Workshop Series on Information Retrieval

TIR @ DEXA 2019
In conjunction with the 30th Conference on Database and Expert Systems Applications (DEXA 2019) in Linz, Austria.

TIR @ DEXA 2018
In conjunction with the 29th Conference on Database and Expert Systems Applications (DEXA 2018) in Regensburg, Germany.

TIR @ DEXA 2017
In conjunction with the 28th Conference on Database and Expert Systems Applications (DEXA 2017) in Lyon, France.

TIR @ DEXA 2016
In conjunction with the 27th Conference on Database and Expert Systems Applications (DEXA 2016) in Porto, Portugal.

TIR @ DEXA 2015
In conjunction with the 26th Conference on Database and Expert Systems Applications (DEXA 2015) in Valencia, Spain.

TIR @ DEXA 2014
In conjunction with the 25th Conference on Database and Expert Systems Applications (DEXA 2014) in Munich, Germany.

TIR @ DEXA 2013
In conjunction with the 24th Conference on Database and Expert Systems Applications (DEXA 2013) in Prague, Czech Republic.

TIR @ DEXA 2012
In conjunction with the 23rd Conference on Database and Expert Systems Applications (DEXA 2012) in Vienna, Austria.

TIR @ DEXA 2011
In conjunction with the 22nd Conference on Database and Expert Systems Applications (DEXA 2011) in Toulouse, France.

TIR @ DEXA 2010
In conjunction with the 21st Conference on Database and Expert Systems Applications (DEXA 2010) in Bilbao, Spain.

TIR @ DEXA 2009
In conjunction with the 20th Conference on Database and Expert Systems Applications (DEXA 2009) in Linz, Austria.

TIR @ DEXA 2008
In conjunction with the 19th Conference on Database and Expert Systems Applications (DEXA 2008) in Turin, Italy.

TIR @ DEXA 2007
In conjunction with the 18th Conference on Database and Expert Systems Applications (DEXA 2007) in Regensburg, Germany.

TIR @ ECAI 2006
In conjunction with the 17th European Conference on Artificial Intelligence (ECAI 2006) in Riva del Garda, Italy. [proceedings]

TIR @ KI 2005
In conjunction with the 28th German Conference on Artificial Intelligence (KI 2005) in Koblenz, Germany. [proceedings]

TIR @ KI 2004
In conjunction with the 27th German Conference on Artificial Intelligence (KI 2004) in Ulm, Germany. [proceedings]

PuK Workshop Series on Configuration and Design

PuK 2004
18th Workshop of the Special Interest Group on Planning, Scheduling, and Design of the German Computer Science Society (GI), organized 2004 in Ulm, Germany. [proceedings]

PuK 1998
12th Workshop of the Special Interest Group on Planning, Scheduling, and Design of the German Computer Science Society (GI), organized 1998 in Paderborn, Germany. [proceedings]