ARQMath Home Task 1 - Answers Task 2 - Formulas
Contact Us

Answer Retrieval for Questions on Math (2022)

Task 1:
Answer Retrieval Task 2:
Formula Retrieval Task 3 (New):
Open Domain QA
ARQMath Resources:
Data, Forum, etc.
 ARQMath-2
(2021)
 ARQMath-1
(2020)
ARQMath is a cooperative evaluation task aiming to advance math-aware search and the semantic analysis of mathematical notation and texts.  These web pages are for the third edition of the task, being run as part of the CLEF 2022 conference being held in Bologna, Italy.
CLEF 2022 - Bologna

To learn about participating, and to access ARQMath data, tools, and paper from previous ARQMath labs, please visit the Resources page. Make sure to join the ARQMath Forum to keep up to date with the lab's activities and announcements.

ARQMath Overview and Participant Papers

ARQMath Overview and Participant Papers:
LNCS ARQMath Overview Papers

(Please note: Overview papers in the CEUR Working Notes contain additional results and information)

ARQMath Overview Video (Tasks 1 and 2)

Video. Brief overviews of ARQMath tasks have appeared in ECIR 2020, ECIR 2021, and will be presented at ECIR 2022 this year. A video from Behrooz Mansouri's presentation at ECIR 2021 is provided above -- please note that there is a new Open Domain QA task for ARQMath-3 this year (Task 3).

News

  • (July 11) Looking forward to meeting up with participants at CLEF this Septmber in Bologna! Registration is open through the end of July for both in-person and virtual attendance. Early registration closes July 20.
  • (July 11) The evaluation portion of ARQMath-3 is now complete. A big thanks from the lab organizers to all participants, and to our student assessors from RIT. In the coming weeks we will compile and release additional topics and other data.
  • (Apr 18) pt-arqmath, a new text-based baseline for Task 1 (Answer Retrieval) is available (built on PyTerrier). 
  • (Mar 1) Topics for ARQMath-3 along with an updated collection are now available.
  • Run file formats are available through the ARQMath Google Drive, including for the new Task 3 (Open QA)
  • We welcome Vít Novotný to the organizing team. Vít will lead Task 3.
  • ARQMath-2 Best Paper Award. We congratulate Anja Reusch, Maik Thiele, and Wolfgang Lehner on having their paper "TU_DBS in the ARQMath Lab 2021" selected as the Best Paper of ARQMath-2. A revised version of their paper will be included as part of a special session for Best Papers at CLEF 2022. 
  • ARQMath-1 Best Paper Award. The Best Paper for ARQMath-1 was awarded to Yin Ki Ng, Dallas Fraser, Besat Kassaie and Frank Tompa for their paper "Dowsing for Math Answers" (original version). A revised version was presented in the CLEF 2021 Best Paper track.

Key Dates ( 2022 )

  • Early Feb Mar 1 - Evaluation topics released (Note: previous topics from 2020 / 2021 are available) 
  • May 6 - Last day to submit runs for CLEF 2022 labs (including ARQMath-3)
  • May 27 - Participant paper deadline
  • June 6 June 11 - Condensed LNCS lab overview due (written by organizers) 
  • June 13 - Notification of acceptance for participant papers
  • June 20 June 22  - Camera Ready LNCS Overview Paper due
  • July 1 - Camera Ready Copy of Participant Papers and Overview Paper for CEUR-WS working notes
  • July 18-22 - CEUR-WS preview available for checking by participants and lab organizers

Organizers


Richard Zanibbi (rxzvcs@rit.edu)
Department of Computer Science
Rochester Institute of Technology
 Behrooz Mansouri (bm3302@rit.edu)
Department of Computer Science
Rochester Institute of Technology
Douglas W. Oard (oard@umd.edu)
College of Information Studies, and
Institute for Advanced Computer Studies   
(UMIACS)
University of Maryland, College Park
Anurag Agarwal (axasma@rit.edu)
School of Mathematical Sciences
Rochester Institute of Technology

Vít Novotný (witiko@mail.muni.cz)
Faculty of Informatics
Masaryk University

Questions:  rxzvcs@rit.edu

ARQMath is supported in part by the National Science Foundation (NSF) USA

ARQMath is supported in part by the National Science Foundation (NSF) USA

ARQMath-3 (2022)