(Video overview ECIR 2021) (ARQMath web page)
ARQMath is a cooperative evaluation exercise aiming to advance math-aware search and the semantic analysis of mathematical notation and texts. The lab has been run for three years, for CLEF 2020, 2021, and 2022.
ARQMath has two main tasks, one for answer retrieval, and one for formula retrieval. In the third year a task for open-domain question answering was added, where answers may come from any source, or may be generated (e.g., by GPT-3). The ARQMath collection has become the largest collection of its kind, with over 200 search topics annotated for the answer retrieval and formula retrieval tasks.
(funded by NSF & Alfred P. Sloan Foundation)