Task 2:  Formula Retrieval

Given a question post with an identified formula as a query, search all question and answer posts and return relevant formulas with their posts.

Above is an example query, with a formula taken from the example search for Task 1 at left, along with formulas with their associated posts (i.e., in-context) returned in search results at right. Relevant formulas are shown in green.

Notes

  • Formulas provided in LaTeX, Presentation MathML, and Content MathML. Formulas with the same appearance are grouped using identifiers (for 'visually distinct formulas'), indicated using numeric identifiers in the provided formula index files. These groupings are used in evaluation for the task. 
  • Manual and automatic runs will be collected.
  • Top-k formulas from participants + additional manual runs by organizers (if necessary) will be pooled. 
  • Most topics will be assessed once, some doubly-assessed to check agreement. Assessors may include volunteers from teams along with hired assessors.