The carousel below shows some example dprl projects, with demos/videos where available.
MathDeck is a first-of-its-kind search interface designed to make it easier to edit, reuse, annotate, and share formulas. The system is based around a 'chip and card' metaphor inspired by board games. (funded by NSF & Alfred P. Sloan Foundation)
AccessMath tracks whiteboard contents in lecture videos through time and generates keyframes. 'Ink' in keyframes can be used to jump to where that 'ink' is drawn in the video, or searched using image queries through the Tangent-V search engine. (NSF-funded project - Video Demo)
The first multi-modal equation editing prototype, with support for handwriting, typing, and image input for formulas, along with support for math + keyword search. (NSF-funded project; Video Demo)
Audio Search in Lectures
A prototype that supports searching lecture videos for spoken keyterms (within-speaker), using a modified Dynamic Time Warping algorithm (NSF-funded project; Example Results).
Formula Search Engines
We created Tangent and refined Approach0, two state-of-the-art formula search engines. (Funded by NSF and the Alfred P. Sloan Foundation)
Back in 2008, Kurt Kluever created the first video captcha, where users enter three words describing a video. (Funded by Xerox Corporation)
The dprl has participated in every CROHME competition since its inception in 2011, and co-organized the competition since 2013. CROHME has become a standard benchmark for handwritten formula recognition. The last CROHME was held in 2019, adding a typeset formula detection task.
(Video overview ECIR 2021) ARQMath is a co-operative evaluation exercise aiming to advance math-aware search and the semantic analysis of mathematical notation and texts. ARQMath has two tasks, one for answer retrieval, and one for formula retrieval. The lab has been run for both CLEF 2020 and CLEF 2021 (lab web page). (funded by NSF & Alfred P. Sloan Foundation)