build a website

News


2019

  • (Mar) Congratulations to Behrooz Mansouri, whose paper "Toward Math-Enabled Digital Libraries: Characterizing Searches for Mathematical Concepts" has been accepted for publication at JCDL 2019, which will be held at the University of Illinois Urbana-Champaign.  The paper presents the first search log analysis of math search behaviors in a general-purpose search engine (Parsijoo, from Iran).
  • (Mar) Congratulations to Parag Mali, who has secured a full-time Software Engineer position with the Microsoft AI and Ink group in Seattle!
  • (Jan) The CROHME + TFD 2019 handwritten formula recognition and typeset formula detection competition is underway. Mahshad Mahdavi is the lead organizer. Please consider participating in the competition!
  • (Jan) Congratulations to Wei Zhong and Kenny Davila, who have both had full papers on math formula search engines accepted to ECIR 2019 in Cologne, Germany this April. Wei's paper is "Structural Similarity Search for Formulas using Leaf-Root Paths in Operator Subtrees," while Kenny's paper is "Tangent-V: Math Formula Image Search Using Line-of-Sight Graphs."
  • (Jan) Congratulations to Mahshad Mahdavi, who has secured a Machine Learning summer internship with Apple in Cupertino!
  • (Jan) The lab welcomes two new research assistants, Gavin Nishizawa and Yancarlos Diaz. They are working on developing a new math-aware search interface for the MathSeer project.

2018

  • (Dec) Wei Zhong and dprl alumni Kenny Davila have both had papers on math formula search accepted for publication at the European Conference for Information Retrieval (ECIR), one of the leading IR venues. The conference is being held in Cologne, Germany this coming April.
  • (Oct) The DPRL PhD students (Thomas Choi, Mahshad Mahdavi, Behrooz Mansouri, and Wei Zhong) presented their research in the poster seesion at the AI@GCCIS event held on Friday, Oct. 19th. It was a good event, celebrating AI research in the Computing College at RIT.
  • (Oct) In mid-October, Thomas Choi and Prof. Zanibbi visited the Distributed Digital Music Archives and Libraries Lab at McGill University in Montréal. Thomas presented his ongoing work in Optical Music Recognition (OMR). Our sincerest thanks to Dr. Fujinaga and his students for an informative and fun visit.
  • (Oct) Mahshad Madhavi presented a poster on recognizing formulas in images, and Prof. Zanibbi gave an invited talk on formula search in videos and lecture notes at the Western New York Image Processing Workshop (WNYIP) held at RIT in early October.
  • (Sept) Kenny Davila's Best Paper award at ICFHR 2018 has been announced on the CS web pages: link.
  • (Sept) A new version of the AccessMath lecture video indexing code and video data sets are available.
  • (Aug) The DPRL welcomes Abishai Dmello to the lab. Abishai will be completing an MSc thesis on math-aware search.
  • (Aug) Kenny Davila and Prof. Zanibbi received the Best Paper award at ICFHR 2018.
  • (July) Prof. Zanibbi is Co-Chairing ICFHR 2018 in early August, the leading international conference on handwriting recognition and applications.
  • (July) Kenny Davila's Tangent-V system is now available for download. This is a new system for visual search of math in lecture notes and lecture videos, which will be presented at ICFHR 2018 in Niagara Falls in early August.
  • (June) The lab welcomes Kwon-Young Choi, a PhD student from the University of Rennes, France. Kwon-Young's PhD is being co-advised by Bertrand Couasnon, Yann Ricquebourg, and Prof. Zanibbi. Kwon-Young is visiting from June through November of 2018.
  • (May) Congratulations to Wei Zhong, who successfully passed his Research Potential Assessment (the 'comprehensive' in our PhD program) after writing and defending a nice paper on an improved path-based model for formula retrieval using operator trees.
  • (May) Kenny Davila and Prof. Zanibbi's paper, "Visual Search Engine for Handwritten and Typeset Math in Lecture Videos and LaTeX Notes" has been accepted for Oral presentation at ICFHR 2018. The conference is being held in Niagara Falls in August.
  • (May) The conference program and registration for ICFHR 2018 are now online. ICFHR is the leading conference on handwriting recognition and related applications. Prof. Zanibbi is a Co-Chair for the conference this year.
  • (May) Kenny Davila came back to RIT to graduate. It was a very nice time to celebrate his excellent work during his PhD, and to meet his family, including his mother, father, and brother who flew in from Honduras (pictures).
  • (May) Congratulations to Ritvik Joshi, who has secured a full-time position at Petuum Corporation. They are a start-up in Pittsburgh, initiated by some faculty and students at Carnegie Mellon University. They are specializing in providing scalable machine learning as a service.
  • (May) Prof. Zanibbi has been promoted to Full Professor. My thanks to the students in the DPRL, without whom this would not have happened!
  • (Apr.) The lab welcomes Behrooz Mansouri, who will be joining the lab as a PhD student in Fall 2018. Behrooz will be studying math-aware search engines, with an emphasis on using text-math embeddings to improve search in technical documents.
  • (Apr.) Supriya Godge and Aditya Murthy will present their MS Project work in a poster session on Thursday, May 3rd from 1-3pm in the GCCIS atrium. They have been doing work on converting LaTeX output to structure trees for neural math image parsers, and improving a system for text detection in natural scenes.
  • (Mar.) Our lab picture has been updated (see the Members Page).
  • (Mar.) Congratulations to Parag Mali, who will be interning with a pattern recognition group at Apple in Cupertino this summer.
  • (Feb.) The lab welcomes Xuechun Wu, who will be working as a visual designer for the MathSeer project.
  • (Feb.) Congratluations to Puneeth Kukkadapu, who will also be interning with a pattern recognition group at Apple this summer!
  • (Jan.) Prof. Zanibbi will be giving a lightning talk about the lab's work in math-aware search on Feb. 15th in the session from 12:30-2pm in the Golisano Auditorium (facebook page is here).
  • (Jan.)The lab is happy to welcome new members starting in Spring 2018: Puneeth Kukkadapu, MSc Student (Computer Science) - working on math-aware search interfaces; Parag Mali, MSc Student (Computer Science) - working on content extraction and search in lecture videos; Aditya Murthy, MSc Student (Computer Science) - working on text detection in natural scenes.

Earlier News

2017

  • (Dec.) Congratulations to Xuan Huang, who has secured an internship with Facebook in summer 2018.
  • (Dec.) In December, the National Science Foundation (USA) awarded a grant to Prof. Zanibbi, Dr. Anurag Agarwal (RIT), Dr. Douglas Oard (Univ. Maryland, College Park), and Dr. Lee Giles (Penn State) in support of research into exploiting text-math relationships in the design of math-aware search engines, and integration with the CiteSeerX technical paper database.
  • (Nov.) Kenny Davila's work on extracting whiteboard contents from lecture videos is being presented as a poster at ICDAR 2017. Source code and data from the work are available online here.
  • (Nov) The lab's work was featured in a meeting with the RIT Board of Trustees. Mahshad Madhavi and Wei Zhong represented the lab. We thank Nicholas Paulus for preparing the poster used in the session.
  • (Sep) The lab welcomes Mahshad Mahdavi, a new PhD student in the Imaging Science program. Mahshad will be working on machine learning techniques for extracting and recognizing formulas in documents.
  • (Sep) Recent DPRL alumnus Lakshmi Ravi (MSc, 2017) has secured a position as a Data Scientist on Amazon's Alexa Machine Learning group in Boston.
  • (Aug) PhD student Thomas Choi (at INSA Rennes (France), co-advised by Prof. Zanibbi) had his paper on bootstrapping small samples for recognizing accidentals in music notation accepted for publication at GREC 2017. There are many talks on Optical Music Recognition (OMR) planned for the workshop so we expect this to be an exciting GREC!
  • (Aug) We are very happy to introduce the new members of the DPRL in Fall 2017:
    • Rahul Dashora (MSc student) will work on improving the lab's Hierarchical Contextual Parsing (HCP) technique for parsing mathematical notation in PDFs, images, and handwriting.
    • Xuan Huan (MSc student) will work on creating a new web-based system for formula entry and search (based on the lab's 'min' system)
    • Ritvik Joshi (MSc student) will work on extracting formulas from PDF documents.
    • Wei Zhong (PhD student) is working on math-aware search engines.
  • (Aug) Kenny Davila has accepted a post-doctoral fellowship with the well-known CEDAR/CUBS lab at the University of Buffalo. Among the lab's many other accomplishments, CEDAR/CUBS contributed to the first automated mail-sorting system for the US Postal Service.
  • (July) The Tangent-S formula retrieval system has been released, and is available as open source. The changes for this version were made by Kenny Davila.
  • (July) Kardo Aziz has successfully defended his MSc thesis, titled Better Text Detection through Improved k-means-based Feature Learning. In his thesis, Kardo proposes a technique called Visual Similarity Sampling (VSS), which selects training samples using the average similarity of image patches within and between text and non-text classes. He has found evidence that this can improve results over pure uniform sampling when the sample size is fixed, by exploiting representative and discriminative patch characteristics when constructing a training set.
  • (July) Kenny Davila has successfully defended his PhD dissertation, titled Symbolic and Visual Retrieval of Mathematical Notation using Formula Graph Symbol Pair Matching and Structural Alignment. Kenny's dissertation was co-advised by Prof. Zanibbi and Dr. Stephanie Ludi (Univ. North Texas). Kenny created a formula search engine capable of using visual structure, semantics, or just the symbol layout of formulae. From this, he is able to use this to do cross-modal searches to locate formulae in lecture videos from rendered LaTeX formulae. He also created a system that can be used to search for formula, and then jump to a video frame where selected ink from the whiteboard is first drawn (YouTube demo).
  • (June) The Alfred P. Sloan Foundation has funded a proposal by Prof. Zanibbi and C. Lee Giles (Penn State) to integrate math-aware search into the CiteSeerX platform.
  • (June) Michael Condon successfully defended his MSc thesis, Applying Hierarchical Contextual Parsing with Visual Density and Geometric Features to Typeset Formula Recognition. Michael made changes to formula structure representations for punctuation and refined other techniques from Lei Hu's HCP parsing algorithm, applying these to recognizing typeset math. His parser obtains very high expression recognition rates for formulas from the infty dataset (almost 91% parsing from connected components). Source code and data used for his work will be made available in the coming weeks.
  • (June) Kenny Davila's paper Whiteboard Video Summarization via Spatio-Temporal Conflict Minimization has been accepted for publication at ICDAR 2017 (the leading document analysis and recognition conference). There is a YouTube video demonstrating his new system for summarization and navigation of lectures using whiteboard content summaries As far as we know, this is the first system that supports lecture video navigation using whiteboard contents directly.
  • (May) Congratulations to Lakshmi Ravi, who received 2nd place for her MSc Project Poster, titled "Parsing Handwritten Math Formulas." Lakshmi will soon be starting work in the Alexa group at Amazon in Seattle. She will be doing work in Natural Language Processing (NLP).
  • (May) Prof. Zanibbi received the Golisano Computing College Outstanding Scholar Award for a strong track record of research and scholarship that is "integral to, and not separated from, all aspects of a student's educational experience at RIT." Thanks from Prof. Zanibbi to all dprl students from the last ten years who made this possible. This award primarily reflects your hard work and accomplishments - thank you.
  • (Apr.) Prof. Zanibbi gave a brief presentation at the RIT GCCIS Research Showcase about the lab, dprl@10: The Document and Pattern Recognition Lab's First 10 Years. A sincere thanks and congratulations to all of the dprl students from these first 10 years!
  • (Apr.) Chinmay Jain has secured a Software Engineer position involving machine learning for customer recommendations with BBVA Compass.
  • (Apr.) Congratulations to Kenny Davila whose paper Layout and Semantics: Combining Representations for Math Formula Search" has been accepted for poster presentation at SIGIR 2017, the leading Information Retrieval conference.
  • (Apr.) Kenny Davila will be giving a public talk at RIT about his research on math formula retrieval in documents and videos Monday, April 17th at noon in the Bamboo Rooms (Campus Center 003), as part of the Move 78 Seminar Series.
  • (Apr.) The lab gives a warm welcome to Wei Zhong, who will be joining the DPRL as a PhD student in Fall 2017. Wei will be doing work in math-aware search engines. For a glance of his current work in this area, you can try his Approach0 system for searching Math StackExchange posts here.
  • (Mar.) Le Duc Anh from the Nakagawa Lab in Japan visited the lab for a week in early March, to talk with us about his work in handwritten math recognition. His picture can be found under the "Members" link. Anh recently defended his PhD, and will begin working on Computer Vision and Machine Learning at a company in Tokyo next month.
  • (Feb.) Michael Condon has secured a job working as a Pattern Recognition Software Engineer at Apple in Cupertino, CA. He is joining a team that worked on handwriting recognition for the Apple Watch.
  • (Jan.) The web page for ICFHR 2018 is now online, including a call for papers. Please consider submitting your work to the conference, and joining us in Niagara Falls!
  • (Jan.) Prof. Zanibbi will be the new Communications Officer for the IAPR Technical Committee No. 11 ('Reading Systems').

2016

  • (Dec.) Congratulations to Chinmay Jain, whose poster received 2nd place in the Best MS Poster competition! Chinmay's poster is available here.
  • (Nov.) The lab welcomes Jahongir Amirkulov to the lab. Jahongir is an undergraduate Computer Science student who will be working on the min math search interface in the spring semester.
  • (Oct.) Congratulations to Lei Hu, who received a best poster award at the international ICFHR conference for his paper, MST-Based Visual Parsing of Online Handwritten Mathematical Expressions.
  • (Oct.) The call for papers and web pages for the International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) are now online. Prof. Zanibbi is Co-Chairing the conference, which will be held August 6-10, 2018 at RIT.
  • (Oct.) Kenny Davila's work in handwriten symbol generation from examples has been cited in a UIST 2016 paper by Taranta, LaViola et al. (Video Demonstration)
  • (Aug) We welcome Lakshmi Ravi, Michael Condon, Kardo Aziz and Chinmay Jain to the lab (all are CS MSc students). Lakshmi, Michael and Chinmay will be doing work on recognizing handwritten and typeset mathematical notation, while Kardo will be doing work on detecting text in natural scenes for both English and Arabic.
  • (July) Lei Hu and Prof. Zanibbi have had two papers on recognizing handwritten math accepted for publication at ICFHR, being held in Shenzhen, China this October. These papers summarize Lei's doctoral dissertation research.
  • (July) The CROHME 2016 handwritten math recognition competition has ended! Congratulations to MyScript and Wiris for obtaining the highest recognition rates. Details of the competition will appear in an upcoming paper at ICFHR.
  • (June) The new Tangent 0.3.1 formula search engine has been released (see Software link, above). This new version was a joint RIT-Univ. Waterloo collaboration, created by K. Davila, A. Kane, F.Wm. Tompa, and Prof. Zanibbi.
  • (May) The NTCIR-12 MathIR Task Overview paper is now available. The paper describes the math search competitions held as part of the upcoming NTCIR-12 conference being held in Tokyo.
  • (May) Kenny Davila, Andrew Kane, Frank Tompa and Prof. Zanibbi's paper about the Tangent math search engine at the NTCIR-12 math retrieval competitions is now available. For the competition we improved support for wildcard matching, and explored new metrics for formula similarity.
  • (May) Congratulations to Yi Huang, who has accepted a position at BOSE corporation in Boston.
  • (May) Congratulations to Kenny Davila who has successfully defended his PhD dissertation proposal entitled "Appearance-Based Retrieval of Mathematical Notation in Documents and Lecture Videos". The proposal builds upon Kenny's earlier research, including his recently accepted SIGIR 2016 paper describing modifications to the Tangent formula search engine.
  • (Apr.) Congratulations to Dr. Siyu Zhu, who has successfully defended his doctoral dissertation entitled "Text Detection for Natural Scenes and Technical Diagrams with Convolutional Features and Cascaded Classification." Part of Siyu's doctoral research was recently accepted for publication at CVPR 2016. Siyu's innovative, but relatively simple text detection system obtained the highest rates for the standard ICDAR 2015 Focused Scene Text benchmark at the time of publication.
  • (Apr.) Congratulations to Dr. Lei Hu, who has sucessfully defended his doctoral dissertation entitled "Features and Algorithms for Visual Parsing of Handwritten Mathematical Expressions." His dissertation presents new visual features and parsing algorithms (including parser ensembles) for recognizing handwritten mathematical notation, which illustrate that competitive recognition results may be obtained using very minimal language models - existing state-of-the-art approaches use expression grammars, which are defined and refined manually.
  • (Apr.) Congratulations to Kenny Davila, who has received a travel award from the ACM to attend SIGIR 2016 in Pisa, Italy.
  • (Apr.) Source code for Siyu Zhu's text detector for natural scenes is available from the Software link above.
  • (Apr.) Congratulations (again) to Kenny Davila, who has been selected to participate in the Doctoral Consortium at SIGIR 2016.
  • (Mar.) Congratulations to Kenny Davila, who co-authored a paper on scalable formula search entitled Multi-Stage Math Formula Search: Using Appearance-Based Similarity Metrics at Scale, which was accepted for publication at ACM SIGIR 2016 being held in Pisa, Italy. Kenny's co-authors include Frank Tompa and Andrew Kane from Univ. Waterloo, and Prof. Zanibbi. SIGIR is the leading international conference on Information Retrieval.
  • (Mar.) Congratulations to Siyu Zhu, whose paper "A Text Detection System for Natural Scenes with Convolutional Feature Learning and Cascaded Classification" was accepted for publication at CVPR, the leading international computer vision conference. Siyu obtained state-of-the-art text detection results for the ICDAR 2015 Robust Reading Competition (Focused Scene Text Localization task).
  • (Jan.) We welcome Yi Huang to the lab. Yi will create a web-based evaluation framework for handwritten math recognition systems, and look to extend this to support other recognition tasks (e.g. from computer vision and/or acoustics).
Address

Room 70-3500, GCCIS
Dept. Computer Science
Rochester Inst. Technology
Rochester, NY, 14623-5608
USA

Contacts

Email: rxzvcs@rit.edu
Phone: +1 (585) 475-4536
Fax: +1 (585) 475-4935