(Sept) Congratulations to Robin Avenoso (BSc/MSc student), who received the Outstanding Undergraduate Scholar Award. This award celebrates the top 1% of undergraduate students who are able to maintain a high standard of academic excellence while also giving back to their community through civic or volunteer work, conducting research, or being engaged in a co-op or work in their field of study.
(Sept) Congratulations to Yancarlos Diaz (BSc/MSc student), who has received a job offer from Facebook. Yancarlos plans to work in their NYC office after graduation.
(August) The Molecule Maker Lab Institute (MMLI) at UIUC has been funded as one of the new NSF AI institutes. For this project, the dprl will be working on extraction, search, and knowledge graph creation in the chemical synthesis literature. An exciting application is using AI to develop better methods for creating solar cells. There were additional news items posted by RIT online, on the local Rochester television and radio, and Chuck Schumer's office sent out a press release (US Senator for New York State).
(August) Congratulations to Mahshad Mahdavi, who successfully defended her dissertation in Imaging Science (title: Query-Driven Global Graph Attention Model for Visual Parsing: Recognizing Handwritten and Typeset Math Formulas)
(August) Congratulations to Gavin Nishizawa, who successfully defended his Master's thesis in Computer Science (title: Visual Structure Editing of Math Formulas)
(August) We welcome Ayush Shah to the project. Ayush is a new RIT PhD student who will work on formula recognition and retrieval.
(June) RIT has released a news story about MathDeck and MathSeer (link).
(May) The lab welcomes Alex Keller, who is a new PhD student joining the lab this summer. Alex's research will focus on formula indexing and retrieval.
(Apr) The first version of the MathDeck search interface has been released! We presented a demonstration of the system at ECIR 2020 (video link). The system is online at mathdeck.cs.rit.edu. Congratulations to Gavin Nishizawa, Yancarlos Diaz, Abishai Dmello, and Jennifer Liu on this important milestone! (RIT 'newsmaker' announcement)
(Apr) Congratulations to Mahshad Mahdavi, who has accepted a research position with Apple in Cupertino. She will join Apple in mid-August, after finishing her PhD dissertation.
(Apr) Congratulations to Yancarlos Diaz, who received the Outstanding 5th-year scholarship from the computer science department. Yancarlos is a BSc/MSc student.
(Apr) The ARQMath task was presented during the CLEF 2020 session at ECIR 2020. Registration closes Apr. 26 - please sign up if you wish to participate ASAP. Our thanks in particular to Behrooz Mansouri, who has been doing most of the hard work with creating the collection and evaluation tools.
(Dec) We will have three papers at ECIR 2020: one full paper on accelerated formula search, a demo paper on our new MathDeck interface, and a short paper on the ARQMath lab at CLEF 2020. Congratulations to everyone on the project who contributed to these papers (details available through the Publications link).
(Dec) The lab welcomes Jessica Diehl, an MSc Computer Science studnet who will be working on formula indexing in Spring 2020. We are also happy to announce that Yancarlos Diaz will be rejoining us to work on the MathDeck interface for Spring 2020 as well.
(Dec)The dprl is looking for a new PhD student to start in Fall 2020, to work on formula extraction, recognition, and indexing in large document collections. Details may be found online here. Please contact Prof. Zanibbi if you are interested (firstname.lastname@example.org).
(Nov) The ARQMath lab has begun! Please visit the lab web page online, and join the ARQMath forum if you are interested in the lab discussions.
(Oct) Congratulations to Abishai Dmello, who has successfully defended his thesis, Representing Mathematical Concepts Associated with Formulas using Math Entity Cards.
(Oct) A number of sotware projects from the MathSeer project are now available online (formula detector (ScanSSD), formula recognition (LPGA), Tangent-CFT formula retrieval model) along with demos for the MathSeer interface. Code is available through the dprl software page.
(Sep) Prof. Zanibbi gave a talk about the MathSeer project at RMIT in Melbourne, Australia.
(Aug) Congratulations to Parag Mali, who has successfully defended his MSc thesis, Scanning Single Shot Detector for Math in Document Images.
(Aug) Puneeth Kukkadapu has accepted a new job at ACV Auctions, where he will be doing work on handwriting recognition.
(Aug) The lab welcomes Jennifer Liu. Jennifer is a BSc student who will be working as a full-time research programmer in Fall 2019.
(June) Behrooz Mansouri's paper on applying text embeddings to formula retrieval has been accepted for publication at ICTIR 2019. The techinque produces much stronger isolated formula search results for partially similar formulas than previous systems, and state-of-the-art results when combined with Wei Zhong's Approach0 path-based formula search engine.
(June) Mahshad Mahdavi's ICDAR 2019paper on math formula recognition has been accepted for Oral Presentation at the conference in Sydney this September. Mahshad will also be presenting a paper at the GREC workshop on the new evaluation model used for formula recognition in CROHME 2019.
(May) The lab welcomes Thomas Lazore, who will be working as an intern in the lab during Summer 2019.
(May) Congratulations to Mahshad Mahdavi and Kwon-Young Choi, who have both had papers accepted at ICDAR 2019! Mahshad's paper presents a new CNN-based model for formula recognition, while Kwon-Young's paper presents a different CNN model for detecting accidentals in images of old piano scores.
(April) Congratulations to Wei Zhong, who received the Best Applications Paper Award at ECIR 2019! His paper. "Structural Similarity Search for Formulas using Leaf-Root Paths in Operator Subtrees" can be found here.
(April) Congratulations to Puneeth Kukkadapu, who has received the TSO Logic Master's Student Scholarship from the Department of Computer Science.
(April) The SymbolScraper Apache PDFBox extension for obtaining precise symbol locations and identities has been released!
(Mar) Congratulations to Behrooz Mansouri, whose paper "Toward Math-Enabled Digital Libraries: Characterizing Searches for Mathematical Concepts" has been accepted for publication at JCDL 2019, which will be held at the University of Illinois Urbana-Champaign. The paper presents the first search log analysis of math search behaviors in a general-purpose search engine (Parsijoo, from Iran).
(Mar) Congratulations to Parag Mali, who has secured a full-time Software Engineer position with the Microsoft AI and Ink group in Seattle!
(Jan) The CROHME + TFD 2019 handwritten formula recognition and typeset formula detection competition is underway. Mahshad Mahdavi is the lead organizer. Please consider participating in the competition!
(Jan) Congratulations to Mahshad Mahdavi, who has secured a Machine Learning summer internship with Apple in Cupertino!
(Jan) The lab welcomes two new research assistants, Gavin Nishizawa and Yancarlos Diaz. They are working on developing a new math-aware search interface for the MathSeer project.
(Dec) Wei Zhong and dprl alumni Kenny Davila have both had papers on math formula search accepted for publication at the European Conference for Information Retrieval (ECIR), one of the leading IR venues. The conference is being held in Cologne, Germany this coming April.
(Oct) The DPRL PhD students (Thomas Choi, Mahshad Mahdavi, Behrooz Mansouri, and Wei Zhong) presented their research in the poster seesion at the AI@GCCIS event held on Friday, Oct. 19th. It was a good event, celebrating AI research in the Computing College at RIT.
(Oct) In mid-October, Thomas Choi and Prof. Zanibbi visited the Distributed Digital Music Archives and Libraries Lab at McGill University in Montréal. Thomas presented his ongoing work in Optical Music Recognition (OMR). Our sincerest thanks to Dr. Fujinaga and his students for an informative and fun visit.
(Oct)Mahshad Madhavi presented a poster on recognizing formulas in images, and Prof. Zanibbi gave an invited talk on formula search in videos and lecture notes at the Western New York Image Processing Workshop (WNYIP) held at RIT in early October.
(Sept)Kenny Davila's Best Paper award at ICFHR 2018 has been announced on the CS web pages: link.
(Sept) A new version of the AccessMath lecture video indexing code and video data sets are available.
(Aug) The DPRL welcomes Abishai Dmello to the lab. Abishai will be completing an MSc thesis on math-aware search.
(Aug)Kenny Davila and Prof. Zanibbi received the Best Paper award at ICFHR 2018.
(July) Prof. Zanibbi is Co-Chairing ICFHR 2018 in early August, the leading international conference on handwriting recognition and applications.
(July) Kenny Davila'sTangent-V system is now available for download. This is a new system for visual search of math in lecture notes and lecture videos, which will be presented at ICFHR 2018 in Niagara Falls in early August.
(June) The lab welcomes Kwon-Young Choi, a PhD student from the University of Rennes, France. Kwon-Young's PhD is being co-advised by Bertrand Couasnon, Yann Ricquebourg, and Prof. Zanibbi. Kwon-Young is visiting from June through November of 2018.
(May) Congratulations to Wei Zhong, who successfully passed his Research Potential Assessment (the 'comprehensive' in our PhD program) after writing and defending a nice paper on an improved path-based model for formula retrieval using operator trees.
(May)Kenny Davila and Prof. Zanibbi's paper, "Visual Search Engine for Handwritten and Typeset Math in Lecture Videos and LaTeX Notes" has been accepted for Oral presentation at ICFHR 2018. The conference is being held in Niagara Falls in August.
(May) The conference program and registration for ICFHR 2018 are now online. ICFHR is the leading conference on handwriting recognition and related applications. Prof. Zanibbi is a Co-Chair for the conference this year.
(May)Kenny Davila came back to RIT to graduate. It was a very nice time to celebrate his excellent work during his PhD, and to meet his family, including his mother, father, and brother who flew in from Honduras (pictures).
(May) Congratulations to Ritvik Joshi, who has secured a full-time position at Petuum Corporation. They are a start-up in Pittsburgh, initiated by some faculty and students at Carnegie Mellon University. They are specializing in providing scalable machine learning as a service.
(May) Prof. Zanibbi has been promoted to Full Professor. My thanks to the students in the DPRL, without whom this would not have happened!
(Apr.) The lab welcomes Behrooz Mansouri, who will be joining the lab as a PhD student in Fall 2018. Behrooz will be studying math-aware search engines, with an emphasis on using text-math embeddings to improve search in technical documents.
(Apr.)Supriya Godge and Aditya Murthy will present their MS Project work in a poster session on Thursday, May 3rd from 1-3pm in the GCCIS atrium. They have been doing work on converting LaTeX output to structure trees for neural math image parsers, and improving a system for text detection in natural scenes.
(Mar.) Our lab picture has been updated (see the Members Page).
(Mar.) Congratulations to Parag Mali, who will be interning with a pattern recognition group at Apple in Cupertino this summer.
(Feb.) The lab welcomes Xuechun Wu, who will be working as a visual designer for the MathSeer project.
(Feb.) Congratluations to Puneeth Kukkadapu, who will also be interning with a pattern recognition group at Apple this summer!
(Jan.) Prof. Zanibbi will be giving a lightning talk about the lab's work in math-aware search on Feb. 15th in the session from 12:30-2pm in the Golisano Auditorium (facebook page is here).
(Jan.)The lab is happy to welcome new members starting in Spring 2018:
Puneeth Kukkadapu, MSc Student (Computer Science) - working on math-aware search interfaces;
Parag Mali, MSc Student (Computer Science) - working on content extraction and search in lecture videos; Aditya Murthy, MSc Student (Computer Science) - working on text detection in natural scenes.
(Dec.) Congratulations to Xuan Huang, who has secured an internship with Facebook in summer 2018.
(Dec.) In December, the National Science Foundation (USA) awarded a grant to Prof. Zanibbi, Dr. Anurag Agarwal (RIT), Dr. Douglas Oard (Univ. Maryland, College Park), and Dr. Lee Giles (Penn State) in support of research into exploiting text-math relationships in the design of math-aware search engines, and integration with the CiteSeerX technical paper database.
(Nov.) Kenny Davila's work on extracting whiteboard contents from lecture videos is being presented as a poster at ICDAR 2017. Source code and data from the work are available online here.
(Nov) The lab's work was featured in a meeting with the RIT Board of Trustees. Mahshad Madhavi and Wei Zhong represented the lab. We thank Nicholas Paulus for preparing the poster used in the session.
(Sep) The lab welcomes Mahshad Mahdavi, a new PhD student in the Imaging Science program. Mahshad will be working on machine learning techniques for extracting and recognizing formulas in documents.
(Sep) Recent DPRL alumnus Lakshmi Ravi (MSc, 2017) has secured a position as a Data Scientist on Amazon's Alexa Machine Learning group in Boston.
(Aug) PhD student Thomas Choi (at INSA Rennes (France), co-advised by Prof. Zanibbi) had his paper on bootstrapping small samples for recognizing accidentals in music notation accepted for publication at GREC 2017. There are many talks on Optical Music Recognition (OMR) planned for the workshop so we expect this to be an exciting GREC!
(Aug) We are very happy to introduce the new members of the DPRL in Fall 2017:
Rahul Dashora (MSc student) will work on improving the lab's Hierarchical Contextual Parsing (HCP) technique for parsing mathematical notation in PDFs, images, and handwriting.
Xuan Huan (MSc student) will work on creating a new web-based system for formula entry and search (based on the lab's 'min' system)
Ritvik Joshi (MSc student) will work on extracting formulas from PDF documents.
Wei Zhong (PhD student) is working on math-aware search engines.
(Aug) Kenny Davila has accepted a post-doctoral fellowship with the well-known CEDAR/CUBS lab at the University of Buffalo. Among the lab's many other accomplishments, CEDAR/CUBS contributed to the first automated mail-sorting system for the US Postal Service.
(July) The Tangent-S formula retrieval system has been released, and is available as open source. The changes for this version were made by Kenny Davila.
(July) Kardo Aziz has successfully defended his MSc thesis, titled Better Text Detection through Improved k-means-based Feature Learning. In his thesis, Kardo proposes a technique called Visual Similarity Sampling (VSS), which selects training samples using the average similarity of image patches within and between text and non-text classes. He has found evidence that this can improve results over pure uniform sampling when the sample size is fixed, by exploiting representative and discriminative patch characteristics when constructing a training set.
(July) Kenny Davila has successfully defended his PhD dissertation, titled Symbolic and Visual Retrieval of Mathematical Notation using Formula Graph Symbol Pair Matching and Structural Alignment. Kenny's dissertation was co-advised by Prof. Zanibbi and Dr. Stephanie Ludi (Univ. North Texas). Kenny created a formula search engine capable of using visual structure, semantics, or just the symbol layout of formulae. From this, he is able to use this to do cross-modal searches to locate formulae in lecture videos from rendered LaTeX formulae. He also created a system that can be used to search for formula, and then jump to a video frame where selected ink from the whiteboard is first drawn (YouTube demo).
(May) Congratulations to Lakshmi Ravi, who received 2nd place for her MSc Project Poster, titled "Parsing Handwritten Math Formulas." Lakshmi will soon be starting work in the Alexa group at Amazon in Seattle. She will be doing work in Natural Language Processing (NLP).
(May) Prof. Zanibbi received the Golisano Computing College Outstanding Scholar Award for a strong track record of research and scholarship that is "integral to, and not separated from, all aspects of a student's educational experience at RIT." Thanks from Prof. Zanibbi to all dprl students from the last ten years who made this possible. This award primarily reflects your hard work and accomplishments - thank you.
(Apr.) Kenny Davila will be giving a public talk at RIT about his research on math formula retrieval in documents and videos Monday, April 17th at noon in the Bamboo Rooms (Campus Center 003), as part of the Move 78 Seminar Series.
(Apr.) The lab gives a warm welcome to Wei Zhong, who will be joining the DPRL as a PhD student in Fall 2017. Wei will be doing work in math-aware search engines. For a glance of his current work in this area, you can try his Approach0 system for searching Math StackExchange posts here.
(Mar.) Le Duc Anh from the Nakagawa Lab in Japan visited the lab for a week in early March, to talk with us about his work in handwritten math recognition. His picture can be found under the "Members" link. Anh recently defended his PhD, and will begin working on Computer Vision and Machine Learning at a company in Tokyo next month.
(Feb.) Michael Condon has secured a job working as a Pattern Recognition Software Engineer at Apple in Cupertino, CA. He is joining a team that worked on handwriting recognition for the Apple Watch.
(Jan.) The web page for ICFHR 2018 is now online, including a call for papers. Please consider submitting your work to the conference, and joining us in Niagara Falls!
(Aug) We welcome Lakshmi Ravi,Michael Condon, Kardo Aziz and Chinmay Jain to the lab (all are CS MSc students). Lakshmi, Michael and Chinmay will be doing work on recognizing handwritten and typeset mathematical notation, while Kardo will be doing work on detecting text in natural scenes for both English and Arabic.
(July) Lei Hu and Prof. Zanibbi have had two papers on recognizing handwritten math accepted for publication at ICFHR, being held in Shenzhen, China this October. These papers summarize Lei's doctoral dissertation research.
(July) The CROHME 2016 handwritten math recognition competition has ended! Congratulations to MyScript and Wiris for obtaining the highest recognition rates. Details of the competition will appear in an upcoming paper at ICFHR.
(June) The new Tangent 0.3.1 formula search engine has been released (see Software link, above). This new version was a joint RIT-Univ. Waterloo collaboration, created by K. Davila, A. Kane, F.Wm. Tompa, and Prof. Zanibbi.
(May) The NTCIR-12 MathIR Task Overview paper is now available. The paper describes the math search competitions held as part of the upcoming NTCIR-12 conference being held in Tokyo.
(May) Congratulations to Yi Huang, who has accepted a position at BOSE corporation in Boston.
(May) Congratulations to Kenny Davila who has successfully defended his PhD dissertation proposal entitled "Appearance-Based Retrieval of Mathematical Notation in Documents and Lecture Videos". The proposal builds upon Kenny's earlier research, including his recently accepted SIGIR 2016 paper describing modifications to the Tangent formula search engine.
(Apr.) Congratulations to Dr. Siyu Zhu, who has successfully defended his doctoral dissertation entitled "Text Detection for Natural Scenes and Technical Diagrams with Convolutional Features and Cascaded Classification." Part of Siyu's doctoral research was recently accepted for publication at CVPR 2016. Siyu's innovative, but relatively simple text detection system obtained the highest rates for the standard ICDAR 2015 Focused Scene Text benchmark at the time of publication.
(Apr.) Congratulations to Dr. Lei Hu, who has sucessfully defended
his doctoral dissertation entitled "Features and Algorithms for Visual Parsing of Handwritten Mathematical Expressions." His dissertation presents new visual features and parsing algorithms (including parser ensembles) for recognizing handwritten mathematical notation, which illustrate that competitive recognition results may be obtained using very minimal language models - existing state-of-the-art approaches use expression grammars, which are defined and refined manually.
(Apr.) Congratulations to Kenny Davila, who has received a travel award from the ACM to attend SIGIR 2016 in Pisa, Italy.
(Apr.) Source code for Siyu Zhu's text detector for natural scenes is available from the Software link above.
(Mar.) Congratulations to Kenny Davila, who co-authored a paper on scalable formula search entitled Multi-Stage Math Formula Search: Using Appearance-Based Similarity Metrics at Scale, which was accepted for publication at ACM SIGIR 2016 being held in Pisa, Italy. Kenny's co-authors include Frank Tompa and Andrew Kane from Univ. Waterloo, and Prof. Zanibbi. SIGIR is the leading international conference on Information Retrieval.
(Mar.) Congratulations to Siyu Zhu, whose paper "A Text Detection System for Natural Scenes with Convolutional Feature Learning and Cascaded Classification" was accepted for publication at CVPR, the leading international computer vision conference. Siyu obtained state-of-the-art text detection results for the ICDAR 2015 Robust Reading Competition (Focused Scene Text Localization task).
(Jan.) We welcome Yi Huang to the lab. Yi will create a web-based evaluation framework for handwritten math recognition systems, and look to extend this to support other recognition tasks (e.g. from computer vision and/or acoustics).