News

What we've been up to lately, and previously.

News: 2023

  • (Nov, 2023) A paper describing the dprl's ChemScraper parser for molecular diagrams in PDF drawing instructions ('born-digital') is available on arXiv here. The system can also generate annotated training data for visual parsers that recognize raster images (i.e., pixel-based, such as PNG). A link to associated code is provided in a footnote. Congratulations to Ayush Kumar Shah, Bryan Manrique Amador, Abhisek Dey, Ming Creekmore, and Blake Ocampo (PhD candidate, UIUC Dept. of Chemistry) on a job well done.
  • (Sept, 2023) Congratulations to former dprl PhD student Wei Zhong, who successfully defended his dissertation on math-aware search at the University of Waterloo (advisor: Jimmy Lin). Wei had to switch schools and countries due to visa restrictions during COVID. This past summer summer he also worked as a research intern at Microsoft research.
  • (Aug 24, 2023) I gave the keynote talk at GREC 2023 in San Jose, which was held as part of ICDAR 2023.  The talk was an overview of MathDeck and related work in math formula recognition and search. My thanks to everyone who attended, it was a very good experience!
  • (July 11, 2023) The lab's poster and longer video demonstration for our MathDeck demo at SIGIR 2023 are now available. Live demo page link is here.
  • (June 27, 2023) The first initial release of the ChemScraper tool developed by the dprl, Denmark Lab, and NCSA has been released.  There are plans to regularly update the tool moving forward; look for updates in the coming months.
  • (June 27, 2023) Abhisek Dey and Ming Creekmore presented a poster on our lab's work on extracting and recognizing chemical diagrams for the MMLI ChemScraper system for an NSF Site visit with the MMLI AI Center at UIUC (in Champaign, Illinois). Ayush Kumar Shah and Bryan Amador, who also contributed to the system participated in the meeting as well.
  • (April 19, 2023) Ayush Kumar Shah published a paper on an improved math formula parsing model using line-of-sight graphs (the Line-of-sight with Graph Attention Parser (LGAP), previously named QD-GGA). The paper will be presented at ICDAR 2023.
  • (April 1, 2023) A demonstration paper for the MathDeck system searching text and formulas in PDF files from the ACL Anthology will appear at SIGIR 2023. Bryan Amador, Matt Langsenkamp, Abhisek Dey, and Ayush Kumar Shah created the demo. A number of past DPRL students made important contributions as part of the MathSeer project.

News: 2022

  • (Oct) Prof. Zanibbi gage a talk for the Topos Institute on Mathematical Information Retrieval on Oct. 27th. The Topos colloquium series, including a YouTube link for the talk may be found online here: https://topos.site/topos-colloquium.
  • (Aug) We welcome Bryan Manrique Amador to the lab, who will be doing PhD research in the area of Information Retrieval.
  • (Aug) Behrooz Mansouri's first paper on contextualized formula search using MathAMR was accepted at CIKM 2022.
  • (Aug) Behrooz Mansouri defended his dissertation on math-aware search on July 26th. The final version of his dissertation is available here.
  • (May) As part of an independent study in Spring 2022, JP Ramissini has created a new YOLO-based detector for math formulas and chemical diagrams (GitLab link).
  • (May) Abhisek Dey is completing an internship at the University of Illinois this summer (as part of the MMLI project).
  • (May) Ayush Kumar Shah will be completing an internship with Amazon this summer as an Applied Research Scientist intern in Computer Vision / Deep Learning.
  • (May) Shaurya Rohatgi is completing another internship at AllenAI this summer.
  • (May) Congratulations to Matt Langsenkamp, who has been admitted to the PhD program at RIT.
  • (May) Congratulations to Abhisek Dey, who successfully completed his comprehensive (Research Potential Assessment) for the RIT PhD program.
  • (March) Congratulations to Behrooz Mansouri, who will be holding a tenure-track faculty position in the Computer Science Department at the University of Southern Maine!

News: 2021


  • (Dec) We welcome JP Ramissini to the lab. JP is a BSc student who will complete his undergraduate thesis in the dprl.
  • (Dec) The Alfred P. Sloan Foundation has provided a small grant to help us complete the implementation of MathSeer within the CiteSeerX system.
  • (Nov) Congratulations to Behrooz Mansouri, who has passed his dissertation proposal defense.
  • (Oct) SymbolScraper Server (update). Our SymbolScraper tool for extracting text and graphics from PDF files has been rewritten, and made faster (including the addition of a docker container).
  • (Sept) MathSeer extraction pipeline released. This tool extracts formula locations and content in PDF documents. The pipeline is available from GitLab, and includes improved versions of SymbolScraper, ScanSSD (now, ScanSSD-XYc), and QD-GGA. The pipeline was prepared by Ayush K. Shah, Abhisek Dey, Matt Langsenkamp, and Prof. Zanibbi.
  • (Sept) The IAPR Graphics Recognition (GREC 2021) workshop and ICDAR 2021 conference are running during the first week of September. Abhisek Dey and Ayush Shah will be presenting their work on visual formula detection and the first version of the MathSeer extraction pipeline. Prof. Zanibbi is one of the program chairs for GREC.
  • (July)  ARQMath-2: Behrooz Mansouri, Doug Oard, Anurag Agarwal and Richard Zanibbi ran the second edition of ARQMath for CLEF 2021. CLEF will be held online in late September - registration is free!
  • (May) Yancarlos Diaz and Robin Avenoso have successfully defended their MSc theses! Their thesis documents are available here.
  • (April) Behrooz Mansouri's paper on the first learning-to-rank model for formula search has been accepted for publication at SIGIR 2021, and he will be participating in the SIGIR 2021 Doctoral Consortium.
  • (April) Ayush Kumar Shah and Abhisek Dey's paper on the MathSeer formula extraction and evaluation pipeline has been accepted for publication at ICDAR 2021.
  • (April) Prof. Zanibbi gave a seminar talk on the MathSeer project in the Boise State Computing PhD seminar series.
  • (March) ARQMath was presented at ECIR 2021 by Behrooz Mansouri (video).
  • (February) MathDeck will be demonstrated in an interactivity session at CHI 2021 (video). Congratulations to Yancarlos Diaz, Gavin Nishizawa, and Behrooz Mansouri, the student authors on the paper.

News: 2020

  • (Dec) The lab welcomes Matt Langsenkamp, who will be working as a research programmer in the dprl. Matt holds a CS degree from Ohio State, and previously worked as a software engineer at Cloudcheckr.
  • (Dec) The lab welcomes Abhisek Day, a new PhD student who will be working chemical information extraction and search for the MMLI project. Abhisek recently completed his MSc in Computer Engineering at RIT in the area of computer vision.
  • (Oct) There is an open PhD position to start in Fall 2021  through the MMLI project. Interested candidates should email Prof. Zanibbi (rxzvcs@rit.edu) to discuss the position.  [ position filled ]
  • (Sept) Congratulations to Robin Avenoso (BSc/MSc student), who received the Outstanding Undergraduate Scholar Award. This award celebrates the top 1% of undergraduate students who are able to maintain a high standard of academic excellence while also giving back to their community through civic or volunteer work, conducting research, or being engaged in a co-op or work in their field of study. 
  • (Sept) Congratulations to Yancarlos Diaz (BSc/MSc student), who has received a job offer from Facebook. Yancarlos plans to work in their NYC office after graduation.
  • (August) The Molecule Maker Lab Institute (MMLI) at UIUC has been funded as one of the new NSF AI institutes. For this project, the dprl will be working on extraction, search, and knowledge graph creation in the chemical synthesis literature. An exciting application is using AI to develop better methods for creating solar cells. There were additional news items posted by RIT online, on the local Rochester television and radio, and Chuck Schumer's office sent out a press release (US Senator for New York State). 
  • (August) Congratulations to Mahshad Mahdavi, who successfully defended her dissertation in Imaging Science (title: Query-Driven Global Graph Attention Model for Visual Parsing: Recognizing Handwritten and Typeset Math Formulas)
  • (August) Congratulations to Gavin Nishizawa, who successfully defended his Master's thesis in Computer Science (title: Visual Structure Editing of Math Formulas)
  • (August) We welcome Ayush Shah to the project. Ayush is a new RIT PhD student who will work on formula recognition and retrieval.
  • (June) RIT has released a news story about MathDeck and MathSeer (link).
  • (May) The lab welcomes Alex Keller, who is a new PhD student joining the lab this summer. Alex's research will focus on formula indexing and retrieval.
  • (Apr) The first version of the MathDeck search interface has been released! We presented a demonstration of the system at ECIR 2020 (video link). The system is online at mathdeck.cs.rit.edu. Congratulations to Gavin Nishizawa, Yancarlos Diaz, Abishai Dmello, and Jennifer Liu on this important milestone! (RIT 'newsmaker' announcement)
  • (Apr) Congratulations to Mahshad Mahdavi, who has accepted a research position with Apple in Cupertino. She will join Apple in mid-August, after finishing her PhD dissertation.
  • (Apr) Congratulations to Yancarlos Diaz, who received the Outstanding 5th-year scholarship from the computer science department. Yancarlos is a BSc/MSc student.
  • (Apr) Congratulations to Mahshad Mahdavi, who had her paper on a novel CNN+attention model for parsing handwritten formulas accepted at the CVPR Workshop on Text and Documents in the Deep Learning Era.
  • (Apr) Wei Zhong's ECIR 2020 talk on accelerating formula search can be found online (YouTube).
  • (Apr) The ARQMath task was presented during the CLEF 2020 session at ECIR 2020. Registration closes Apr. 26 - please sign up if you wish to participate ASAP. Our thanks in particular to Behrooz Mansouri, who has been doing most of the hard work with creating the collection and evaluation tools.

News: 2019

  • (Dec) We will have three papers at ECIR 2020: one full paper on accelerated formula search, a demo paper on our new MathDeck interface, and a short paper on the ARQMath lab at CLEF 2020. Congratulations to everyone on the project who contributed to these papers (details available through the Publications link).
  • (Dec) The lab welcomes Jessica Diehl, an MSc Computer Science studnet who will be working on formula indexing in Spring 2020. We are also happy to announce that Yancarlos Diaz will be rejoining us to work on the MathDeck interface for Spring 2020 as well.
  • (Dec) The dprl is looking for a new PhD student to start in Fall 2020, to work on formula extraction, recognition, and indexing in large document collections. Details may be found online here. Please contact Prof. Zanibbi if you are interested (rxzvcs@rit.edu).
  • (Nov) The ARQMath lab has begun! Please visit the lab web page online, and join the ARQMath forum if you are interested in the lab discussions. 
  • (Oct) Congratulations to Abishai Dmello, who has successfully defended his thesis, Representing Mathematical Concepts Associated with Formulas using Math Entity Cards.
  • (Oct) A number of sotware projects from the MathSeer project are now available online (formula detector (ScanSSD), formula recognition (LPGA), Tangent-CFT formula retrieval model) along with demos for the MathSeer interface. Code is available through the dprl software page.
  • (Oct) Prof. Zanibbi gave a talk on MathSeer at the University of Waterloo in Canada.
  • (Sep) Prof. Zanibbi gave a talk about the MathSeer project at RMIT in Melbourne, Australia.
  • (Aug) Congratulations to Parag Mali, who has successfully defended his MSc thesis, Scanning Single Shot Detector for Math in Document Images.
  • (Aug) Puneeth Kukkadapu has accepted a new job at ACV Auctions, where he will be doing work on handwriting recognition.
  • (Aug) The lab welcomes Jennifer Liu. Jennifer is a BSc student who will be working as a full-time research programmer in Fall 2019.
  • (June) Behrooz Mansouri's paper on applying text embeddings to formula retrieval has been accepted for publication at ICTIR 2019. The techinque produces much stronger isolated formula search results for partially similar formulas than previous systems, and state-of-the-art results when combined with Wei Zhong's Approach0 path-based formula search engine.
  • (June) Mahshad Mahdavi's ICDAR 2019 paper on math formula recognition has been accepted for Oral Presentation at the conference in Sydney this September. Mahshad will also be presenting a paper at the GREC workshop on the new evaluation model used for formula recognition in CROHME 2019.
  • (June) Behrooz Mansouri's JCDL paper on Characterizing Searches for Mathematical Concepts was nominated for a Best Paper award. 
  • (May) The lab welcomes Thomas Lazore, who will be working as an intern in the lab during Summer 2019.
  • (May) Congratulations to Mahshad Mahdavi and Kwon-Young Choi, who have both had papers accepted at ICDAR 2019! Mahshad's paper presents a new CNN-based model for formula recognition, while Kwon-Young's paper presents a different CNN model for detecting accidentals in images of old piano scores.
  • (April) Congratulations to Wei Zhong, who received the Best Applications Paper Award at ECIR 2019! His paper. "Structural Similarity Search for Formulas using Leaf-Root Paths in Operator Subtrees" can be found here.
  • (April) Congratulations to Puneeth Kukkadapu, who has received the TSO Logic Master's Student Scholarship from the Department of Computer Science.
  • (April) The ECIR 2019 version of Tangent-v for visual formula search has been released. Results from Kenny and Ritvik's ECIR 2019 paper are included in the package.
  • (April) The SymbolScraper Apache PDFBox extension for obtaining precise symbol locations and identities has been released!
  • (Mar) Congratulations to Behrooz Mansouri, whose paper "Toward Math-Enabled Digital Libraries: Characterizing Searches for Mathematical Concepts" has been accepted for publication at JCDL 2019, which will be held at the University of Illinois Urbana-Champaign.  The paper presents the first search log analysis of math search behaviors in a general-purpose search engine (Parsijoo, from Iran).
  • (Mar) Congratulations to Parag Mali, who has secured a full-time Software Engineer position with the Microsoft AI and Ink group in Seattle!
  • (Jan) The CROHME + TFD 2019 handwritten formula recognition and typeset formula detection competition is underway. Mahshad Mahdavi is the lead organizer. Please consider participating in the competition!
  • (Jan) Congratulations to Wei Zhong and Kenny Davila, who have both had full papers on math formula search engines accepted to ECIR 2019 in Cologne, Germany this April. Wei's paper is "Structural Similarity Search for Formulas using Leaf-Root Paths in Operator Subtrees," while Kenny's paper is "Tangent-V: Math Formula Image Search Using Line-of-Sight Graphs."
  • (Jan) Congratulations to Mahshad Mahdavi, who has secured a Machine Learning summer internship with Apple in Cupertino!
  • (Jan) The lab welcomes two new research assistants, Gavin Nishizawa and Yancarlos Diaz. They are working on developing a new math-aware search interface for the MathSeer project.

News: 2018

  • (Dec) Wei Zhong and dprl alumni Kenny Davila have both had papers on math formula search accepted for publication at the European Conference for Information Retrieval (ECIR), one of the leading IR venues. The conference is being held in Cologne, Germany this coming April.
  • (Oct) The DPRL PhD students (Thomas Choi, Mahshad Mahdavi, Behrooz Mansouri, and Wei Zhong) presented their research in the poster seesion at the AI@GCCIS event held on Friday, Oct. 19th. It was a good event, celebrating AI research in the Computing College at RIT.
  • (Oct) In mid-October, Thomas Choi and Prof. Zanibbi visited the Distributed Digital Music Archives and Libraries Lab at McGill University in Montréal. Thomas presented his ongoing work in Optical Music Recognition (OMR). Our sincerest thanks to Dr. Fujinaga and his students for an informative and fun visit.
  • (Oct) Mahshad Madhavi presented a poster on recognizing formulas in images, and Prof. Zanibbi gave an invited talk on formula search in videos and lecture notes at the Western New York Image Processing Workshop (WNYIP) held at RIT in early October.
  • (Sept) Kenny Davila's Best Paper award at ICFHR 2018 has been announced on the CS web pages: link.
  • (Sept) A new version of the AccessMath lecture video indexing code and video data sets are available.
  • (Aug) The DPRL welcomes Abishai Dmello to the lab. Abishai will be completing an MSc thesis on math-aware search.
  • (Aug) Kenny Davila and Prof. Zanibbi received the Best Paper award at ICFHR 2018.
  • (July) Prof. Zanibbi is Co-Chairing ICFHR 2018 in early August, the leading international conference on handwriting recognition and applications.
  • (July) Kenny Davila's Tangent-V system is now available for download. This is a new system for visual search of math in lecture notes and lecture videos, which will be presented at ICFHR 2018 in Niagara Falls in early August.
  • (June) The lab welcomes Kwon-Young Choi, a PhD student from the University of Rennes, France. Kwon-Young's PhD is being co-advised by Bertrand Couasnon, Yann Ricquebourg, and Prof. Zanibbi. Kwon-Young is visiting from June through November of 2018.
  • (May) Congratulations to Wei Zhong, who successfully passed his Research Potential Assessment (the 'comprehensive' in our PhD program) after writing and defending a nice paper on an improved path-based model for formula retrieval using operator trees.
  • (May) Kenny Davila and Prof. Zanibbi's paper, "Visual Search Engine for Handwritten and Typeset Math in Lecture Videos and LaTeX Notes" has been accepted for Oral presentation at ICFHR 2018. The conference is being held in Niagara Falls in August.
  • (May) The conference program and registration for ICFHR 2018 are now online. ICFHR is the leading conference on handwriting recognition and related applications. Prof. Zanibbi is a Co-Chair for the conference this year.
  • (May) Kenny Davila came back to RIT to graduate. It was a very nice time to celebrate his excellent work during his PhD, and to meet his family, including his mother, father, and brother who flew in from Honduras (pictures).
  • (May) Congratulations to Ritvik Joshi, who has secured a full-time position at Petuum Corporation. They are a start-up in Pittsburgh, initiated by some faculty and students at Carnegie Mellon University. They are specializing in providing scalable machine learning as a service.
  • (May) Prof. Zanibbi has been promoted to Full Professor. My thanks to the students in the DPRL, without whom this would not have happened!
  • (Apr.) The lab welcomes Behrooz Mansouri, who will be joining the lab as a PhD student in Fall 2018. Behrooz will be studying math-aware search engines, with an emphasis on using text-math embeddings to improve search in technical documents.
  • (Apr.) Supriya Godge and Aditya Murthy will present their MS Project work in a poster session on Thursday, May 3rd from 1-3pm in the GCCIS atrium. They have been doing work on converting LaTeX output to structure trees for neural math image parsers, and improving a system for text detection in natural scenes.
  • (Mar.) Our lab picture has been updated (see the Members Page).
  • (Mar.) Congratulations to Parag Mali, who will be interning with a pattern recognition group at Apple in Cupertino this summer.
  • (Feb.) The lab welcomes Xuechun Wu, who will be working as a visual designer for the MathSeer project.
  • (Feb.) Congratluations to Puneeth Kukkadapu, who will also be interning with a pattern recognition group at Apple this summer!
  • (Jan.) Prof. Zanibbi will be giving a lightning talk about the lab's work in math-aware search on Feb. 15th in the session from 12:30-2pm in the Golisano Auditorium (facebook page is here).
  • (Jan.)The lab is happy to welcome new members starting in Spring 2018: Puneeth Kukkadapu, MSc Student (Computer Science) - working on math-aware search interfaces; Parag Mali, MSc Student (Computer Science) - working on content extraction and search in lecture videos; Aditya Murthy, MSc Student (Computer Science) - working on text detection in natural scenes.

Earlier News

2017

  • (Dec.) Congratulations to Xuan Huang, who has secured an internship with Facebook in summer 2018.
  • (Dec.) In December, the National Science Foundation (USA) awarded a grant to Prof. Zanibbi, Dr. Anurag Agarwal (RIT), Dr. Douglas Oard (Univ. Maryland, College Park), and Dr. Lee Giles (Penn State) in support of research into exploiting text-math relationships in the design of math-aware search engines, and integration with the CiteSeerX technical paper database.
  • (Nov.) Kenny Davila's work on extracting whiteboard contents from lecture videos is being presented as a poster at ICDAR 2017. Source code and data from the work are available online here.
  • (Nov) The lab's work was featured in a meeting with the RIT Board of Trustees. Mahshad Madhavi and Wei Zhong represented the lab. We thank Nicholas Paulus for preparing the poster used in the session.
  • (Sep) The lab welcomes Mahshad Mahdavi, a new PhD student in the Imaging Science program. Mahshad will be working on machine learning techniques for extracting and recognizing formulas in documents.
  • (Sep) Recent DPRL alumnus Lakshmi Ravi (MSc, 2017) has secured a position as a Data Scientist on Amazon's Alexa Machine Learning group in Boston.
  • (Aug) PhD student Thomas Choi (at INSA Rennes (France), co-advised by Prof. Zanibbi) had his paper on bootstrapping small samples for recognizing accidentals in music notation accepted for publication at GREC 2017. There are many talks on Optical Music Recognition (OMR) planned for the workshop so we expect this to be an exciting GREC!
  • (Aug) We are very happy to introduce the new members of the DPRL in Fall 2017:
    • Rahul Dashora (MSc student) will work on improving the lab's Hierarchical Contextual Parsing (HCP) technique for parsing mathematical notation in PDFs, images, and handwriting.
    • Xuan Huan (MSc student) will work on creating a new web-based system for formula entry and search (based on the lab's 'min' system)
    • Ritvik Joshi (MSc student) will work on extracting formulas from PDF documents.
    • Wei Zhong (PhD student) is working on math-aware search engines.
  • (Aug) Kenny Davila has accepted a post-doctoral fellowship with the well-known CEDAR/CUBS lab at the University of Buffalo. Among the lab's many other accomplishments, CEDAR/CUBS contributed to the first automated mail-sorting system for the US Postal Service.
  • (July) The Tangent-S formula retrieval system has been released, and is available as open source. The changes for this version were made by Kenny Davila.
  • (July) Kardo Aziz has successfully defended his MSc thesis, titled Better Text Detection through Improved k-means-based Feature Learning. In his thesis, Kardo proposes a technique called Visual Similarity Sampling (VSS), which selects training samples using the average similarity of image patches within and between text and non-text classes. He has found evidence that this can improve results over pure uniform sampling when the sample size is fixed, by exploiting representative and discriminative patch characteristics when constructing a training set.
  • (July) Kenny Davila has successfully defended his PhD dissertation, titled Symbolic and Visual Retrieval of Mathematical Notation using Formula Graph Symbol Pair Matching and Structural Alignment. Kenny's dissertation was co-advised by Prof. Zanibbi and Dr. Stephanie Ludi (Univ. North Texas). Kenny created a formula search engine capable of using visual structure, semantics, or just the symbol layout of formulae. From this, he is able to use this to do cross-modal searches to locate formulae in lecture videos from rendered LaTeX formulae. He also created a system that can be used to search for formula, and then jump to a video frame where selected ink from the whiteboard is first drawn (YouTube demo).
  • (June) The Alfred P. Sloan Foundation has funded a proposal by Prof. Zanibbi and C. Lee Giles (Penn State) to integrate math-aware search into the CiteSeerX platform.
  • (June) Michael Condon successfully defended his MSc thesis, Applying Hierarchical Contextual Parsing with Visual Density and Geometric Features to Typeset Formula Recognition. Michael made changes to formula structure representations for punctuation and refined other techniques from Lei Hu's HCP parsing algorithm, applying these to recognizing typeset math. His parser obtains very high expression recognition rates for formulas from the infty dataset (almost 91% parsing from connected components). Source code and data used for his work will be made available in the coming weeks.
  • (June) Kenny Davila's paper Whiteboard Video Summarization via Spatio-Temporal Conflict Minimization has been accepted for publication at ICDAR 2017 (the leading document analysis and recognition conference). There is a YouTube video demonstrating his new system for summarization and navigation of lectures using whiteboard content summaries As far as we know, this is the first system that supports lecture video navigation using whiteboard contents directly.
  • (May) Congratulations to Lakshmi Ravi, who received 2nd place for her MSc Project Poster, titled "Parsing Handwritten Math Formulas." Lakshmi will soon be starting work in the Alexa group at Amazon in Seattle. She will be doing work in Natural Language Processing (NLP).
  • (May) Prof. Zanibbi received the Golisano Computing College Outstanding Scholar Award for a strong track record of research and scholarship that is "integral to, and not separated from, all aspects of a student's educational experience at RIT." Thanks from Prof. Zanibbi to all dprl students from the last ten years who made this possible. This award primarily reflects your hard work and accomplishments - thank you.
  • (Apr.) Prof. Zanibbi gave a brief presentation at the RIT GCCIS Research Showcase about the lab, dprl@10: The Document and Pattern Recognition Lab's First 10 Years. A sincere thanks and congratulations to all of the dprl students from these first 10 years!
  • (Apr.) Chinmay Jain has secured a Software Engineer position involving machine learning for customer recommendations with BBVA Compass.
  • (Apr.) Congratulations to Kenny Davila whose paper Layout and Semantics: Combining Representations for Math Formula Search" has been accepted for poster presentation at SIGIR 2017, the leading Information Retrieval conference.
  • (Apr.) Kenny Davila will be giving a public talk at RIT about his research on math formula retrieval in documents and videos Monday, April 17th at noon in the Bamboo Rooms (Campus Center 003), as part of the Move 78 Seminar Series.
  • (Apr.) The lab gives a warm welcome to Wei Zhong, who will be joining the DPRL as a PhD student in Fall 2017. Wei will be doing work in math-aware search engines. For a glance of his current work in this area, you can try his Approach0 system for searching Math StackExchange posts here.
  • (Mar.) Le Duc Anh from the Nakagawa Lab in Japan visited the lab for a week in early March, to talk with us about his work in handwritten math recognition. His picture can be found under the "Members" link. Anh recently defended his PhD, and will begin working on Computer Vision and Machine Learning at a company in Tokyo next month.
  • (Feb.) Michael Condon has secured a job working as a Pattern Recognition Software Engineer at Apple in Cupertino, CA. He is joining a team that worked on handwriting recognition for the Apple Watch.
  • (Jan.) The web page for ICFHR 2018 is now online, including a call for papers. Please consider submitting your work to the conference, and joining us in Niagara Falls!
  • (Jan.) Prof. Zanibbi will be the new Communications Officer for the IAPR Technical Committee No. 11 ('Reading Systems').

2016

  • (Dec.) Congratulations to Chinmay Jain, whose poster received 2nd place in the Best MS Poster competition! Chinmay's poster is available here.
  • (Nov.) The lab welcomes Jahongir Amirkulov to the lab. Jahongir is an undergraduate Computer Science student who will be working on the min math search interface in the spring semester.
  • (Oct.) Congratulations to Lei Hu, who received a best poster award at the international ICFHR conference for his paper, MST-Based Visual Parsing of Online Handwritten Mathematical Expressions.
  • (Oct.) The call for papers and web pages for the International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) are now online. Prof. Zanibbi is Co-Chairing the conference, which will be held August 6-10, 2018 at RIT.
  • (Oct.) Kenny Davila's work in handwriten symbol generation from examples has been cited in a UIST 2016 paper by Taranta, LaViola et al. (Video Demonstration)
  • (Aug) We welcome Lakshmi Ravi, Michael Condon, Kardo Aziz and Chinmay Jain to the lab (all are CS MSc students). Lakshmi, Michael and Chinmay will be doing work on recognizing handwritten and typeset mathematical notation, while Kardo will be doing work on detecting text in natural scenes for both English and Arabic.
  • (July) Lei Hu and Prof. Zanibbi have had two papers on recognizing handwritten math accepted for publication at ICFHR, being held in Shenzhen, China this October. These papers summarize Lei's doctoral dissertation research.
  • (July) The CROHME 2016 handwritten math recognition competition has ended! Congratulations to MyScript and Wiris for obtaining the highest recognition rates. Details of the competition will appear in an upcoming paper at ICFHR.
  • (June) The new Tangent 0.3.1 formula search engine has been released (see Software link, above). This new version was a joint RIT-Univ. Waterloo collaboration, created by K. Davila, A. Kane, F.Wm. Tompa, and Prof. Zanibbi.
  • (May) The NTCIR-12 MathIR Task Overview paper is now available. The paper describes the math search competitions held as part of the upcoming NTCIR-12 conference being held in Tokyo.
  • (May) Kenny Davila, Andrew Kane, Frank Tompa and Prof. Zanibbi's paper about the Tangent math search engine at the NTCIR-12 math retrieval competitions is now available. For the competition we improved support for wildcard matching, and explored new metrics for formula similarity.
  • (May) Congratulations to Yi Huang, who has accepted a position at BOSE corporation in Boston.
  • (May) Congratulations to Kenny Davila who has successfully defended his PhD dissertation proposal entitled "Appearance-Based Retrieval of Mathematical Notation in Documents and Lecture Videos". The proposal builds upon Kenny's earlier research, including his recently accepted SIGIR 2016 paper describing modifications to the Tangent formula search engine.
  • (Apr.) Congratulations to Dr. Siyu Zhu, who has successfully defended his doctoral dissertation entitled "Text Detection for Natural Scenes and Technical Diagrams with Convolutional Features and Cascaded Classification." Part of Siyu's doctoral research was recently accepted for publication at CVPR 2016. Siyu's innovative, but relatively simple text detection system obtained the highest rates for the standard ICDAR 2015 Focused Scene Text benchmark at the time of publication.
  • (Apr.) Congratulations to Dr. Lei Hu, who has sucessfully defended his doctoral dissertation entitled "Features and Algorithms for Visual Parsing of Handwritten Mathematical Expressions." His dissertation presents new visual features and parsing algorithms (including parser ensembles) for recognizing handwritten mathematical notation, which illustrate that competitive recognition results may be obtained using very minimal language models - existing state-of-the-art approaches use expression grammars, which are defined and refined manually.
  • (Apr.) Congratulations to Kenny Davila, who has received a travel award from the ACM to attend SIGIR 2016 in Pisa, Italy.
  • (Apr.) Source code for Siyu Zhu's text detector for natural scenes is available from the Software link above.
  • (Apr.) Congratulations (again) to Kenny Davila, who has been selected to participate in the Doctoral Consortium at SIGIR 2016.
  • (Mar.) Congratulations to Kenny Davila, who co-authored a paper on scalable formula search entitled Multi-Stage Math Formula Search: Using Appearance-Based Similarity Metrics at Scale, which was accepted for publication at ACM SIGIR 2016 being held in Pisa, Italy. Kenny's co-authors include Frank Tompa and Andrew Kane from Univ. Waterloo, and Prof. Zanibbi. SIGIR is the leading international conference on Information Retrieval.
  • (Mar.) Congratulations to Siyu Zhu, whose paper "A Text Detection System for Natural Scenes with Convolutional Feature Learning and Cascaded Classification" was accepted for publication at CVPR, the leading international computer vision conference. Siyu obtained state-of-the-art text detection results for the ICDAR 2015 Robust Reading Competition (Focused Scene Text Localization task).
  • (Jan.) We welcome Yi Huang to the lab. Yi will create a web-based evaluation framework for handwritten math recognition systems, and look to extend this to support other recognition tasks (e.g. from computer vision and/or acoustics).
Address

Room 70-3500, GCCIS
Dept. Computer Science
Rochester Inst. Technology
Rochester, NY, 14623-5608
USA

Contacts

Email: rxzvcs@rit.edu
Phone: +1 (585) 475-4536
Fax: +1 (585) 475-4935