Biologically-Inspired Intelligent Systems Research Final

Biologically-Inspired IS Research Project

Due dates: Papers/Projects selected March 30, Final report May 8
Presentations: (May 7)

For this work, you will work in teams of 3 students (which will be the same for paper presentations). The goal of this assignment is for your group to conduct research in the area of nature-inspired optimization. This project can be approached in one of three ways:

Reproducibility Study: You can select a paper (or related set of papers), strongly related to the topics covered in this class, and implement the models/algorithms yourselves and reproduce the results of the target paper(s). Note that you are also required to try out your implementation on another (perhaps related) dataset that the paper did not use itself.
Application Study: You can apply several of the various algorithms (and/or in your textbook or in related papers) covered in this course to an interesting (optimization) problem of your choice, meaning that you will need obtain a dataset for your problem. Note this will require careful specification of the problem/task.
Fundamental Work: You can design your own metaheuristic algorithm (or novel extension, variant, or hybrid of one of those we have studied) and evaluate it on several benchmark functions in the book and/or any real-world datasets/cost functions. Note that you will have to evaluate it rigorously to illustrate some form of generality or apply it to an actual dataset/problem to show its value on a specific problem type. You will also need to compare it to at least a few of the other metaheuristics discussed in class (although you can also report find prior results reported on an actual dataset if you use an actual dataset).

Paper selection

Your papers should come from reputable sources (conferences or journals) related to the topic at hand. Your topic should be related to the course content. You can use the library database to find papers - look in the ACM Digital Library or IEEE Explore databases for best results. Another place to consider is the end of each chapter in the textbook, where you can find the references/bibliography. For those papers, you might also use Google Scholar or CiteSeerX to find forward citations (i.e. those papers that cite the mentioned paper) which will be newer and may be more interesting. Depending on which type of project you choose to do, the papers you link me to will be different. For a reproduciblity study, this is simple -- I expect the paper(s) you will be implementing/reproducing. For an application study, I expect at least 1-2 papers that describe the problem you want to approach and/or related work. For a fundamental work project, I expect at least 2 papers that describe work related/relevant to your idea(s).

Deliverable: An email to me and the grader by February 25 (11:59pm), stating the members of your group and the type of project you will conduct. You will also need to write a brief 1-2 page PDF proposal document submitted to MyCourses (see below) that also contains relevant papers (citations with links only please, do not include the papers themselves). In case your papers are unacceptable for any reason, you will be given some suggestions and required to resubmit - without penalty if done within two days. Failure to complete this portion of the assignment on time will result in a 15% deduction to your final grade. Presentation order (for the last several weeks) will be generated randomly after February 25, unless a very strong scheduling conflict occurs (wherein you must inform me and the grader well in advance of said potential conflict).

Proposal PDF

You will write me a brief, concise document (PDF) describing to me the following:

a rough timeline / basic plan of what you will do with the time alloted
an overview / description of the overall study to be conducted, e.g., potential experimental design, data to be used, etc.
motivation - why should we care? Why is this worth pursuing or interesting to the class? (relevance to topic/course)
references/relevant paper citations

Deliverable: Proposal submitted to a MyCourses dropbox by February 25, 11:59pm.

Project Materials/Code

You will need to write code for whatever you need to implement in Python. We will expect you to become familiar or already are familiar with version control software, particularly creating yourself a free GitLab or Github account (make sure you have the ability to create a private repo). Your team is expected to implement nature-inspired metaheuristic(s) itself and the software design and development of its key elements must be discussed in your presentation and final report. You will be required to write a basic README describing how one should use your code to execute and reproduce your experimental results. This might mean you will want to write some basic Bash shell scripts to wrap up functionality if this will make it easier to use your code. We will need to be able to execute your code and it will be expected to compile/run correctly. So be sure to write good, clean code that you or others could use to experimentally explore your work/study further and specifically be able to reproduce the results you obtained if they had the (computing) resources you used. More details are to come.

Deliverable: A zip file containing the README and all necessary, organized source code to be submitted to MyCourses dropbox by May 8, 11:59pm.

Final Presentation & Writeup

You need to make sure your talk/presentation adheres to the Heilmeier format for (technical) research proposals, answering as many of the questions found here as relevant. Note that not every single question will be relevant to your talk, so adapt to your needs as necessary. Note that we will be checking for at least the following: 1) background/clear problem description, 2) prior/related work/methods, 3) approach/project design/steps, 4) experimental design, 5) experimental results/comparisons, 6) insights/observations/lessons learned, 7) limitations/future work/next steps.
Your slides can be submitted in PPT or PDF form (if you're using Beamer to create your slides, just make sure you compile them to PDF before final submission to MyCourses later) and should be submitted before 1:29pm May 7, on MyCourses. You will give your team talk on your project and results May 7 in the RIT-scheduled block of 1:30-4pm EST; label your slide deck file according to the following naming convention:
"[lastname_userID]-[lastname_userID]-[lastname_userID]_finaltalk.pptx" (or ".pdf").

Deliverable: A (compiled) PDF copy of your (typeset) final report and your presentation slides. These will be submitted to MyCourses dropbox by May 8, 11:59am (noon of that day). You will be expected to use LaTex to write your final report and use NeurIPS/NIPS format (using the right template is part of the grade!). The basic template and style files have been provided to here on this page for you to readily use on something like Overleaf (which you can create a free account on, though you do not have to use Overleaf if you have a different tool/site you prefer to use for crafting LaTex). Your paper must at least answer (thoughtfully) the questions that your presentation/talk must also try to answer (Heilmeier format). However, its general structure should approximately follow that of what you would submit to a conference for publication. In essence, this means your writeup should have (at least!) sections with content organized as follows:

Introduction -- contains motivation, background
Related work -- literature review, describes related work or prior work
Problem description -- formal description/presentation of the problem to be solved, data
Proposed solution -- present the method(s) you propose or reproduce clearly/formally, might describe baselines here as relevant
Experimental results/design -- benchmarks/baseline methods you will compare to, experiments you will conduct (things you will test for), data you will use
Discussion -- includes insights, lessons learned, observations based on experiments conducted in prior section
Conclusion -- should include future work

You will hand in any slides that you use to a MyCourses dropbox. The presentation will be graded on content (i.e. appropriate level for your fellow students), materials as appropriate, and presentation style (coherence, clarity, ability to answer questions).
Peer/Team Evaluation: you will be required to submit a bit of text outlining your team contributions and evaluating your team performance. In this bit of text, you must answer 2 questions: 1) what did you specifically focus on / contribute to the project (code-wise, paper-wise, etc.), and 2) how would you describe and evaluate your team performance qualitatively (in a few sentences)? This is where you can list any possible issues or concerns and/or strengths/positives. Due on MyCourses May 8, 11:59pm.

Grading

Proposal: 10%
Executable code base: 25%
Final report: 35%
Final presentation: 30%