Demo Day Recap
Deep Learning Applications Demo Day Spring 2024
Demo Day for the Spring 2024 “batch” of AI engineers in Deep Learning Applications (spring core course for the Duke Artificial Intelligence MEng) was yesterday and it was a smashing success. We were joined by guests from all over across various industries to see demos of the final projects.
Projects were spread across seven cores: Gaming, Health & Wellness, FinTech, Education, Public Good, Helpful Tools, and Entertainment. You can get a visualization of the breadth of projects in the snapshot of all course demos, below:
In the Gaming Core, we had mirrormate.ai, where Abhishek Murthy demonstrated cloning your chess opponent with AI. Not only was the frontend slick, but the AI generated “Move Analysis” offers impressive insights. Nick Strauch shared AI Pictionary, where you could play pictionary and learn more about explainable AI at the same time! Amazingly, Nick curated all of his data with another app that he built. With over 25k unique cards and over 3k rules, the game “Magic the Gathering” is incredibly complex to judge; Leslie Dees brings us an AI solution, combining recently released Llama 3 and a carefully crafted RAG pipeline. A major constraint of computer vision gaming systems is that it must operate in a well-lit environment. Dominique Buford tackled this with a two pronged approach: first, she fine tuned a pose estimation model on images in dimly lit environments. Next, she brought light into dark images with a GAN approach.
The Health and Wellness Core featured Katie Greed demonstrating her application for tracking Parkinson’s Disease progression through three modalities: stand to sit transitions, audio, and handwriting analysis. Katie may win the award for most ambitious - while most projects focus on modeling one type of data, Katie did three! Protein researchers are in need of better tools to explore protein characteristics and function. Daniel Medina created a comprehensive tool for researchers to explore similar proteins by structure and function in addition to visualizing the protein sequence embedding space. Unsurprisingly, it is wrapped in a beautiful user interface. Samyukta Palle sets out to tackle disparities in women’s health with her Women Health Chatbot, where users can ask questions about their health and find recipes and fitness routines that align with their cycle. Pathology is one area that machine learning could help augment caregivers. One student in the class shared impressive results from his chondrosarcoma tumor necrosis classification model. His fine-tuned ResNet18 model could accurately distinguish necrosis, bone, dedifferentiation, chondroid/cartilage, and normal soft tissue with >95% precision.
In our FinTech Core, Mrinoy Banerjee introduced us to Bank-BERT, the model he built to demonstrate the ability to have a local model on your laptop that is fine-tuned on your personal financial data. In addition to topic modeling, he created nice visualizations of personal finance data to help you take control of your finances.
Our Education Core brought us ZotMind from Hongxuan (Leo) Li, where you can create a chatbot with your Zotero papers (where was this when I was doing my PhD??). Jingwei Li William introduced us to AI Career Consultant, where you can upload your resume and a job description and it will identify missing skills and suggest Coursera courses to learn the skills. Aryan Poonacha shared his pipeline for building a chatbot for any Wiki and demonstrated it with his favorite video game series. LLMs are notoriously bad at math word problems. Shuaiming Jing fine tuned an LLM on math word problems and created a chatbot that outperforms regular ChatGPT with or without chain of thought!
In our Public Good Core, Sriramteja Veerisetti shared Habitat Hero, his app that detects wildlife animals in an uploaded photo and gives you comprehensive information about the animal(s), including endangered status and conservation efforts.
The Helpful Tools Core included a desktop application developed by Rishabh Shah for sign language recognition. An incredibly difficult problem, Rishabh demonstrated his two pronged approach to the problem in a nice user interface. Thomas McGuire created DeerAI, a tool to effectively age and manage deer. Tommy wins the award for most out of the box thinking with this one - a really interesting application that had incredibly impressive results. The last project in the helpful tools core, ChatChat, was created by Zhihan Xu for local chat with your PDF or text file.
Our Entertainment Core deserves a standing ovation - our largest core, and some of the most interesting projects all year! First up, Joao Cardosa created a Music Journal integrated into popular music streaming platforms (he demonstrated integration with Spotify), where you can turn your day into a playlist. I got a sneak preview of this one earlier this week and I can confirm, it does a great job creating a playlist that matches the mood of your day. Did you know the average time to select a movie is 19 minutes?! Nicholas Conterno solves this problem with FlickFinder, a movie recommendation platform that has an easy to use, clean user interface. Zach Xing shared his adorable app “Imaginative Tales”, an AI storyteller for kids that combines LLMs and Stable Diffusion LoRAs to create children stories with cohesive imagery. Mike Keohane demonstrated JamSesh, a tool for musicians to use to generate chords, scales, and music to facilitate improvisation via a natural language prompt for key and style. He explored the question “can one use LLMs to understand and generate music?” Based on the examples shared in the demo, the answer is YES! Haoyang Ma created LOL LoRAs, a tool for creating comics in the style of League of Legends, facilitated by LoRAs. If you want to relive your childhood, here is your chance: Jared Bailey presented his app to find Waldo and his Friends from a photo of a book page snapped by your phone. Kids these days really do have it easier! The data labeling effort for this project was impressive - Jared found and labeled Waldo (and friends!) in the entire “Where’s Waldo?” collection of books. Last, but certainly not least, Suneel Nadipalli introduced us to Nexus AI, where you can convert your favorite book, movie, or story into a choose-your-own-adventure! (Didn’t like how that book or movie ended? Well here’s your opportunity to rewrite history!)
If any of you are hiring, I think there are a couple of folks from this impressive group who have not yet accepted an offer for summer internships and a few of our graduates are not yet committed for full time roles.
Well, that’s a wrap on Deep Learning Applications Spring 2024. Thank you to the class - your work was fantastic. To everyone else, stay tuned for our Summer 2024 Demo Day in August.
