# Nirmal Vegesna > VLM Research | Decentralized AI | MS CS @ Columbia Location: San Francisco, California, United States Profile: https://flows.cv/nirmalvegesna Current focus is on Vision models to unlock AI powered media. Our team at Mission Technologies thrives on pushing the boundaries of computer vision, where my role as a Research Engineer blends my Columbia University education in Machine Learning with practical applications in automation and navigation. With a passion for both theoretical research and real-world problem-solving, we're shaping the future of technology. Previously at Columbia Business School, our collaboration led to pioneering a GPT-3 based recommendation system, marrying AI expertise with creative solutions. My journey in full-stack development and bias detection at The New School was also a testament to my drive towards ethical AI development, using skills in Python and JavaScript to foster innovation and integrity in technology. Projects Retrieval Augmented Generation Leveraged GPT 3.5 Turbo to perform synthetic data generation of Real Estate Listings and store listings using ChromaDB in a vector database. Implemented RAG with semantic search to tailor the listings with buyer specific preferences. AI based Photo editing Developed an app that can edit pictures by first generating a segmentation mask using SAM model from Meta. User specific text prompts are used to perform the infilling of pictures using a Stable Diffusion Model. Self-driving cars workflow Designed a PID controller in C++ to perform vehicle trajectory tracking and integrated with CARLA simulator. Optimized the steering angle of a self-driving car using image recognition. Implemented behavior planner using FSM and a path/trajectory generator using cubic spirals. NLP, Python Performed sentiment classification using the Amazon review dataset. Data was processed from XML, used Naive Bayes and K-nearest Neighbor classifiers, verified how these models worked on a cross-domain dataset. Machine Learning, MATLAB, Python Worked on a 4 class label gene expression dataset to train a decision tree, random forest and Adaboost classifier with classification error as the evaluation metric. Used a dataset to detect fraudulent credit card transactions, used KNN classifier and decision tree model with F-score as the evaluation metric and optimized the model with random and grid search. Artificial Intelligence, Python Worked on projects involving Text reconstruction, Sentiment classification, Course scheduling, Blackjack, Driverless car and Pac-Man ## Work Experience ### Founding Engineer @ Rumi Labs Inc. Jan 2025 – Present | San Francisco, California, United States Agents catered to enhance consumer experience using computer vision, audio and LLMs. ### Computer Vision Research Engineer @ Mission Technologies, a division of HII Jan 2023 – Jan 2025 | Fairborn, Ohio, United States Projects related to Computer Vision in Automation and Navigation Technology Lab at Air Force Institute of Technology ### Graduate Research Assistant @ Columbia Business School Jan 2022 – Jan 2023 | New York, United States Working with Professor Kriste Krstovski. Building a new language model recommendation system for movies and songs using GPT-3. Predicting choice sequences and comparing generated predictions to current recommendation systems. ### Full Stack Engineer @ The New School Jan 2022 – Jan 2022 | New York, United States Using BERT model to develop a Full stack Application to detect racial, gender, and age bias in text. Used react for frontend, and flask for backend. Application is sourced on Amazon Web Services (AWS) ### Research Intern @ Raytheon Intelligence & Space Jan 2021 – Jan 2021 | Cambridge, Massachusetts, United States Part of Unmanned Innovations Lab, Raytheon BBN Technologies Working on projects involving unmanned/semi-autonomous systems. Developed a model to help drones and rovers in door detection. Took part in the FX-6 OFFSET DARPA event in Fort Campbell as a part of the CCAST team. ### Intern @ STR Jan 2021 – Jan 2021 | Woburn, Massachusetts, United States Actively contribute to the Intelligence Tools and Applications team that builds deep learning models utilizing geospatial data and visualized areas of high and low activity. Help identify specific events for this technology solutions provider supporting the Defense & Space industries. ### Intern @ Systems & Technology Research Jan 2020 – Jan 2020 | Woburn, Massachusetts, United States Applied domain adaptation on stylized images for face recognition. The generalized stylized model was validated on ~half a million images. ### Director @ MathAndCoding Jan 2016 – Jan 2018 | San Jose Responsible for creating Python programming curriculum for middle school students. Scheduled and coordinated classes across libraries and low-income schools. Presidential Service Award. ### Summer Intern @ Open Networking Foundation (ONF) Jan 2017 – Jan 2017 | Menlo Park Set up probes in CORD and viewing analytics on Grafana for this nonprofit organization with a mission to help drive transformation of mobile and broadband network operator infrastructure and business models. ### Research Staff @ Stanford University Jan 2017 – Jan 2017 | Palo Alto Developed a tool to accurately determine the prognosis for cancer patients. Built models to efficiently analyze huge whole-slide images and summarize patient details into an unsupervised, informative profile. Automated a tool to accurately determine prognosis, a key step towards personalized treatment for cancer patients. ## Education ### Master of Science - MS in Computer Science - Machine Learning Columbia University ### Computer Science - BS Commonwealth Honors College in Major: Computer Science, Minor: Mathematics University of Massachusetts Amherst ### Summer Quarter in Computer Science Stanford University ## Contact & Social - LinkedIn: https://linkedin.com/in/nirmal-vegesna-411237123 --- Source: https://flows.cv/nirmalvegesna JSON Resume: https://flows.cv/nirmalvegesna/resume.json Last updated: 2026-04-05