Search Quality

Training LLMs and Multi-modal foundational for all

1. Audio Data Pipeline Development

2. Long-context LLM Improvement

3. Low Latency E2E Audio Infra

4. Training and Finetuning data improvement

Stitching different anchor models to generate an efficient stitched vision and language transformer architectures

Researching about SWIN, DeIT, ViT, LeVIT and other vision transformers and understanding what each layer learns and difference in activation spaces

Applying Pytorch distributed training techniques such as Distributed Data-Parallel to train parts of vision transformers

Instructed artificial Intelligence in games and developed assignments on path planning, navigation, behavior trees, deep reinforcement learning with Prof. Mark Riedl.

Instructed Deep Learning and updated Deep Learning Assignments by adding transformers, diffusion models, and GANs

Instructed ~200 students by hosting regular office hours, debugging student code, and online forums.

• Engineered enhanced scaling strategies for autoregressive model inference to extend them for LLMs

• Reviewed literature to identify attention mechanism weaknesses in transformers to create optimal LLM architectures

• Explored concepts of Grouped Query Attention, Dynamic Scaling of multi-head, Newer approaches to reduce attention cost

Part of Systems for AI Lab