Building next generation business metric platform for Uber using the ongoing real time trip events/signals.

Also work on persisting of the state data on Google Cloud Spanner. With efficient querying to the storage layer by maintaining an in-house cache layer on top of spanner.

Currently working on open source Microsoft hyperspace project, indexing sub-system built on Delta Lake storage for big data engine Apache Spark.

Designed and implemented finalization design for Azure blob file system Gen2 for Structured Stream v5 (Scope) to bring Microsoft compliant for JEDI requirement.

Design and implement a scalable parallelized architecture of the task scheduling service to support token/resource accounting for job with respect to container consumption.

Support for smart allocation and assignment of variable size containers between AM-RM to schedule tasks with different resource requirements.

Support for identifying half token/container support at runtime. Identifying tasks that can run in half of the resources prescribed, allows us to schedule 2x more work for a job.

Build Job Manager for Scope on Azure in YARN Cosmos cluster. Working on building on top of Open Source Apache Tez https://tez.apache.org/ DAG execution framework.

Deploy and extend Tez framework for execution and control of scope(query engine) algebra/plan for big data processing jobs on the cosmos clusters.

Worked on creating the routing and shuffle module of scope for Tez.