TL for datalake infra to support Flink based ingestion for Hudi and Iceberg.
Built and shipped a control plane at Uber to abstract format, facilitate rolling upgrades and automatically manage the background activities (clustering, compaction etc) to keep the data & metadata healthy.
Manage Parquet and integration with query engines (Presto & Velox) internally.
Previously TL for Data dispersal platform, our team builds platform to move data across offline(Data lakes/ Parquet Hudi) and online stores ( Cassandra, Docstore, Redis, Kafka, Habu, GCS, AWS) at scale. Built on top of spark, airflow the platform handles petabytes of data/week.
Prior to this role, built a realtime regulatory reporting platform as one of founding engineers based on Flink, Kafka for safety & compliance of riders/drivers across the markets Uber operates in.
