Databases - Change Data Capture Team, Storage Services Team

Transactional Databases Team, Change Data Capture Team

Core Storage Team

Schemaless and Docstore are general-purpose key-value stores and transactional databases at Uber serving core business online transactional workloads such as Core Trips, Settlements and User Platform.

• Led the development of Spark ETL framework for index backfills, cross-region consistency checks and user workloads, decreasing the index build/repair issues by 90% and job wait time by 3x.

• Built the first quota management and distributed rate limiting capabilities for Docstore using Redis.

• Optimized Schemaless secondary index queries and reduced write amplification during index repairs.

• Developed Docstore's write and read path APIs and improved reliability of TTL features.

Cassandra Team

Cassandra team runs Uber's large-scale Cassandra managed services that serve write-heavy mission-critical use cases including Marketplace, Surge, and uGraph.

• Led the development of CDC for Cassandra, enabling a real-time data ingestion pipeline to Hadoop and an offline ETL pipeline to bootstrap data to HDFS.

• Built a universal task framework for Cassandra to support massive data export/import of SSTables snapshots, node operations and self-service long-running jobs across tens of thousands of Cassandra nodes.

• Maintained Cassandra clusters at 99.99% availability and enhanced its reliability and scalability to meet operational demands at Uber's scale.

Kafka Team

• Built the first hybrid cloud, multi-zone Kafka infrastructure at Uber to power the next-generation payments platform, Gulfstream.

• Maintained and evolved the Kafka infrastructure at high durability and availability and at Uber's scale across ecosystems of brokers, ZooKeeper, REST proxies, and MirrorMakers.