BigQuery

https://eng.uber.com/research/keeping-master-green-at-scale/

Worked on a large scale, highly available, and fault-tolerant platform for Internet of Things.

Selected project: https://www.usenix.org/conference/atc19/presentation/sengupta

Worked on a novel actor based programming model along with a runtime system called AEON [Middleware’16]. AEON allows programmers to easily develop elastic and highly scalable applications for the cloud. With AEON, programmers only need to reason about sequential semantics of their applications. Our runtime guarantees safe (linearizable) execution of multi-actor events in the cloud, while maximizing parallel execution.

Worked on (1) scalability limitations of well-known consistency criteria, and (2) ensuring consistency in large scale geo-replicated systems.

I identified the following essential scalability properties for a distributed transactional system: (1) Genuine Partial Replication: only replicas updated by a transaction T make steps to execute T. This property ensures that non-conflicting transactions do not interfere with each other, hence the intrinsic parallelism of a workload can fully be exploited; (2) Wait-Free Queries: a read-only (query) transaction never waits for concurrent transactions and always commits. In workloads with a high portion of read-only transactions, this property is crucial for the scalability of the system; (3) Forward Freshness: a transaction may read object versions committed even after it started. This property decreases staleness of reads, and abort rate; and (4) Minimal Commit Synchronization: two transactions synchronize with each other only if their writes conflict. Therefore, synchronization is avoided unless absolutely necessary.

I also showed that none of the popular strong consistencies (e.g., Strict Serializability, Full Serializability, and Snapshot Isolation) is able to satisfy all the above four scalability properties.

I introduced Non-monotonic Snapshot Isolation (NMSI) as the strongest consistency criterion allowing all the above four properties.

I also developed a framework, to better study, and compare various (deferred) update replication protocols (DURs) [Middleware’14]. Protocols of the DUR family differ only in behaviors of few generic functions. Based on this insight, and in order to fairly compare DUR protocols, I introduced a generic DUR framework, called G-DUR, along with a library of finely-optimized plug-in implementations of the required behaviors. This framework allows us to easily assemble, analyze, and compare DUR protocols.