Reimagining Redshift.

Designed the distributed catalog layer for Redshift data sharing. Led a team to implement the same. Lot more fancy features are yet to come (stay tuned)! For publicly announced data sharing feature refer to https://docs.aws.amazon.com/redshift/latest/dg/datashare-overview.html

Helped the team delivering Redshift serverless.

Led a team across multiple AWS services to deliver LakeFormation governed Redshift datasharing.

Made Redshift under the hood startup time several times faster to improve customer experience as well as to cut down cost on AWS side.

Core contributor of Redshift concurrency scaling execution engine (https://aws.amazon.com/blogs/aws/new-concurrency-scaling-for-amazon-redshift-peak-performance-at-all-times). Worked on many parts such as distributed execution logic, planning, storage layer portability etc.

Worked on other critical components such as Spectrum and AQUA.

Worked on query processing engine for Apache HAWQ.

Numerous contributions in design and implementation of different subsystems. Lead query execution team for Greenplum Database. Won multiple company awards and was awarded US patent# 10037144. Some of my projects are:

Design and implementation of memory accounting framework to accurately track memory usage across different subsystems. The framework also provides collaborative logging across multiple sessions to snapshot system memory usage as required.

Design and implementation of runaway query termination framework to detect and terminate queries with abusive resource usage pattern.

Design and implementation of a resilient join operator that can intelligently adopt different join

algorithms, including nested loop join and hash join. The resilient join operator can also detect skew, swap inner and outer (if permitted), adopt different hashing functions, dynamically spill different subsets of hash buckets and coalesce such spilled buckets to maximize the utilization of available memory.

Design and implementation of an unified dynamic scan operator that significantly reduced spaghetti code across several different partitioned and non-partitioned scan operators for relations and indexes.

Redesign and implementation of bitmap scan to reduce memory usage by orders of magnitude

by establishing a clean cross-operator memory ownership model across BitmapAnd/BitmapOr, the BitmapTableScan and BitmapIndexScan.

Optimization of idle session memory consumption to bring down gigabytes of leaking memory to a constant few megabytes (earned a platinum recognition award for this).

Contributed to numerous design discussions and customer bug fixes.