Tech lead for designing and building our core internal and external facing services APIs. REST, GRPC, GraphQL.

Recursive graph queries. Large payload variety. CDN (cloudfront) offloading. Query optimizations.

Tech Lead for Twitter's Real Time Analytics team. The custom analytics platform backs all systems requiring real-time analytics (e.g. https://analytics.twitter.com).

Offline systems (including Machine Learning applications) requiring analytics are also built on top of our platform.

It consists of cross cloud stream aggregation pipelines (Kafka, PubSub, Beam/Dataflow).

A custom analytics store on top of Bigtable and Bigquery.

Highly distributed serving infrastructure, which includes custom OLAP systems and SQL frontends.

I'm one of the top contributers to all and designed some of the systems.

Scale of the platform:

Aggregating 30 million events per second.

Processing ~3PB of data a day.

Serving 100,000 queries per second at 300ms p99 latency.

Storing > 1500 definitions of datapoints / metrics.

Serving > 50,000 analytical queries (rolled up/re-aggregated data, spanning multiple tables, pivoted, etc) at < 1s p99 latency.

Part of Twitter's Ads Analytics and Ads Data Platform Team.

We rebuilt all of Twitter's ads analytics infrastructure and software on top of Google Cloud.

I built core processing systems for our platform using Scalding (Hadoop), Beam/Dataflow, BigQuery, Bigtable in various formats, thrift, avro, parquet...

I migrated Twitter's core ads analytics datasets to GCP, specifically to BigQuery and Bigtable (> 4PB of data).

All of Twitter's external Ads Analytics APIs are build on top of this technology.

(e.g. ads.twitter.com) is backed by our analytics platform.

I presented at Google Next 2018: https://youtu.be/sitnQxyejUg

I am an Apache Committer of Apache Superset https://superset.apache.org/

Part of Twitter's Ads Cloud infrastructure team. Helped to build out a Twitter compatible data center on top of AWS.