I worked in Serving Systems, that later became KeyValue Systems team at Pinterest for all my time at Pinterest. Worked on variety of projects, including deprecation of legacy system, re-designing north-start key-value store system, requirements gathering and feature evaluations, system evaluations, API design for WideColumn KeyValue system, implementation of PII Isolation stack for KeyValue system, hot-key detection and avoidance on server side for legacy system to keep it alive while new system was being built, fast metadata delivery, apache helix, distributed metadata publisher (spectator). Also contributed significantly to Pinterest's open source Rocksplicator library using helix to strengthen control plane for stateful services at Pinterest (https://github.com/pinterest/rocksplicator/commits?author=gopalrajpurohit)

I worked in Twitter Ads Serving team for 4 years and presently working with Ads Prediction team. As one of the key application systems engineer in ads serving team, I architected and implemented core ad serving framework components of the largest application service in Twitter, as well as other auxiliary systems, re-usable libraries implementing core abstractions around data retrieval, indexing, in-memory caching, event publishing, event processing, runtime feature changes, debugging under serving campaign, real-time aggregation and data ingestion framework, live data migrations, sharding, hot key detection and avoidance, id generators, scale, performance and resiliency. I am also recognised as thorough code reviewer able to see scale and performance issues before they hit production.

Sample projects I have worked on

AdServer: Ad Serving Pipeline

Ads Database In-Memory Index for Serving

Streaming Data Processing Framework

AdServer Data Sharding Architecture

Hot Key (Iceberg Query) Detection and Avoidance

Real time Analysis of Under Serving Campaigns

Adserver Contingency

Live Data Migrations

Oiling Revenue Machine

I worked in Search, Recommendation and Infrastructure Team at StumbleUpon.

In StumbleUpon, I build a configuration driven Data Processing Framework, with core re-usable components of generic map-reduce implementations. Using above components in oozie workflow, I optimized pipeline of size more then 150 map-reduce steps, of keyword extraction and index building applications, achieving speed-up by 400%.

Build audience estimation pipeline of size 20 map-reduce steps within 2 weeks time, re-using the above data processing framework.

Achieved speed up by factor of 12 ( 120 minutes to 10 minutes) in a computation intensive complex data mining application by optimizing, preprocessing, partitioning and other hybrid strategies.

Also worked on bits and pieces of notification infrastructure ( kafka message processing ), search and recommendation infrastructure, elasticsearch plugins etc.