Ingested data from kafka stream and build real time aggregates in clickhouse for ad serving data

Led the end-to-end development of real-time ingestion pipelines using technologies like GraphQL, Kafka, DynamoDB Streams, and AWS Kinesis, delivering both real-time and offline data into ClickHouse and Druid and presto

Developed GraphQL-based ingestion to stream data into real-time (ClickHouse, Druid) and offline ( S3 parquet) data stores.

Designed and implemented real-time CDC (Change Data Capture) pipelines from DynamoDB into ClickHouse and Apache Druid, enabling sub-second ingestion and millisecond-latency querying for user-facing dashboards.

Built GraphQL-based APIs to serve both streaming and historical data with unified query interfaces.

Architected resilient data ingestion systems with replay, ordering guarantees, and schema evolution handling.

Integrated streaming and batch pipelines to support both real-time insights and offline reporting/analytics from a single unified data model.

3rd party data ingestion - google adwords, twilio etc

Built webapp to commision aws EMR resources for on demand data pipelines

various data tooling webapps for on demand aurora data backups

spark streaming data ingestion into snowflake

mapreduce based data pipelines

membership segmentation lucene/elastic index for on demand data refreshes

events pipeline for lead generation product using Hadoop Mapreduce.

real time and intuitive analytical decisions by running hive queries on the HDFS files to facilitate quick decisions in data scrubbing/ pixel fire,