Design and implement complex stream and batch applications for programmatic advertising businesses using Apache Kafka, Apache Beam on Apache Flink, and Google Dataflow runners, processing terabytes of time-series data into Apache Druid and ClickHouse.

Manage state and observability of streaming applications, ensuring smooth updates and modifications to align with evolving business requirements.

Build and manage batch pipelines using Rill Cloud, dbt, and Apache Airflow on Kubernetes (GKE).

Optimize distributed applications and Druid ingestion specs for maximum performance and scalability.

Develop database connectors to extract data from data warehouses (Snowflake, Redshift) and data lakehouses (Delta Lake, Apache Iceberg), enabling seamless ingestion into Apache Druid and ClickHouse.

Quickly prototype and build POCs on DuckDB to validate data pipelines and accelerate solution design.

Possess deep understanding of database internals, applying tuning techniques to maximize query performance and resource utilization.

Implement tools and frameworks to automate data processing and ingestion, while improving overall database performance.

Implement and manage incremental ingestion features on Rill Cloud, enabling efficient data updates with reduced cost and latency.

Drive all development and operations with a strong focus on cost control, ensuring high efficiency without compromising performance.

Build and maintain streaming applications and data pipelines using Apache Storm which processes and enriches the application data.

Develop analytical services and maintain integrations between data level microservices and other backend services using Kafka, Apache thrift, and GRPC.

Develop and maintain fault-tolerant complex ETL and ELT jobs using Apache Airflow.

Design and maintain AWS Redshift data warehouse.

Build tableau reports on top of AWS Redshift for business owners and digital marketing teams. Assist them with ad-hoc SQL queries on demand which helps in business key decisions.

Successfully developed cost-effective data-intensive streaming analytical applications from scratch.

Developed and maintained ~30 node Hadoop Spark clusters on AWS EMR, which can process ~100GB sensor data effectively in a day.

Building ad-hoc reports using tableau and hive queries to answer business questions.

Worked with Engineering teams in developing integrations with data tech stack using Restful & GRPC.

Built several engineering services using golang, java, scala, and MongoDB, PostgreSQL as database.