With over 6 years of experience, I'm a Big Data Software Engineer adept at transforming complex data into actionable insights. Skilled in Hadoop, Spark, Airflow, and more, I excel in designing scalable solutions that drive business growth. Let's connect to explore how I can contribute to your team's success.

2022 — NowZillowSenior Software Engineer Big Data

2022 — Now

San Jose, California, United States

Built a robust data pipeline to create a high quality dataset, facilitating seamless integration between two business lines at Zillow. Enabled data science partners to drive an estimated $30 million revenue enhancement.

Developed a Python cookiecutter for self-serving Airflow DAG generation. Increased developers efficiency and facilitated the construction of over 20 DAGs for the team.

Collaborated with principal architects to architect an enterprise dimension model, establishing best practices for its implementation.

Built comprehensive dimension models to seamlessly onboard critical L1 metrics through a semantic layer

2021 — 2022ZillowSoftware Engineer II Big Data

2021 — 2022

United States

Created a self-service tool facilitating seamless onboarding of raw JSON data into Parquet-based Hive tables. Widely adopted by numerous customers for efficient data lake integration

Developed a scalable, standardized Scala-based pipeline structure for the team, segregating business logic from input/output. Reduced dev time, enhanced MR review, and eased unit test integration

Built high quality data pipelines to create business critical data products by working with data science partners and cross functional teams

2019 — 2021EmailageSenior Big Data Engineer

2019 — 2021

Greater Phoenix Area

Designed, built and maintained scalable, serverless, near real-time data warehouse; Ingested ~5M transactions daily; Reduced analytical query execution time by 10x

Collaborated with security and compliance team to make the data secure and GDPR compliant

Implemented lambda architecture for real-time dashboard to monitor ML model and model features

Created Spark applications which reduced time of daily SQL queries from 3 hours to under an hour

Deployed neo4j and implemented Pregel Algorithm to leverage Graph analysis for fraud prevention

2018 — 2019EmailageBig Data Developer

2018 — 2019

Phoenix, Arizona Area

Dockerized Shiny app dashboard and deployed on K8s cluster to enable CI/CD pipeline

Implemented interactive web app to generate high/low-risk rules to streamline Decision Scientists workload

Built an ETL pipeline to load the data in PostgreSQL data warehouse for data analysis

2016 — 2019Arizona State UniversityResearch Assistant

2016 — 2019

Tempe, Arizona

UN Project:

Scraped dark web, social media data anonymously and maintained ETL pipeline for its sentimental analysis

Migrated Apache Solr to Elasticsearch – Designed distributed multi-node cluster to improve reliability, availability, and performance; indexed over half billion data records in Elasticsearch for fast retrieval

Developed and deployed six rich interactive visualization dashboards to find actionable insights

Integrated React.js into all the dashboard websites to ensure high quality and reusability of code

Maintained ETL pipeline on the stream of Tweets

Lockheed Martin Project:

Worked on analytical NLP project for real-time discovery of issues, their frames, and evolutionary topic tracking

Wrote batch script to automate project pipeline; incorporated hashing to reduce time complexity of string matching

Created visualization website to explore the content of large set of news articles to leverage quick reading

Education

2016 — 2018

Arizona State University

Master of Science (MSc)

2016 — 2018

2009 — 2013

Pune Institute of Computer Technology

Bachelor of Engineering (BE)

2009 — 2013

Experience+

Education

Master of Science (MSc)

Bachelor of Engineering (BE)