Built a scalable recommendation system to automate distribution of high quality petitions to a relevant audience, tools to scale and streamline data and ML workflows.
* Set up a distributed system to enable large scale processing of petition, user and event data
* Implemented ETLs to capture, normalize and partition data in cloud object storage
* Designed and built a location and topic based recommendation system to automate distribution of high quality petitions to a relevant audience, optimizing conversions
* Designed and built tools to streamline research and deployment of ML/statistical models by simplifying and unifying the process of dataset & feature generation
* Mentored engineers on architecture and effective software engineering practices
Technologies
Scala, Spark, Elastic MapReduce, S3/Delta Lake, Kafka, Airflow