Responsible for managing and maintaining the entire data platform. Optimizing pipelines and workflow whenever necessary.

• Built the data platform/infrastructure to support machine learning researchers

• Worked on an internal Dataloader in Python, that worked similar to PyTorch's Dataloader, that supported the needs of the Machine Learning engineers

Data Engineering @ Kik

• Inferred schemas for legacy data (stored in S3) using Apache Spark, necessary for implementing the data retention policy and GDPR requirements

• Built pipelines using Python, scheduled through Apache Airflow to support data science needs

• Built on call process (pager duty) from scratch which cut platform downtime by upto 12 hours

• Modified existing pipelines to write data to AWS RDS (postgres) as well as write logs to AWS cloudwatch which triggered pager duty alarms when appropriate