Responsibilities
•Built a pipeline orchestration system using AWS and Kubernetes which improved reliability and speed of customer data delivery.
•Implemented data corrections and transformations in Spark and Scala that enabled new insights by both the data team and customers.
•Deployed and managed services in Kubernetes using Helm, facilitating simpler, more reliable deployments and releases.
•Managed infrastructure with Terraform, automating repetitive tasks and increasing the recoverability and resilience of systems.
•Automated legacy code to enable regular updates of internal tooling vital to the data scientists' work.
•Mentored and paired with junior engineers, improving the productivity and culture of the team.