Maintaining and enhancing a critical ETL platform that transfers data from HDFS to multiple downstream systems including Cassandra and Snowflake.

Responsibilities:

Design and implement new features to enhance ETL service functionality

Optimize Spark job performance through transformation logic analysis and configuration tuning

Implement data quality checks and validation using Deequ framework

Build monitoring integrations with internal systems for pipeline observability

Enhance incident response processes and implement automated alerting

Troubleshoot production issues to minimize downtime

Onboard new teams and jobs to the platform

Contracted to Apple, delivering infrastructure and platform solutions across multiple teams.

Project 1: Report Download Service Migrated a reporting service to Kubernetes and improved its observability and reliability.

Migrated service to Kubernetes and integrated with monitoring systems

Implemented RED method metrics and load testing with Gatling

Added integration tests and migrated build system from Maven to Gradle

Environment: Kubernetes, Docker, Helm, Java (Spring), Python, Gatling

Project 2: On-Premise Sentry Maintained and upgraded a high-load, distributed error-tracking system serving internal engineering teams.

Updated Sentry-related services (Sentry, Snuba, Relay) to newer versions

Added custom integrations with OpenDirectory and Jira

Fixed existing integrations with Slack and GitHub

Implemented PostgreSQL/ClickHouse backup and migration pipelines

Built a backend service for managing and versioning reference data used across multiple banking applications.

Responsibilities:

Developed features end-to-end from specification to deployment

Optimized SQL queries for improved performance

Implemented digital signature creation and verification

Set up OpenShift/Kubernetes with service mesh for secure traffic routing

Created Jenkins pipelines for CI/CD automation