Improved data freshness from 2 days to < 1 hour by building real-time data pipeline with Spark Streaming, HBase, and Kafka to target millions of users in real-time

Migrated existing backend API from Playframework to Dropwizard providing 5-10 times faster response time, RESTful API, and clean OOP design

Delivered multiple high risk features involving heavy migrations such as adding metadata and code flow migration with Adaptor Pattern, significantly improving user workflow

Achieved stronger fault tolerance by implementing feature to upload data from Spark to Cassandra tables, worked around multiple Cassandra framework issues

Led discussions and formalized development, release, and branching process resulting in a simple and effective process

Encouraged a growth and sharing team culture by hosting monthly tech talk sessions

Simplified Spark job submission from synchronous job submission with SQS to asynchronous submission using Apache Livy, significantly reduced codebase complexity and production issues while providing accurate job status

Migrated batch jobs from MapReduce / Hive to SparkSQL improving the performance by 5 times

Identified and mitigated Spark Job bottlenecks by utilizing spark caching, further improving performance by 20%

Delivered multiple high impacting business features such as OR operator support and attaching custom payload to API response