Tech Lead

scaling complicated backend services to accommodate traffic spikes

1. Developed Abacus 2.0, a complete overhaul of Klaviyo's ingestion model, read API and database schema. This new generation of streaming microservice is able to run 10 times faster than its predecessor.

2. Developed migration service to migrate of data to the new schema.

3. Scaled the legcay event pipeline, of which the throughput was 500 events/s, to be able to handle 150,000 events/s (3 million database writes per second). This new system is not only able to handle 300x traffic but also can be readily scaled horizontally.

1.Developed Abacus 1.0 using Apache Flink, a modern streaming micro service, to replace Klaviyo's legacy ingestion pipeline. Abacus processed 1.5 billion events smoothly on Cyber Monday alone.

2.Developed Klaviyo's first Kafka microservice, which is now handling billions of ingress/egress traffic everyday.

3.Maintained various core data stores including Mysql, Cassandra and caching layers including Redis, Memcache.

4.Refactored legacy ingestion pipeline code using a fully OOP design. The refactored pipeline is much more readable, reusable and extendable.