Backend Engineer, Catalog Data Platform

Developed a generic data API layer with one single endpoint and unified query syntax to aggregate catalog data queries targeting multiple data sources such as Druid, Cosmos Db, SQL and internal APIs.

Built end-to-end data solutions to provide category performance insights to Flightdeck team, components including ETL pipeline from Teradata to Azure Blob, Druid indexing job orchestration Airflow, data providers with customized pre-process and post-process functionality in API.

Developed business enhancement features for API and built merchant data streaming and indexing microservices using Kafka, ElasticSearch and EventStore.

Implemented and productionized the geo-replication of merchant API and microservices.

Led holiday stress testing and failover exercises on API and order workflow microservices.

Designed and implemented a new solution framework on Hadoop to process GPS and browsing data (over 8GB daily) for customer profiling using Apache Spark, reducing 40% of processing time.

Integrated extra features by parsing HTML and JSON file and Applied scalable k-means and LDA model for browsing patterns recognition, and achieved 70% precision of demographic prediction.

Built an automatic Data Visualization tool to show customer segmentations and content watching patterns with Google Charts and D3.js.