Designed bookkeeper gRPC API with Protobuf and ingest data into Kafka, Iceberg and AWS S3.

Utilized Apache Spark, Airflow and Hadoop to develop data pipelines to process 256 billion records.

Guided more than 10 teams to integrate, reduce 40% alerting delay time and improve control granularity.

Designed service provider interface and utilized remote procedure call to gather message content from other systems when sending out periodic messages. Reduced disk usage for message content by 70%.

Cached message recipient’s information in Redis and speed up message sending by 10x.

Utilized RocketMQ for peak load shifting and achieved over 10 million messages sending per day.

Designed file sink operator to save workflow execution results to json, excel and Google Sheet. Solved memory problems for huge results and allowed the system to process TB wide files.

Developed user system using Dropwizard RESTful web services and utilized JOOQ to save workflow metadata and user information to MySQL database.

Coached over 60 students about the syntax of programming language and debugging skills.