Generative Recommendation

Large Recommendation Model

ML Infra

TikTok Feeds Recommendation System

TikTok Shop&Search Monetization, Advertising Platform

Designed and built a next-generation Distributed ML Model Training Framework for the ad recommendation tasks. The platform handles >400 million user ad click events per day and real time serving 500B-parameter models.

• Backend parts: HDFS-Kafka-Flink-ThriftServer input data streaming pipeline, Flatbuffer-gRPC distributed server communication, online model inference healthiness Monitoring, platform engineering metrics Monitoring, auto model quality validation.

• ML parts: Distributed model parallelism & data parallelism architecture using TF Distributed Strategy, FTRL sparse optimizer using CUDA Programming, synchronized ML model Checkpointing, HBM-RAM-ParameterServer embedding storage caching architecture, FP16 mixed precision training

• Yahoo Video AI Tagging Platform.

The platform is serving CV, NLP, and ML models in dockerized K8S environments to finish tasks like speech recognition, celebrity detection, video sensitive content detection, named entity extraction, video categorization, etc.

• NFL Football Field Detection & AR System

Individually researched and developed a football field computer vision segmentation model training and inference pipeline, including data augmentation, ICNet based neural network training, hyper-parameter tuning, model parameter validation, XLA compiler optimization and tflite model serving optimization.

This work was patented in the US and integrated into NFL OnePass App, achieving 99% reliability in the 2020 Super Bowl LIV.

Utilize ResBlock based 3D U-net and RPN/FPN to detect medical landmarks in the low dose CT images. Crop 3D image into 128*128*128 patches to fit one image into GPU's memory. Group norm technic is used between ResBlocks.