Tech lead of Spark at LinkedIn. The team is focused on building and scaling LinkedIn's general purpose batch compute engine based on Apache Spark to empower the multiple use cases at LinkedIn ranging from data analytics, data engineering, to ML feature engineering and model training.

Tech lead of Hadoop YARN at LinkedIn. Improve YARN CapacityScheduler to handle the fast increasing compute needs and the diversified demands from different internal orgs.

• Lead the design and implementation of the OrgQueue project, which aims to decentralize the management of compute resources across the various internal orgs. OrgQueue feature was contributed back to Apache Hadoop YARN.

• Lead the design and implementation of project ElasticityTuner, which automates the OrgQueue management (tuning queue elastic capacity) based on real-time feedbacks of resource demands and allocation fairness among queues. This project converted the YARN cluster queue capacity management at LinkedIn entirely into an autonomous process, significantly freeing up dev and SRE team efforts.