# Hongbo Shi

> Software Development Engineer at Amazon

Location: Emeryville, California, United States
Profile: https://flows.cv/hongboshi

•	VLLM, TRTLLM, Pytorch
•	Inference Optimization for latency/throughput/cost
•	Speculative decoding, page attention, continuous batching, Quantization etc

## Work Experience
### Software Engineer @ Anuttacon
Jan 2025 – Present

### Software Development Engineer @ Amazon
Jan 2020 – Jan 2025 | San Francisco Bay Area
•  Leading Amazon’s model processing platform to optimize models in post-training for best inference performance
• Research and Integrate the cutting edge Inference technologies (Speculative decoding, Flash attention,
Quantization, Page attention) to achieve among between latency, throughput and cost
• Design and implement tools and pipelines for model optimization and deployment, speed up iteration by 5X
• Lead and integrate Hugging-face, Triton, FasterTransformer and TRT-LLM to production

### Software Engineer - Machine learning @ Petuum, Inc.
Jan 2019 – Jan 2020 | Sunnyvale
Developing great AI platforms to make machine learning easier than ever for gas&oil companies, health industry, and finance industry


implemented auto-tune algorithm for hyper-parameter searching  into training pipeline

### Senior Design Enablement Engineer(Software Tool Development) @ GLOBALFOUNDRIES
Jan 2017 – Jan 2018 | San Francisco Bay Area
* Evaluating machine learning models and rule-based model for lithography simulation (Large Geometry Data)
* Constructed pipeline using python, bash and Tcl to accelerate and automate data processing, resulting in a 80% decrease in time needed for evaluating software and model quality 
* Increased accessibility and usability of customer data by redesigning data visualization techniques to include statistical graphs and information
* Improved object-oriented Python codes to check manufacturing parameters and reduced labor errors by 90%
* Developed new components in object-oriented Tcl codes and maintain codes with version control tools


## Education
### Bachelor's degree in Chemical Engineering
Beijing Institute of Technology

### Master's degree in Chemical Engineering
Tsinghua University

### Doctor of Philosophy (PhD) in Chemical Engineering
University of Massachusetts Amherst


## Contact & Social
- LinkedIn: https://linkedin.com/in/hongboshi

---
Source: https://flows.cv/hongboshi
JSON Resume: https://flows.cv/hongboshi/resume.json
Last updated: 2026-04-05