# Noah Raynor

> Engineering @ Vapi

Location: San Francisco, California, United States
Profile: https://flows.cv/noahraynor

## Work Experience
### Forward Deployed Engineer @ Vapi
Jan 2025 – Present | San Francisco Bay Area
The most configurable API to build leading voice AI products and scale phone operations.

### Software Engineer & Co-Founder @ LLMonade
Jan 2025 – Jan 2025
LLMonade is an open-source AI evaluation tool that helps development teams systematically improve their LLM powered applications through guided, manual error analysis. The platform ingests trace data, provides an optimized annotation UI, automatically categorizes failure patterns, and identifies the most impactful improvement areas.

• Engineered a serverless ETL pipeline (Lambda, EventBridge, RDS) to ingest high volumes of trace data from Phoenix via GraphQL, implementing cursor-based pagination for efficient, scalable, and continuous data synchronization.
• Designed a React-based annotation and categorization dashboard, served securely (S3, CloudFront, Cognito, Lambda@Edge), to create a frictionless manual error analysis experience.
• Architected a serverless backend (Lambda, RDS, API Gateway) using a RESTful API, integrating with the OpenAI API to automatically categorize failure modes and intelligently format trace data.
• Developed a CLI utilizing AWS CDK for automated, one-command deployment of all required infrastructure to AWS, reducing setup time and operational overhead for users.
• Published a technical case study documenting engineering challenges, architectural design decisions, and industry best practices to evaluate and improve LLM-based applications.
• Collaborated within a 4-person development team to design, build, and deploy the full-stack application, system documentation, and cloud infrastructure.

### Software Engineer @ Open-Source Projects
Jan 2023 – Jan 2025
Developed freelance applications including:
• Wine-Rag: Retrieval-Augmented Generation (RAG) chatbot that ingests proprietary winery data into a pgvector database and turns natural‑language preferences into recommended wineries (Python, OpenAI, SQL, pgvector).
• Request-Tub: Tool for collecting and debugging webhooks (TypeScript, Express, React, MongoDB, PostgreSQL).

### Co-Founder @ SLO Safe Ride
Jan 2010 – Jan 2023
Started, operated, and successfully divested a charter bus and limousine company.
• Led the product vision, feature definition, and UAT for two custom mobile applications (iOS/Android).
• Grew company revenue to $4M annually with a team of 40 employees.
• Congressionally recognized with the 2015 UC Merced Rising Star Award for outstanding service to the community.

### Business Systems Intern @ Warner Bros. Entertainment Group of Companies
Jan 2007 – Jan 2007 | Burbank, California
Internship in the Business Systems Planning division.


## Education
### Computer Software Engineering
Launch School

### Computer Engineering
California Polytechnic State University-San Luis Obispo


## Contact & Social
- LinkedIn: https://linkedin.com/in/noahraynor
- Portfolio: https://llmonade.github.io/

---
Source: https://flows.cv/noahraynor
JSON Resume: https://flows.cv/noahraynor/resume.json
Last updated: 2026-04-10