# Scott Sandre

> Senior Software Engineer @ Databricks

Location: San Francisco, California, United States
Profile: https://flows.cv/scottsandre

I'm a Software Engineer at Databricks, working on the Delta Ecosystem team. I enjoy designing, leading, implementing, and shipping innovative features for open-source Delta Lake. I excel at problem-solving and breaking down ambiguous challenges into concrete solutions, with a particular passion for tackling performance issues. I hold a Bachelor's of Software Engineering degree from the University of Waterloo.

## Work Experience
### Senior Software Engineer, Delta Ecosystem @ Databricks
Jan 2024 – Present | San Francisco, California, United States

### Software Engineer IV, Delta Ecosystem @ Databricks
Jan 2022 – Jan 2024 | San Francisco, California, United States
• Led the development of the Delta Flink connector, allowing Apache Flink to write Delta Lake tables. Engineering point of contact for multi-million-dollar customer. Investigated and shipped critical correctness (data loss) and performance improvements, reducing initialization time by 45x and CPU usage by 8x. Designed and productionized cross-engine, multi-cluster concurrent writes from Flink and Databricks Runtime into S3.
• Helped develop Delta Kernel, a new abstraction over the Delta protocol that allows engines to build simpler Delta connectors using narrow APIs. Specifically, focused on Delta log metadata replay and performance. Designed and shipped 
new 'hint' algorithm to improve initial snapshot schema loading by 34x. Investigated CPU bottlenecks to further improve performance 3x.
• Helped develop Delta Universal Format, a Delta feature that allows converting Delta metadata to Apache Iceberg. Designed and implemented the Iceberg Compatibility V1 table feature, which protects Delta tables from operations that would make them incompatible with Iceberg.
• Unblocked Model Serving Inference Tables on AWS by designing a single-node client to coordinate Delta metadata commits to S3 via the Databricks S3 commit service.
• Designed from the ground up internal tooling, auditing, and test systems for all Delta development at Databricks. Decreased test run time 30x from 3+ hours to ~6 minutes.

### Software Engineer III, Delta Ecosystem @ Databricks
Jan 2021 – Jan 2022 | San Francisco, California, United States
• Developed the Delta Standalone Writer, a Spark-less connector to Delta Lake that writes transaction log metadata while maintaining ACID guarantees.
• Designed and developed S3 multi-cluster write functionality for Delta Lake, implementing a new cloud store API that uses DynamoDB to provide the mutual exclusion that S3 is lacking.
• Developed new performance and functionality features for Delta Lake including data-skipping using per-column statistics and Change Data Feed .

### SWE Intern, Delta Ecosystem @ Databricks
Jan 2020 – Jan 2020 | Remote
Designed, built, and released a Spark-less transaction log and data reader Java SDK (written in Scala) to make it easier for other query engines to connect to Delta Lake.

Also developed a new end-to-end benchmarking framework to test the performance of Delta Lake using simulated real-world workloads.

### SWE Intern, Build Infrastructure @ Databricks
Jan 2020 – Jan 2020 | San Francisco, California
• Improved build stability and failure observability by migrating key Jenkins jobs to internal build system Runbot. Included simpler configuration, automated metrics, Spinnaker pipeline integrations [Python, Jsonnet]
• Let devs easily find origin of flaky test failures through new search, filter Runbot API endpoints [Scala, Quill]
• Increased runtime exception triaging accuracy to teams using owner-wrapping framework [Scala, Sentry]
• Designed Runbot dashboard views, automated job-tagging infrastructure for concise per-team UIs [ScalaJS]

### SWE Intern, Backend @ Veeva Systems
Jan 2019 – Jan 2019 | Toronto, Canada Area
• Developed backend API for custom entities in Network Explorer, a graph-exploration tool for healthcare relationships. Added node expansion, custom filtering, and efficient search capabilities using Java, Spring MVC, SQL, Elastic Search and JPA
• Created and optimized in-memory index to perform robust address validation lookups using Apache Lucene
• Migrated over 100,000 customer geocode records using Groovy script that automated parsing and upload

### SWE Intern, Automation @ SAP
Jan 2018 – Jan 2018 | Montreal, Quebec, Canada
• Decreased regression test runtime by 40% by designing automated testing framework that allowed for more modular tests, customized validations and quicker development using Java, Selenium and Spring MVC
• Fixed variety of critical issues affecting storefront search faceting and product display using Angular and RxJS

### SWE Intern, Mobile @ Bunch
Jan 2018 – Jan 2018 | Toronto, Canada Area
• Worked on Bunch, a social gaming iOS app, and became a pro with React Native, Google Firebase and Redux in the process
• Re-engineered the main feed used throughout the app to be faster, more modular and far more maintainable
• Improved onboarding completion rates by 10% by redesigning and streamlining redux architecture

• Implemented entire frontend of v3 of the app

### SWE Intern, Full Stack @ LyricFind Inc.
Jan 2017 – Jan 2017 | Toronto, Canada Area
• Worked with backend engineers to design and implement bulk-DAO framework to improve query speed for large data sets
• Improved speed of key lyric analytics search tool using Apache Solr
• Re-implemented assignment-manager web application using React

### SWE Intern, Backend @ TD Securities
Jan 2016 – Jan 2016 | Toronto, Canada Area
• Global Equity Derivatives Team
• Worked with backend engineers to improve performance and efficiency of financial report generation system


## Education
### Bachelor of Software Engineering in Honours, Co-op
University of Waterloo
Jan 2016 – Jan 2021


## Contact & Social
- LinkedIn: https://linkedin.com/in/scottsandre

---
Source: https://flows.cv/scottsandre
JSON Resume: https://flows.cv/scottsandre/resume.json
Last updated: 2026-03-23