Amsterdam Area, Netherlands
Designed and delivered a minimum viable product (MVP) of a unified ETL tool for the Pricing Architecture department, processing trading data across multiple teams. Previously, each team maintained its own ETL solution, which created challenges in resource sharing, scalability, and operational efficiency. I took initiative to gather requirements across teams, conducted analysis, and led the full software development lifecycle — from design and implementation to testing and deployment.
The solution standardized the trading data ETL process by extracting XML-based trading data, leveraging Apache Spark for high-throughput batch processing, and loading it into Apache Kudu as a low-latency, lightweight persistence layer — thereby enabling a unified technology stack. This unification:
• Reduced the learning curve and simplified cross-team collaboration
• Improved scalability and maintainability with fewer lines of code
• Established a foundation for future integration and extension