Inflect is an open-source stream processing framework for Kafka that simplifies stateless event transformations between microservices in an event-driven architecture
• Engineered a custom Javascript stream processor capable of transforming 40,000+ events per second per app instance
• Deployed Kubernetes infrastructure that dynamically scales from 1 to 50+ instances to optimize real-time event processing efficiency under fluctuating workloads, achieving peaks of 500,000+ events per second confirmed by load testing
• Implemented strategic caching and parallel batch processing of Kafka events to resolve event pipeline bottlenecks caused by event fetching overhead, increasing throughput by 4x.
• Designed React dashboard for defining, testing, updating, and managing multi-step transformation pipelines
• Implemented functionality for mock event generation to reduce time-to-production by 50% and improve visibility into real-time event processing
• Architected Node.js backend and PostgreSQL database system to enable seamless pausing, resumption, and real-time updates of transformation pipelines in production environments
• Integrated Prometheus instrumentation for system monitoring and performance optimization
• Automated a 15+ step deployment process, streamlining setup of containerization, database initialization, monitoring, and auto-scaling components, resulting in an estimated 90% time savings per deployment
• Authored Inflect's technical case study (inflect-framework.github.io)
