Lead developer / SME for Kafka & RabbitMQ services for the team of 8 engineers providing messaging & caching solutions
Designed the deployment and operations processes allowing the team to provide reliable and performant Kafka and RMQ services.
Designed multiple upper-layer usecases' (e.g. query trickle, core database transactions, search ingester, distributed job framework) usage of Kafka & RMQ, and acting as a technology expert in the case of incidents.
Created Java-based client libraries providing company-specific functionalities over plain Kafka/RMQ such as service discovery, metrics, encryption, configuration resolution and its change at runtime. Provided large message support in Kafka (chunking or using an external store like Redis) and controlled failover (with MirrorMaker2 and custom client code for automatic partition switch).
Replaced existing file-driven client configuration by creating a configuration-resolution service contacted by the client libraries during the using service's bootstrap.
Improved Kafka/RMQ operations by creating sidecar services responsible for the management of the RMQ and Kafka installations: providing service discovery data, RMQ cluster health monitoring, Kafka dead topic detection, ensuring backend configuration via policies.
Prototyped a joint Kafka-and-RMQ client library allowing for faster end-to-end communication while providing Kafka persistence guarantees.
Currently working on increasing service visibility and decreasing operator pain by creating the services for service topology tracking and request tracing.
Tech: Java, Spring + Spring Boot, Kafka, RabbitMQ, Zookeeper, Redis, SQL (Hibernate), JUnit + Mockito, Gatling, Scala, Python, Gradle, git, Docker, Envoy proxy, Chef, Kubernetes, TeamCity, Jenkins