Advancing the frontier of streaming storage engines.

• Write Caching: https://redpanda.com/blog/write-caching-performance-benchmark

Identified the potential for a dedicated platform team to manage and drive the continued evolution of the Cloudflare’s online analytical platform based on ClickHouse. This platform is used for processing, storing and serving online queries on data arriving at an aggregated rate of 100+ million events per second. I kickstarted the team and led development and operations.

• Co-designer and co-implementor of data rebalancing in ClickHouse.

• Contributed, together with the team, over 100 pull requests to the open-source ClickHouse project, and worked with the community to advocate for a roadmap aligned with Cloudflare’s goals.

• Researched and identified new architecture directions for efficient, multi-tenant, big-data analytics systems to support the next order(s) of magnitude growth.

• Co-designer of a custom orchestration platform for managing stateful clusters (thousand of bare-metal machines across the globe).

• Co-designer and co-implementor of the Purpose-Based Access Control system for managing analyst/support-team access to data.

• Built a self-serve platform for remote development environments, which reduced the duration of build-test iteration cycles from hours to minutes. Initially, I targeted our own team, but it was quickly adopted by other teams working on large codebases.

• Collaborated with multiple product engineering teams to help them design workloads in a way that work efficiently on our particular platform.

• Collaborated with the hardware team to improve the performance and efficiency of the server designs.

Co-implementor of Security Events Logs. Stores information about all matched requests and enables customers to retrieve them (millions of events per second across the globe).

Co-implementor of Filter Based Firewall. FBF allows customers to configure security rules (with a Wireshark like syntax) that are delivered to all Cloudflare POPs (~150) in real-time and are applied to all traffic.

Contributed to infrastructure, platform and product.

Designed, built and operated distributed system for real-time notification delivery from internal services to web and mobile clients. Used Go to achieve low resource usage and low latency, Redis for storage and fan-out, gRPC for inter-service communication/event publishing and server-sent events for client communication.

Designed and led platform development for media storage and delivery. Used Go for image processing and serving (C FFIs for performance critical sections), GCP for storage, Cloud Datastore for metadata and CDN for caching.

Contributed to social and discovery feeds services built with Scala (on Twitter’s Finagle stack) and MySQL. To improve performance, I’ve developed a MySQL replication protocol client to sync derived data to Cassandra (feeds fan-out) and Elasticsearch (faceted search).

Built tools to automatically monitor and set up essential alerts for services running inside Kubernetes, leveraging open-source software like Prometheus, Alertmanager and Grafana. For advanced metrics and alerts, designed a DSL for describing application level metrics. Built internal libraries for Go and Scala (Finagle) to provide unified experience for observability (logging, metrics, tracing).

Collaborated closely with the frontend and mobile teams on API design.

Built and launched Retently’s first product with a small team of engineers.

Used technologies such as React, NGINX, NodeJS, MongoDB, Elasticsearch and Redis, all deployed on bare-metal infrastructure. Participated in product decision-making. Designed an improved NPS aggregation method to provide a better picture to Retently’s clients about their customers while making sure it could run efficiently as a MongoDB aggregation pipeline. Integrated external services for data import and billing.

Selected contributions: built a distributed build system (controller/worker model); built the software distribution system (automatic updates, license management, billing integration); automated the infrastructure management.