SnapLogic is an Integration Platform as a Service (iPaaS) that enables quick development of sophisticated data and application pipelines using a visual drag and drop user interface. The SnapLogic Intelligent Integration Platform delivers a multi-tenant cloud service for all styles of integration - application or data, batch, or streaming. It is designed to handle any variety, volume, velocity, and location of structured and unstructured information. The Engineering team at SnapLogic is divided into 6 parts as UX, Platform, Big Data/Extreme, Snaps, Data Science, and QA teams.

As part of my internship project, I worked on SnapLogic's Platform team. The Platform team works on both the Control Plane (Python-based REST services) and the Java Component Container (Java execution engine that executes customer-designed integrations). Together, they power all SnapLogic iPaas services that run millions of customer-built data and application integrations per month.

During my internship, I created a reference microservice application for modernizing the

SnapLogic Control Plane. Today, the Control Plane consists of four monolithic applications and they use an older version of Python, UWSGI (Python webserver), and Werkzeug (Python web framework). The Platform team wanted to evolve the architecture into a microservices-based design.

Project Title - SnapLogic Persistent Event Streaming (Masters Research Project)

SnapLogic is an Integration Platform as a Service (iPaaS) that enables quick development of sophisticated data and application pipelines using a visual drag and drop user interface. The current SnapLogic execution engine does not provide durable and persistent storage for in-flight data, instead, most processing is done in memory.

We developed a prototype of a SnapLogic Persistent Event Streaming service that extends the existing architecture by providing a native streaming service. Ultimately, this service can be integrated with the native Snaplex execution environment so that SnapLogic customers can easily deploy and leverage event streaming. In this project, we built Kafka-like components, a durable and persistent logging mechanism, and distributed consensus. Our implementation can survive node failures. We also built Streaming Service Snaps that allows a SnapLogic user to build visual pipelines that can read and write to our streaming service.