•Initiated and incorporated design changes through thorough consultations with the product teams to modernize eQ’s monolith products (eQubeMI and eQubeBI) by shifting to a microservice architecture, to leverage the products on cloud.
•Segregated and spearheaded the development of the standard modules of eQ’s products – Scheduler, Admin Console, and Adapter as loosely-coupled services.
•Designed and developed a simple service discovery module for the products based on the deployment modes demonstrated - Docker Swarm, Kubernetes, and on-premise environments.
•Built and delivered connectors for Hadoop and MindSphere by creating suitable data models for seamless integration with eQ’s data migration (eQubeMI) and analytics (eQubeBI) products, leading to the implementation of cubes and processes to facilitate and orchestrate ingestion, ETL, and reporting.
•Demonstrated ingestion of streaming data, collected from F1 game (version 2017) run on PS4, into MindSphere's Data Lake using eQubeMI with MindSphere connector.
•Created cubes to demonstrate real-time reports based on the ingested data, on eQubeBI using MindSphere connector. Tracked the location of the car and other metrics like wear of tyre, speed, engine and brake temperatures, etc.
•Designed and built a multi-node, Docker-based Hadoop data lake to consolidate different data stores and facilitate ingestion and reporting using heterogeneous data pipelines built using Apache Spark and eQubeMI.
•Developed APIs and an efficient data pipeline to access aggregated datasets in RStudio for predictive maintenance use cases. Reduced the fetch time by 65% in comparison to the most commonly available solution - Sparklyr.
Tech Stack: Java, R, Apache Hadoop, Spark, MapReduce, Kafka, Zookeeper, Hive, MySQL, HBase, Oracle 11g, Spring Boot, Spring Cloud, Eureka, Nginx, JUnit, Docker Swarm, Kubernetes, OpenShift.