Software Engineer @ CarbonArc | MSc in Machine Learning and Data Science @ UC San Diego | Fidelity Investments | IIT Guwahati
Over my professional and academic career, I have acquired expertise in databases, Cloud-based development, Python, Java, C++, Amazon Web Services (AWS), SQL, REST APIs, and related domains.
As a multi-faceted Software Engineer, I have developed expertise across various domains, including Cloud (AWS), REST Frameworks, DevOps, CI/CD, Graph databases, PostgreSQL and more. My key responsibilities include:
1. Building FastAPI services interacting with Neo4j, PostgreSQL, and Iceberg tables to support the Data Platform of Carbon Arc empowering customer insights from raw data sources.
2. Developing backend (FastAPI, PostgreSQL) and frontend components for Carbon Arc’s internal admin portal, streamlining client onboarding, data vendor onboarding and access, and user role and permission management.
3. Integrating third-party consumption tracking services (e.g., Openmeter) to implement subscription and pay-as-you-go billing models for multiple client organizations.
As a graduate student researcher, I have worked on the applications of Generative Modeling in Computational Physics. The goal was to generate particle showers, i.e., the distribution of particles as recorded by a detector after heavy particles were made to collide at CERN's Large Hadron Collider. During my time as a Graduate Student Researcher, I did the following:
1. Developed an enhanced version of a SOTA model based on a Generative Adversarial Transformer Model using PyTorch to generate 3D Point Clouds.
2. Executed data wrangling, cleaning, and pre-processing on a particle shower dataset with ∼100, 000 samples.
3. Solved the issue of vanishing gradients in the neural network by using the method of Straight Through Estimation.
As part of Fidelity's modernization process, I developed cloud-based REST microservces and backend architectures that facilitated the migration of legacy on-premise databases to AWS Cloud. My work played a pivotal role in this transition, enabling many of these applications to go into production and serve over a million customers.
During my tenure, I was responsible for the following:
1. Developing REST APIs using Java that allowed customers to read and modify their data on the Cloud (Amazon Web Services) through various access patterns. These APIs catered to the customers' stock watchlists and trading preferences and exhibited remarkable improvements in scalability and performance when compared to the legacy SOAP APIs. In addition to designing these APIs, I also developed unit and integration tests to ensure their reliability and monitored their performance metrics.
2. Building migration pipelines using multi-node Apache NiFi clusters that facilitated the extraction of data from Oracle databases, transformed it into key-value format, and loaded it into the Cloud databases.
3. Building AWS resources like DynamoDB, S3, and Lambda for data storage in the new systems. My work also included developing a reverse sync pipeline using AWS Lambda and SQS to sync data between the new and old databases to provide a fallback option in case of failures.
During my research internship, I explored the applications of data science in Astrophysics, with the objective of generating insights into the nature of the present-day universe. As part of this internship, I accomplished the following:
1. Analyzed cosmological properties such as density and luminosity distance, using various visualization techniques. I studied how these properties behaved with respect to an astronomical parameter known as red-shift, which helped me gain a better understanding of the universe's structure and evolution.
2. Utilized maximum likelihood estimation in Python to achieve optimization and obtain optimal solutions for curvature parameters for the present-day universe. This helped me derive accurate conclusions on the nature of the universe and its current state.
Overall, my research internship enabled me to apply data science techniques to a fascinating field such as Astrophysics and helped me gain valuable experience in analyzing complex datasets and drawing meaningful insights from them.