I am a software engineer who is excited by all things related to building systems for storing, crunching, and serving data at scale. I come from a background rooted in biomedical imaging, computer vision and statistics, but found my stride and passion in backend engineering.
Created design proposal for horizontal scaling of key web service on Kubernetes using a multilevel cache and async workers to mitigate a single point of failure and reduce request latency by up to two orders of magnitude.
•
Reduced technical debt by 80% for a critical web service by upgrading it to Python 3 (asyncio and type hints), setting up ArgoCD-based deployment, and migrating it to a team-specific Kubernetes namespace and Terraform workspace.
•
Expanded tooling for data-intensive application design by introducing the Yahoo! Cloud Serving Benchmark (Java framework for load testing data stores), including containerizing it for ease of use and extending it to meet team needs.
•
Fixed complex bug in production system causing feature degradation for offloading data from trucks by analyzing metrics and logs, stepping through stack traces, and investigating 3rd party code, reducing downtime by 90%.
•
Developed features and fixed bugs for services across the data platform including Kafka microservices, Airflow pipelines leveraging Apache Spark, and distributed services communicating over RabbitMQ.
Led architecture of data platform on Google Cloud Platform (GCP) enabling scalable and automated processing on petabyte-scale data lake to accelerate deep learning (DL) development and data analytic workflows by up to 80%.
•
Developed data collection features for on-board robotics platform in C++ and ROS 2, reducing CPU consumption by 79%, reducing storage requirements by 67%, and increasing reliability and automation.
•
Implemented horizontally-scalable Kubernetes infrastructure for processing unstructured data (image, video, and ROS bags) consisting of containerized data transforms operating on a Cloud Storage data lake and orchestrated with Dagster.
•
Developed data search system consisting of BigQuery data warehouse, Postgres database, Node.js (Express) RESTful API, and Python client library enabling search and retrieval of data and machine learning (ML) annotations.
•
Developed image content and ML-based data pipelines for prioritization of unlabeled data for annotation, optimizing metrics such as scene object density (85% increase) to improve model training within constrained annotation budget.
•
Organized and managed cloud infrastructure by setting up Terraform for resource management, establishing a team git workflow, standing up an internal Python package repository, and creating documentation for end users.
Developed and optimized data pipelines and machine learning models in Apache Spark (PySpark) operating on tens of millions of transactions resulting in insights leading to up to 45% savings for client.
•
Prototyped interactive dashboard application leveraging Natural Language Processing (NLP) to provide intuitive business development insights. (PostgreSQL, Python, Plotly Dash, NLTK).
•
Architected code and data visualization review processes and standards using git workflows for delivery of data science solutions while operating as part of an Agile, product-oriented team.
Implemented Python scripts to extract and transform geospatial data from web API and FTP sources into ArcGIS, performed statistical modeling using Python (statsmodels, scikit-learn) and R, and visualized and presented results to client’s senior leadership.
•
Developed pdf ingestion and entity/relation-extraction features for Natural Language Processing dashboard application in an Agile team (Python, NLTK, scikit-learn, NetworkX).
•
Deployed Collibra metadata management prototype for client including developing use cases, mapping enterprise architecture and existing metadata stores, and developing metrics and surveys to assess solution performance, demonstrating a 23% increase in productivity.
