# MADHURA DIGHE > Software Engineer (Data) Location: San Francisco Bay Area, United States Profile: https://flows.cv/madhuradighe I am a Data Engineer focused on building scalable distributed data systems that power real-time logistics and delivery platforms. My work centers on designing resilient, high-throughput infrastructure that enables: - Real-time data processing at scale - Distributed system reliability and observability - Cost-efficient architecture for high-volume platforms - Performance optimization across large data pipelines I have extensive experience in designing ETL pipelines, databases, and implementing a data warehouse infrastructure. The following are my technical skills: • Programming Languages: Python, SQL, R, SAP ABAP, ABAP on HANA • Databases: MySQL, MS SQL, Oracle, HANA, PostgreSQL, NoSQL (Redis, DynamoDB) • Data Warehouses: Snowflake, Redshift, AWS Elastic MapReduce, Google BigQuery, Microsoft Azure • Big Data technologies and Tools: Kubernetes, Hadoop (HDFS, Hive, Pig, Spark, MapReduce), Docker, Airflow, SAP HANA, RStudio, AWS (Amazon S3, SNS, SQS, Kinesis, EC2, AWS Lambda, Athena), Tableau, GitHub, Grafana • Data science and statistics: Data Mining and Machine learning (R, Python), Text Mining, Time-series forecasting, Exploratory and Predictive modeling, Regression (Logistic, Lasso, ridge, KNN, random forest, decision tree), Google Analytics, SAS Happy to to connect with you! ## Work Experience ### Software Engineer Data @ DoorDash Jan 2023 – Present | Sunnyvale, California, United States ### Software Engineer III @ Walmart Global Tech Jan 2023 – Jan 2023 | Sunnyvale, California, United States ### Software engineer II (Data Products) @ Twitter Jan 2022 – Jan 2023 | San Francisco Bay Area - Led team for building experimentation data infrastructure at Twitter, partnered with data science teams. - Build data warehousing solutions to make twitter’s Ads revenue data available by leveraging the largest-scalable distributed data processing technologies in the world. - Built a petabyte-scale data warehouse on GCP using a big query. Built and maintained legacy and new data pipelines to transfer data between HDFS, Mysql, big query and Druid databases. Utilized tools like BQE, BQR, scalding, Big Query to Druid data migration tools, Apache Airflow for orchestration, Aurora for workflow deployment - Implemented data layer for metrics and dimensions for Experimentation dashboard at twitter, which played critical role in Ship no ship decisions for Ads and user experiments launched by revenue teams Tools used - BigQuery, Dataflow, Airflow, GCP, Scalding, Map Reduce, Scala, Batch Processing, Data Engineering, Hadoop, Druid(Distributed analytics platform), Kafka, Aurora, Data quality watcher (great expectation) ### Software Engineer @ Twitter Jan 2021 – Jan 2021 | San Francisco Bay Area - Developed Thin datasets in Big query on top of core datasets for various product use cases by closely partnering with product teams at twitter. - Built data pipelines to avail data from Hadoop to big query and Druid system for product teams via Airflow, BQE jobs, scalding jobs, druid loaders, BQR jobs etc using python and SQL - Oncall support for various revenue datasets, maintained different legacy data pipelines as needed ### Graduate Research Assistant @ University of Maryland - College of Computer, Mathematical, and Natural Sciences Jan 2020 – Jan 2021 - worked on NSF Granted project that focus on the creation of novel methods and approaches for the development of a privacy-respectful toolkit to identify and characterize the multi-factorial challenges that low-income residents in Baltimore City endure when using public transit. Development of backend infrastructure on AWS for android application and data analysis using AWS services ### Graduate Teaching Assistant @ College of Information Studies (UMD iSchool), University of Maryland Jan 2020 – Jan 2021 - Working as a Grader for INST354 - decision making for information science (undergrad) for the entire fall of 2020. ### Data Engineer Intern @ Shipt Jan 2020 – Jan 2020 | San Francisco, California, United States - Build ETL processes, scalable and fault-tolerant data pipelines to ingest data from microservices to Petabyte-scale Snowflake data lake for data science-Tableau dashboarding team using AWS, Kafka, Fivetran, Python and SQL - Developed dimensional models for coupons microservice in Snowflake Marketing Datawarehouse for better analysis and visualization of trends of promotion assignment and redemptions per campaign per customer using Tableau - Developed and deployed SQL scripts for parsing JSON data in data lakes coming from multiple microservices and exposed them for analysis in a data warehouse. Created views based on the requirement and PI constraints (SSN, address, etc.) if any. - Designed tasks, and streams in Snowflake for data ingestion from data on AWS S3 external stage. - Collaborated with other teams across the organization (e.g., Partner Success, Data Science) to enable the better use and understanding of data. Tools used - DataGrip, Snowflake, Fivetran, Grafana, Kubedashian, Github, Lucidchart, Docker, Kubernetes, Kafka, Airflow, AWS (Amazon S3, SNS, SQS, Kinesis, EC2, AWS Lambda, Athena, Redshift, Dynamodb) ### Graduate Teaching Assistant - Market Microstructure ( Python and R ) @ University of Maryland - Robert H. Smith School of Business Jan 2020 – Jan 2020 | College Park, Maryland, United States ### Graduate Teaching Assistant - Marketing Data Science ( SAS/SQL) @ University of Maryland - Robert H. Smith School of Business Jan 2020 – Jan 2020 | College Park, Maryland ### Graduate Teaching Assistant - Finance programming in Python @ University of Maryland - Robert H. Smith School of Business Jan 2020 – Jan 2020 | college park, Maryland ### Business Technology Analyst @ Deloitte Jan 2018 – Jan 2019 | Mumbai Area, India - Analyzed time-critical programs and functionalities for Siminn an Iceland based telecommunication client. Performed S/4 HANA System Migration using S/4HANA Migration cockpit from SAP ECC 6.0 to S/4HANA 1709 - Re-designed reports, Smartforms, Adobe Forms, data dictionary objects using ABAP SQL HANA on Eclipse and Migrated data (ETL) using SAP DATA services, Business object data services (BAPI/LSMW) and BDCs ### Data Engineer/ SQL (ABAP) developer @ Tata Consultancy Services Jan 2016 – Jan 2018 | Powai - Development of new custom processes for SAP modules like sales & distribution, Finance, Material Management, Warehouse Management, SAP basis & security, EDI, and Production planning. Along with new implementations, I supported the old configurations and process by performing bug fixing, batch job monitoring, IDOC monitoring, and maintenance activities on daily basis. - Resolved issues related to SAP Objects like BAPIs, BADIs, SAP enhancements, ALV, DDIC, BDCs, SAP workflow, and authority objects. - Worked on SAP PI integration with the ECC system and IDOC related issues. Over time gained knowledge of IDOC data coming from PI to ECC in the form of XML files. Worked on XML related issues for the interface of SAP PI to ECC for different vendor and customer-specific applications. - As a part of the BOSE, Virtual chain optimization project, I worked on sub-part of SAP data migration project. Developed objects to migrate data from one BOSE ECC system to a new client's (Flex) specific system. Developed reports and processes to format data and extract from ECC system and upload it to the application directory (AL11). From Directory then data was extracted in the form of excel files and uploaded to a new system for use. - Worked on Process improvement for the customer's container ID process. Worked from the technical side with the functional team to fix the loopholes in container ID tracking and reporting it to vendors. This process improvement helped the business to save crucial business hours and cost. - Migrated data (ETL) from BOSE SAP ECC to the BAAN system for vendor Flextronics. - Developed and automated ERP applications using AGILE Methodology that saved 24 business hours per week. Gained knowledge on SAP FIORI, SAP ARIBA, CRM, EPO snd SAP GTS, SAP notes and Adobe forms. ## Education ### Master's degree in Management Information Systems, General University of Maryland - Robert H. Smith School of Business ### Bachelor of Engineering - BE in Electrical, Electronics and Communications Engineering University of Mumbai ## Contact & Social - LinkedIn: https://linkedin.com/in/mdighe - Portfolio: https://balto.umd.edu/people/ --- Source: https://flows.cv/madhuradighe JSON Resume: https://flows.cv/madhuradighe/resume.json Last updated: 2026-03-29