# Vishal Vyas > Principal Software Engineer @ Amazon AWS Analytics (EMR, Glue, Athena) | Microsoft Azure | Columbia University | Morgan Stanley Location: San Francisco Bay Area, United States Profile: https://flows.cv/vishalvyas * Principal software engineer with 15+ years of experience at Top Technology companies like Amazon (AWS), Microsoft (Azure), Columbia University (GRA Scholar) and Morgan Stanley (Equity Swaps / Trading Systems). * With my tenured industry experience in large scale distributed systems and Big Data, I can lead entire team and deliver complex enterprise level solutions which are scale-able, robust and has optimum performance. * Ivy League Graduate - MS in Computer Science from Columbia University, New York. My studies focused on Machine Learning, Natural Language Processing (NLP), Computer Vision, Cloud computing and distributed systems. * Expert in Open source Big Data Technologies like Spark, Yarn, Trino, HDFS, Mapreduce with impactful features/ changes to their source codes. * Expert in Distributed system and led systems that ran multi billion dollar businesses. *** Distributed Technologies: Redis, Elasticache, Kafka, MSK, Zookeper, Ec2, EventBus, SNS, SQS, SWF / Step Functions, S3, Cloudwatch, Journal DB / Write ahead Log. *** No SQL DBs : Dynamo DB, MongoDB, Casandra, Level DB. *** Relational DB's: Redshift, RDS, Aurora, MySQL, HSQL, IBM DB2, Sybase. * Currently i am a Principal Software Engineer at AWS Analytics (EMR, Glue, Athena) and I am solving complex System designs to help improve Scale / Performance and Experience of Big Data Analytics Customers. * At Azure I added support for IPv6 (IPv6 CA) in Azure VPN / SDN infrastructure which enabled millions of users to use IPv6 on Azure. My work was related to one of the largest distributed system in Azure Networking (SDN) which manages millions of Virtual Machines and Hosts through Software Defined Networking (SDN). * During Internship at Amazon AWS I delivered the MVP release for AWS Customer insights portal which is used even till date by thousands of internal Users. * At Morgan Stanley I worked for 5 years solving complex finance and software engineering problems across elite teams like Equity Swaps Trading System and Regulatory Reporting System. Do reach out to me to know more. I will be glad to answer any of your queries. Vishal Vyas (646) 667-6347 vnv2102@columbia.edu ## Work Experience ### Principal Software Engineer @ Amazon Web Services (AWS) Jan 2017 – Present | San Francisco Bay Area Tenured Principal Engineer (PE) in AWS Analytics Org (EMR, Glue, Athena). Solving Exciting new Challenges for AWS Elastic MapReduce (EMR) / Glue / Athena. Here's some of my past work where i was a tech lead and helped drive the product end to end : 1). EMR Managed Scaling : Saves huge cluster costs (upto 60%) for EMR customers https://aws.amazon.com/blogs/big-data/introducing-amazon-emr-managed-scaling-automatically-resize-clusters-to-lower-cost/ 2). EMR Log Encryption with KMS: Improves Security for sensitive EMR cluster logs on S3. https://aws.amazon.com/about-aws/whats-new/2020/07/amazon-emr-supports-encrypting-log-file-with-aws-kms-customer-management-cmks-with-more-flexible-security/ ### Senior Software Engineer @ Microsoft Jan 2016 – Jan 2017 | Greater Seattle Area ✔ Innovating in Software Defined Networking Space at Microsoft Azure / Microsoft Cloud. ✔ One of the largest Distributed Systems in World (managing Millions of Nodes) ✔ In free time I also develop creative Machine Learning and Natural Language Processing solutions. ### Masters in Computer Science @ Columbia University in the City of New York Jan 2014 – Jan 2015 | Greater New York City Area ### Graduate Research Assistant @ Columbia University in the City of New York Jan 2014 – Jan 2015 | Greater New York City Area Working on research project with Prof. Junfeng Yang and Yinzhi Cao in Machine Unlearning and Dynamic Code Analysis Project at Columbia University. ### Software Development Engineer @ AWS Solution Architecture @ Amazon Jan 2015 – Jan 2015 ✔ Full Stack Developer. I delivered a one stop customer insights portal for AWS customers to be used by Solutions Architects, Support, Sales, BizDev users. ✔ Developed CORAL services. Added SigV4 Authentication and Aws account based whitelisting and service throttling. ✔ Deployed this 3 tier portal on AWS cloud with auto-scaling and released to alpha and beta. ★★★ Technology :- Java, Ruby on Rails, RedShift Database, Web Services (CORAL), Bootstrap Dashboard, Spring, AWS Sig V4 Authentication. ### Senior Software Engineer @ Morgan Stanley Jan 2010 – Jan 2014 | Mumbai Area, India Working on Multiple projects ranging from Trading systems, Regulatory Disclosure Automation, New Greenfield projects from Ideas phase to the successful implementation and launch. ★★★★★★ Project 1: ( Equity Swaps ) ★★★★★★ ✔ Worked on Projects in Equity Swaps where Morgan Stanley ranked in top 2 places across the Global Investment Banks. ✔ Delivered Multiple Projects in Equity Swaps Business to meet Client and ever evolving business needs. ✔ Worked on Swaps Basket Management System to enable key business requirements. ✔ Improved Performance of System by adding support for K.D.B. - in memory caches. ★★★ Technology :- Advanced Java ,Concurrency, Threading, Spring, K.D.B., DB2, MQ, Content Pub. Sub.(CPS), Perl, H2 Database, Spring, Hibernate, Informatica, Sybase, Shell Scripting, Autosys Job Scheduler. ★★★★★★ Project 2: ( Regulatory Apps ) ★★★★★★ ✔ Worked on System to Automate complicated disclosures filed by Morgan Stanley to U.S. Securities and Exchange Commision (SEC) and Federal Reserve (FED) ✔ Designed and developed ETL solutions involving technologies including Informatica, Perl, Shell. ✔ Familiar with Relational databases (DB2, Sybase). ✔ Key Member in the team that helped the firm to meet multiple MRA's given by F.E.D related to Regulatory Apps. ✔ Ability to Tune SQL to optimize performance. ✔ Ability to Analyze and solve problems at their root, stepping back to understand the broader context. ✔ Handled Multiple high priority requests from senior business leaders. ✔ Continuosly improved ongoing reporting and analysis processes, automated and simplified ETL and Validation processes. ✔ Excellent communication skills - Articulated with the client on design / architecture considerations, best practices etc. ✔ Job tasks include entire SDLC Cycle. ### Software Engineer @ Neebal Technologies Jan 2009 – Jan 2009 ✔ Developed project for an Indian Pharma Company - Piramal Healthcare. ✔ Performed Analysis, Design, Development, Integration Testing and Release Management of the various modules / components. ✔ Delivered technical solution efficiently with effective use of design concepts (optimized SQL or OO concepts or reusable code, etc), ✔ Also worked on portal's performance optimization for quicker and seamless page loads. ★★★ Technology :- Java , Spring, Struts , Hibernate, JSP. ### Software Engineer @ PromoPundit.com Jan 2007 – Jan 2007 | Enterpreneur Cell @ Sardar Patel Institute of Technology, My work involved website optimization by working on various aspects including image optimization, css / scripts compression, html optimization , using CSS Sprites for improving response of site. ## Education ### Master's Degree in Computer Science Columbia University ### Diploma in Computer Technology in Computer Science B.G.I.T. , Mumbai ### Bachelor Of Engineering in Information Technology, IT / Computers Sardar Patel Institute Of Technology, Mumbai ## Contact & Social - LinkedIn: https://linkedin.com/in/vishalvyas30 - Portfolio: https://www.crunchbase.com/person/vishal-vyas-6db0 - Portfolio: https://dl.acm.org/doi/10.1145/3722212.3724443 - Portfolio: https://www.amazon.science/publications/managed-resource-scaling-in-amazon-emr --- Source: https://flows.cv/vishalvyas JSON Resume: https://flows.cv/vishalvyas/resume.json Last updated: 2026-04-12