● Work with product managers to design projects and lead development for full scale end to end projects - retailscape and brandscape for customer demos and SAAS launch.
● Work on-call rotation to ensure data collection, data ingestion pipelines and SAAS platforms have 99.99% accuracy/up-time.
● Conduct first round of technical interviews to build out both US and India based engineering and data science teams.
● Provide guidance to new engineers, interns and assist the sales/marketing team with creating slide decks for prospective customers and investment pitch.
● Implement state of the art data lake architecture utilizing AWS S3, AWS Athena and AWS Glue.
● Implement CI/CD pipelines with AWS CodePipeline to reduce deployment cycles.
● Analyze, design and implement data collection streams to support projects and new client requests. Audit these data collection pipelines to flag data quality issues and proactively find and reduce customer issues before they happen.
● Work with CTO, India development team and data auditors to curate our data so that it can be properly ingested and surfaced for use – updating database schemas and handling edgecases within data collection/report generation.
● Integrate 3rd party API’s for data collection and Retailscape platform to assist with anti-bot precautions and to turn enterprise b2b into a SAAS platform.
● Create Kibana visualizations for our marketing and sales team to easily see and share breakdown of data.
● Create anti-bot, data enhancement and data collection microservices to enhance data collection pipelines.
● Led the migration of main database and all platforms from NoSQL (elasticsearch) database to SQL (postgreSQL) database – AWS Aurora and Snowflake.
● Led the migration from csv to parquet file format which increased performance and lowered storage costs.
