Reduced 20% of the false-positive performance anomalies via conducting repetitive analysis on the batch generated anomalies

Built a thread-safe Hive query interface over JDBC in Python with Kerberos authentication support on Hadoop Edge Nodes

Utilized Hive table partitioning to improve read query time when ingesting 3.6 billion data points from Spark into Hive

Eliminated 30% of false positives compared to the baseline model by building a configuration-driven Keras Deep Autoencoder

Constructed a scalable machine learning pipeline processing over 2.5 million data points daily with Kafka and Cassandra

Maintained reliability of the pipeline by achieving over 90% code coverage in automated Jenkins tests with Docker

Reduced Spark executor memory consumption by over 500% for online queries by integrating Cassandra with Spark

Decreased online query latency by more than 20% by developing the Cassandra schema around the historical query patterns

Created a data pipeline that ingests over 3 million historical and current price data points into the Cassandra cluster