• Design and develop FortiSIEM Application Server features to enable technology migration to Fortinet security appliance.

• Design and implement large distributed security data analysis systems running on Big Data based technologies (Elasticsearch, Clickhouse, HDFS, Spark).

• Design and implement of data service to manage data on Spark and HDFS with Lambda streaming architecture using Java and Python.

• Deliver top quality, well organized, and algorithmically sound code on time.

• Track official releases and technical documents, analyze issue reports and provide fix patches or workaround accordingly.

Participate in the design and implementation of a unified data acquisition and processing platform, using the platform to reform multiple data services across Mastercard.

Tasks Performed:

• Developed a distributing Merchant Location Lookup Service on near real-time transactions.

• Designed and developed an Intelligent Matching Engine for Aggregate Merchant Service.

• Designed and developed a Rule Engine for Merchant Attributes data quality check and enhancement.

• Built a lightweight Data Lake Platform to Support Mastercard core transaction process.

• Developed multiple pipelines for DWH ETL batch jobs, including merchant url enhancement, merchant data sets transformation to Azure Blob storage, etc.

Achievements:

• Build a modern data lake platform with on-promise object storage, make it production ready in three months.

• Developed and Tested new architecture for near real-time Merchant Location Service on transactions, made it on production in two months. This NRT Merchant Location Service provided help to Mechants business during the COVID-19 and Hurricane Laura.

Technologies Used: Java, Oracle, Hadoop, NiFi, Livy, Spark, MinIO, Hive, Bert, Milvus

Tasks Performed:

• Built data center framework using Hadoop, HBase, Spark and Redis

• Managed the cluster over 50 nodes over Alibaba cloud platform

• Built Rest API and interfaces with python for the data center, support other information systems

• Built the data pipeline with python and shell scripts to transfer, process, and maintain large-scale and complex financial data (in tens of TB), sourced from RDBMS, data file, and log file

• Built model with PySpark to conduct time-series predication on the cash flow of mutual fund, linear regression on customers transaction trends, classification customer profiles, and provided solid data support for business decisions

• Built portal with Plotly to demonstrate data analysis result

Achievements:

• Projects included Yu'ebao Data Center, as well as Transfer Agent, Office Automation, and Application Monitoring systems.