Data ingestion in lacework's cloud security platform

1. Increased the resource coverage velocity 20+ times by automation and code-generation in 6 months.

2. Added real-time monitoring, tracing, and alerting into lacework’s resource collection system; This effort has significantly reduced our time to detect incomplete data collection and speed up the time on mitigating the issues.

3. Designed, implementing a configuration system allowing the system to dynamically read configs from database, and hence paved the way for our system to be self-serve

Logging: designed core data encoding, public logging API, and sampling strategy for Meta’s logging platform. Helped saved 2%+ global cpu (~$6M capex saving) with type safe API redesign.

Data schematization: Changed the entire data warehouse’s serialization format from hive text serialization into thrift based binary serialization. Migrated 100% of data logging into the new system.

Tracing and metrics governance: Designed and implemented explicit tracing library for integrity review system; Initiated and designed automatic metrics governance system.

Ingestion: Worked on auto scaling resource manager of data ingestion jobs in the entire warehouse, which ingests ~4B events/sec, 4TB/s data using 30k machines

Working on SDN, NFV, and cloud solutions for access networks. Mainly working with open source projects, e.g., ONAP, ODL, ONOS, OpenCORD, OpenStack, Docker, OVS, OCP, etc.

*SDN: high-efficient intent-based SDN controllers for access networks taking advantage of the tree topology. We do POC demo of the efficiency of our controller and sharding algorithms.

*NFV: investigated, developed, and compared performances of VNF performances with OVS and VPP.

*Cloud: developed micro-service based virtual network function by using various open source components such as consul, zookeeper, kafka, docker swarm etc.

Awards:

4 patents pending on SDN controllers for access networks and in-band network telemetry.

Network BU "Golden Network" Award - 2017

Mainly working on the design and development of ASIC-based packet forwarding engine software, a key component of Arista's Extensible Operating System (EOS). I have implemented the following features in Arista's switches:

1. advanced port mirroring feature in Arista’s 7150 series switches.

2. buffer management in Arista’s 7050 series switches.

3. SNMP support for shared-memory queue counters, Arista-queue-mib definition and implementation.

4. optimized the TCAM usage in programming ACLs in the chip.

My specific tasks include:

• Gather functional and technical requirements; write functional specifications and design specifications.

• Independently implement solutions for packet forwarding engine using C/C++ and python in Linux.

• Write test plans and implement automated test programs to execute the cases in the test plans.

• Triage and fix problems found by test programs and problems occurring at customers’ networks.

• Maintain multiple infrastructural packages in the Arista EOS modular software system, to ensure robustness, efficiency and manageability of these packages.