Spark Structured Streaming

• Developed course project optimizing code under message passing model using OpenMPI.

• Regularly held office hour to help students debug/profile their codes and offer additional conceptual reviews.

• Developed a service that supports same container task migration for Borg jobs (with Borg public APIs, Spanner DB and Queue, Scaffolding, gMock for testing).

• The service supported Borg job upgrade without rescheduling outside the container, thereby reducing downtime and saving resources when new packages need to be installed or updated.

• Scaled out and improved efficiency by having multiple processors of state machine to support concurrent job updates.

• Worked with Prof. Dimitrios on benchmark the performance increase of hugepage usage in virtual machine.

• Benchmarked MEMCACHED workload in different settings of hugepage usages for both hosts and guests.

• Found out hugepage benefits might come from reduced page walk cycles more than TLB misses, contrary to previous expectation.

• Developed course projects (Bustub Database), designed two phase locking protocols with different

isolation level and autograder testcases.

• Led weekly office hours to help students debug and additional office hours to clarify complex database

concepts