The slowing of Moore’s Law and increased concerns about the environmental impacts of computing are exerting pressure on datacenter operators to use resources such as CPUs and memory more efficiently. However, it is difficult to improve efficiency without degrading the performance of applications.

In this talk, I will focus on CPU efficiency and how we can increase efficiency while maintaining low tail latency for applications. The key innovation is to reallocate cores between applications on the same server very quickly, every few microseconds. First I will describe Shenango, a system design that makes such frequent core reallocations possible. Then I will show how policy choices for core reallocation and load balancing impact CPU efficiency and tail latency, and present the policies that yield the best combination of both.