SHF: Small: Massively Parallel Server Processors

Ever-growing demand for online services places data centers under constant pressure to achieve higher performance and improved energy efficiency. However, server CPUs have not made significant improvements in performance and efficiency in more than a decade. Notably, during the same time frame, GPUs made breakthrough improvements in both performance and energy efficiency in the Artificial-Intelligence / Machine-Learning (AI/ML) and High-Performance-Computing (HPC) domains. These massive improvements in AI/ML and HPC come from GPUs embracing the Single-Instruction-Multiple-Threads (SIMT) organization. This project aims to demonstrate the benefits of applying the SIMT organization to the server CPUs responsible for running online services. The majority of the energy spent by modern server computing is not in the functional units, but in the peripherals responsible for fetching, decoding, and scheduling instructions and memory accesses. The SIMT organization amortizes these overheads across many concurrent threads of execution. This project leverages the observation that busy high-performance servers frequently service similar or even identical requests concurrently, leading to the opportunity of applying SIMT across server threads. This research offers a detailed cross-layer study of the microarchitecture and memory system toward the development of a high-performance energy-efficient SIMT server architecture. The project will yield innovations in SIMT microarchitecture (branch prediction, out-of-order execution, register files and other core parameters), memory subsystems (cache and memory hierarchy, virtual memory, coherence, instruction and data prefetch), and operating systems (vector system calls and thread scheduling) in the context of server systems. The team plans extensive hands-on training of students in cross-stack (Architecture, Operating System, Compilers) research work. Public infrastructure development and release for exploration of server-space micro-architecture studies are planned. The simulator that will be developed has the potential to become a community resource for processor parameter design space explorations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.