SHF: Small: A Versatile, Adaptive Network Architecture for Combating Traffic Heterogeneity in Data Centers

Driven by technology advances, massive data centers consisting of tens or even hundreds of thousands servers have been built as infrastructures by large online service providers, in which the performance of data center networks plays a critical role. Traffic in data center networks (DCNs) generally exhibits tremendous heterogeneous characteristics, at different locations in the network, among various applications, between distinct companyís implementations, and from time to time. On the other hand, today's data center networks still lack sufficient flexibility to handle such huge heterogeneity of traffic in an efficient way. Meanwhile, as virtualization techniques undergo a profound development, cloud applications do not directly run on physical hardware or operating systems. Instead, they are assigned on a simulated operating system layer which is virtualized from a substrate network. Thus, a single infrastructure provider can support numerous virtual machines/networks for different users while keeping each user with a vision that the entire infrastructure is uniquely occupied by the user. To meet the challenges, this research designs a versatile network architecture oriented to the ever-increasing performance demands and heterogeneous traffic characteristics of today's data centers. The architecture provides an adaptive full-stack paradigm which includes interconnect topologies, addressing and routing schemes, and virtualization algorithms. More specifically, the research focuses on several closely coupled issues: (1) design a novel, cost-efficient DCN architecture that can dynamically and systematically handle traffic heterogeneity in data centers; (2) design the corresponding routing algorithms for the architecture under unicast and multicast traffic models; (3) develop efficient network virtualization algorithms in the architecture, including virtual network embedding, virtual network function placement and virtualization over geographically distributed DCNs; (4) conduct a comprehensive performance evaluation through extensive simulations and implementation of the schemes in a realistic network prototype. The research combines theoretical analysis, algorithm design, network optimization, simulation and prototyping techniques to provide a comprehensive working solution that enables high performance next generation DCNs. This research is expected have a profound impact on fundamental design principles of future DCNs. The outcome of this research will not only greatly boost the performance of DCN flexibility and energy efficiency while keeping low cost, but also facilitate numerous cloud computing applications currently hosted in data centers. As cloud computing is penetrating into all aspects of the society, this research will have a broader impact on the society and help change the world.