Skip to main navigation Skip to search Skip to main content

Modeling traffic of big data platform for large scale datacenter networks

  • Zhen Xie
  • , Zheng Cao
  • , Zhan Wang
  • , Dawei Zang
  • , En Shao
  • , Ninghui Sun

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

6 Scopus citations

Abstract

Prior to deployment, network designers often use simulators to pre-evaluate the performance of designed network with artificial network traffic. The traditional way of separating network design from real applications will not only result in over-designed network configurations, wasting money and energy, but also miss the real network demands of applications, degrading system performance. In this paper, we provide a method to model the network traffic of current popular big data platforms, which can observably improve the matching between network design and applications. The new method extracts communication behavior from the popular big data applications and replays the behavior instead of the packet traces. Experiments show that the traffic generated by the model is almost match the real traffic and the model can easily scale to thousands of nodes.

Original languageEnglish
Title of host publicationProceedings - 22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016
EditorsXiaofei Liao, Robert Lovas, Xipeng Shen, Ran Zheng
PublisherIEEE Computer Society
Pages224-231
Number of pages8
ISBN (Electronic)9781509044573
DOIs
StatePublished - Jul 2 2016
Event22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016 - Wuhan, Hubei, China
Duration: Dec 13 2016Dec 16 2016

Publication series

NameProceedings of the International Conference on Parallel and Distributed Systems - ICPADS
Volume0

Conference

Conference22nd IEEE International Conference on Parallel and Distributed Systems, ICPADS 2016
Country/TerritoryChina
CityWuhan, Hubei
Period12/13/1612/16/16

Keywords

  • Big data platform
  • Datacenter
  • Hadoop
  • Modeling traffic
  • Storm

Fingerprint

Dive into the research topics of 'Modeling traffic of big data platform for large scale datacenter networks'. Together they form a unique fingerprint.

Cite this