An analysis of node sharing on HPC clusters using XDMoD/TACC-stats

Joseph P. White; Robert L. DeLeon; Thomas R. Furlani; Steven M. Gallo; Matthew D. Jones; Amin Ghadersohi; Cynthia D. Cornelius; Abani K. Patra; James C. Browne; William L. Barth; John Hammond

doi:10.1145/2616498.2616533

An analysis of node sharing on HPC clusters using XDMoD/TACC-stats

Joseph P. White
, Robert L. DeLeon
, Thomas R. Furlani
, Steven M. Gallo
, Matthew D. Jones
, Amin Ghadersohi
, Cynthia D. Cornelius
, Abani K. Patra
, James C. Browne
, William L. Barth
, John Hammond

Center for Computational Research

University at Buffalo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

When a user requests less than a full node for a job on XSEDE's large resources-Stampede and Lonestar4-, that is less than 16 cores on Stampede or 12 cores on Lonestar4, they are assigned a full node by policy. Although the actual CPU hours consumed by these jobs is small when compared to the total CPU hours delivered by these resources, they do represent a substantial fraction of the total number of jobs (~18% for Stampede and ~15% for Lonestar4 between January and February 2014). Academic HPC centers, such as the Center for Computational Research (CCR) at the University at Buffalo, SUNY typically have a much larger proportion of small jobs than the large XSEDE systems. For CCR's production cluster, Rush, the decision was made to allow the allocation of simultaneous jobs on the same node. This greatly increases the overall throughput but also raises questions whether the jobs that share the same node will interfere with one another. We present here an analysis that explores this issue using data from Rush, Stampede and Lonestar4. Analysis of usage data indicates little interference.

Original language	English
Title of host publication	Proceedings of the XSEDE 2014 Conference
Subtitle of host publication	Engaging Communities
Publisher	Association for Computing Machinery
ISBN (Print)	9781450328937
DOIs	https://doi.org/10.1145/2616498.2616533
State	Published - 2014
Event	2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014 - Atlanta, GA, United States Duration: Jul 13 2014 → Jul 18 2014

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014
Country/Territory	United States
City	Atlanta, GA
Period	07/13/14 → 07/18/14

Keywords

HPC
Node sharing
SUPReMM
TACC-Stats
XDMoD

Access to Document

10.1145/2616498.2616533

Cite this

White, J. P., DeLeon, R. L., Furlani, T. R., Gallo, S. M., Jones, M. D., Ghadersohi, A., Cornelius, C. D., Patra, A. K., Browne, J. C., Barth, W. L., & Hammond, J. (2014). An analysis of node sharing on HPC clusters using XDMoD/TACC-stats. In Proceedings of the XSEDE 2014 Conference: Engaging Communities Article 31 (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/2616498.2616533

@inproceedings{07fd69ee7c5e4b74b763f289bfee1710,

title = "An analysis of node sharing on HPC clusters using XDMoD/TACC-stats",

abstract = "When a user requests less than a full node for a job on XSEDE's large resources-Stampede and Lonestar4-, that is less than 16 cores on Stampede or 12 cores on Lonestar4, they are assigned a full node by policy. Although the actual CPU hours consumed by these jobs is small when compared to the total CPU hours delivered by these resources, they do represent a substantial fraction of the total number of jobs (\textasciitilde{}18\% for Stampede and \textasciitilde{}15\% for Lonestar4 between January and February 2014). Academic HPC centers, such as the Center for Computational Research (CCR) at the University at Buffalo, SUNY typically have a much larger proportion of small jobs than the large XSEDE systems. For CCR's production cluster, Rush, the decision was made to allow the allocation of simultaneous jobs on the same node. This greatly increases the overall throughput but also raises questions whether the jobs that share the same node will interfere with one another. We present here an analysis that explores this issue using data from Rush, Stampede and Lonestar4. Analysis of usage data indicates little interference.",

keywords = "HPC, Node sharing, SUPReMM, TACC-Stats, XDMoD",

author = "White, \{Joseph P.\} and DeLeon, \{Robert L.\} and Furlani, \{Thomas R.\} and Gallo, \{Steven M.\} and Jones, \{Matthew D.\} and Amin Ghadersohi and Cornelius, \{Cynthia D.\} and Patra, \{Abani K.\} and Browne, \{James C.\} and Barth, \{William L.\} and John Hammond",

year = "2014",

doi = "10.1145/2616498.2616533",

language = "English",

isbn = "9781450328937",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

booktitle = "Proceedings of the XSEDE 2014 Conference",

address = "United States",

note = "2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014 ; Conference date: 13-07-2014 Through 18-07-2014",

}

White, JP, DeLeon, RL , Furlani, TR, Gallo, SM, Jones, MD, Ghadersohi, A, Cornelius, CD, Patra, AK, Browne, JC, Barth, WL & Hammond, J 2014, An analysis of node sharing on HPC clusters using XDMoD/TACC-stats. in Proceedings of the XSEDE 2014 Conference: Engaging Communities., 31, ACM International Conference Proceeding Series, Association for Computing Machinery, 2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014, Atlanta, GA, United States, 07/13/14. https://doi.org/10.1145/2616498.2616533

An analysis of node sharing on HPC clusters using XDMoD/TACC-stats. / White, Joseph P.; DeLeon, Robert L.; Furlani, Thomas R. et al.
Proceedings of the XSEDE 2014 Conference: Engaging Communities. Association for Computing Machinery, 2014. 31 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - An analysis of node sharing on HPC clusters using XDMoD/TACC-stats

AU - White, Joseph P.

AU - DeLeon, Robert L.

AU - Furlani, Thomas R.

AU - Gallo, Steven M.

AU - Jones, Matthew D.

AU - Ghadersohi, Amin

AU - Cornelius, Cynthia D.

AU - Patra, Abani K.

AU - Browne, James C.

AU - Barth, William L.

AU - Hammond, John

PY - 2014

Y1 - 2014

N2 - When a user requests less than a full node for a job on XSEDE's large resources-Stampede and Lonestar4-, that is less than 16 cores on Stampede or 12 cores on Lonestar4, they are assigned a full node by policy. Although the actual CPU hours consumed by these jobs is small when compared to the total CPU hours delivered by these resources, they do represent a substantial fraction of the total number of jobs (~18% for Stampede and ~15% for Lonestar4 between January and February 2014). Academic HPC centers, such as the Center for Computational Research (CCR) at the University at Buffalo, SUNY typically have a much larger proportion of small jobs than the large XSEDE systems. For CCR's production cluster, Rush, the decision was made to allow the allocation of simultaneous jobs on the same node. This greatly increases the overall throughput but also raises questions whether the jobs that share the same node will interfere with one another. We present here an analysis that explores this issue using data from Rush, Stampede and Lonestar4. Analysis of usage data indicates little interference.

AB - When a user requests less than a full node for a job on XSEDE's large resources-Stampede and Lonestar4-, that is less than 16 cores on Stampede or 12 cores on Lonestar4, they are assigned a full node by policy. Although the actual CPU hours consumed by these jobs is small when compared to the total CPU hours delivered by these resources, they do represent a substantial fraction of the total number of jobs (~18% for Stampede and ~15% for Lonestar4 between January and February 2014). Academic HPC centers, such as the Center for Computational Research (CCR) at the University at Buffalo, SUNY typically have a much larger proportion of small jobs than the large XSEDE systems. For CCR's production cluster, Rush, the decision was made to allow the allocation of simultaneous jobs on the same node. This greatly increases the overall throughput but also raises questions whether the jobs that share the same node will interfere with one another. We present here an analysis that explores this issue using data from Rush, Stampede and Lonestar4. Analysis of usage data indicates little interference.

KW - HPC

KW - Node sharing

KW - SUPReMM

KW - TACC-Stats

KW - XDMoD

UR - https://www.scopus.com/pages/publications/84905441857

U2 - 10.1145/2616498.2616533

DO - 10.1145/2616498.2616533

M3 - Conference contribution

SN - 9781450328937

T3 - ACM International Conference Proceeding Series

BT - Proceedings of the XSEDE 2014 Conference

PB - Association for Computing Machinery

T2 - 2014 Annual Conference on Extreme Science and Engineering Discovery Environment, XSEDE 2014

Y2 - 13 July 2014 through 18 July 2014

ER -

An analysis of node sharing on HPC clusters using XDMoD/TACC-stats

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this