Load management and high availability in the borealis distributed stream processing engine

Nesime Tatbul; Yanif Ahmad; Uǧur Çetintemel; Jeong Hyon Hwang; Ying Xing; Stan Zdonik

doi:10.1007/978-3-540-79996-2_5

Load management and high availability in the borealis distributed stream processing engine

Nesime Tatbul
, Yanif Ahmad
, Uǧur Çetintemel
, Jeong Hyon Hwang
, Ying Xing
, Stan Zdonik

Computer Science

University at Albany

Swiss Federal Institute of Technology Zurich
Brown University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

10 Scopus citations

Abstract

Borealis is a distributed stream processing engine that has been developed at Brandeis University, Brown University, and MIT. It extends the first generation of data stream processing systems with advanced capabilities such as distributed operation, scalability with time-varying load, high availability against failures, and dynamic data and query modifications. In this paper, we focus on aspects that are related to load management and high availability in Borealis. We describe our algorithms for balanced and resilient load distribution, scalable distributed load shedding, and cooperative and self-configuring high availability. We also present experimental results from our prototype implementation showing the effectiveness of these algorithms.

Original language	English
Title of host publication	GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers
Pages	66-85
Number of pages	20
DOIs	https://doi.org/10.1007/978-3-540-79996-2_5
State	Published - 2008
Event	2nd International Conference on GeoSensor Networks, GSN 2006 - Boston, MA, United States Duration: Oct 1 2006 → Oct 3 2006

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	4540 LNCS

Conference

Conference	2nd International Conference on GeoSensor Networks, GSN 2006
Country/Territory	United States
City	Boston, MA
Period	10/1/06 → 10/3/06

Access to Document

10.1007/978-3-540-79996-2_5

Cite this

Tatbul, N., Ahmad, Y., Çetintemel, U., Hwang, J. H., Xing, Y., & Zdonik, S. (2008). Load management and high availability in the borealis distributed stream processing engine. In GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers (pp. 66-85). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4540 LNCS). https://doi.org/10.1007/978-3-540-79996-2_5

Tatbul, Nesime ; Ahmad, Yanif ; Çetintemel, Uǧur et al. / Load management and high availability in the borealis distributed stream processing engine. GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers. 2008. pp. 66-85 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{03d5cf5606b54ed2b93fea3e0a84f58b,

title = "Load management and high availability in the borealis distributed stream processing engine",

abstract = "Borealis is a distributed stream processing engine that has been developed at Brandeis University, Brown University, and MIT. It extends the first generation of data stream processing systems with advanced capabilities such as distributed operation, scalability with time-varying load, high availability against failures, and dynamic data and query modifications. In this paper, we focus on aspects that are related to load management and high availability in Borealis. We describe our algorithms for balanced and resilient load distribution, scalable distributed load shedding, and cooperative and self-configuring high availability. We also present experimental results from our prototype implementation showing the effectiveness of these algorithms.",

author = "Nesime Tatbul and Yanif Ahmad and Uǧur {\c C}etintemel and Hwang, \{Jeong Hyon\} and Ying Xing and Stan Zdonik",

year = "2008",

doi = "10.1007/978-3-540-79996-2\_5",

language = "English",

isbn = "3540799958",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "66--85",

booktitle = "GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers",

note = "2nd International Conference on GeoSensor Networks, GSN 2006 ; Conference date: 01-10-2006 Through 03-10-2006",

}

Tatbul, N, Ahmad, Y, Çetintemel, U, Hwang, JH, Xing, Y & Zdonik, S 2008, Load management and high availability in the borealis distributed stream processing engine. in GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 4540 LNCS, pp. 66-85, 2nd International Conference on GeoSensor Networks, GSN 2006, Boston, MA, United States, 10/1/06. https://doi.org/10.1007/978-3-540-79996-2_5

Load management and high availability in the borealis distributed stream processing engine. / Tatbul, Nesime; Ahmad, Yanif; Çetintemel, Uǧur et al.
GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers. 2008. p. 66-85 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 4540 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Load management and high availability in the borealis distributed stream processing engine

AU - Tatbul, Nesime

AU - Ahmad, Yanif

AU - Çetintemel, Uǧur

AU - Hwang, Jeong Hyon

AU - Xing, Ying

AU - Zdonik, Stan

PY - 2008

Y1 - 2008

N2 - Borealis is a distributed stream processing engine that has been developed at Brandeis University, Brown University, and MIT. It extends the first generation of data stream processing systems with advanced capabilities such as distributed operation, scalability with time-varying load, high availability against failures, and dynamic data and query modifications. In this paper, we focus on aspects that are related to load management and high availability in Borealis. We describe our algorithms for balanced and resilient load distribution, scalable distributed load shedding, and cooperative and self-configuring high availability. We also present experimental results from our prototype implementation showing the effectiveness of these algorithms.

AB - Borealis is a distributed stream processing engine that has been developed at Brandeis University, Brown University, and MIT. It extends the first generation of data stream processing systems with advanced capabilities such as distributed operation, scalability with time-varying load, high availability against failures, and dynamic data and query modifications. In this paper, we focus on aspects that are related to load management and high availability in Borealis. We describe our algorithms for balanced and resilient load distribution, scalable distributed load shedding, and cooperative and self-configuring high availability. We also present experimental results from our prototype implementation showing the effectiveness of these algorithms.

UR - https://www.scopus.com/pages/publications/54249142308

U2 - 10.1007/978-3-540-79996-2_5

DO - 10.1007/978-3-540-79996-2_5

M3 - Conference contribution

SN - 3540799958

SN - 9783540799955

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 66

EP - 85

BT - GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers

T2 - 2nd International Conference on GeoSensor Networks, GSN 2006

Y2 - 1 October 2006 through 3 October 2006

ER -

Tatbul N, Ahmad Y, Çetintemel U, Hwang JH, Xing Y, Zdonik S. Load management and high availability in the borealis distributed stream processing engine. In GeoSensor Networks - Second International Conference, GSN 2006, Revised Selected and Invited Papers. 2008. p. 66-85. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-540-79996-2_5

Load management and high availability in the borealis distributed stream processing engine

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this