TY - GEN
T1 - Context-Aware Regression Test Selection
AU - Chen, Yizhen
AU - Chaudhari, Ninad
AU - Chen, Mei Hwa
N1 - Publisher Copyright: © 2021 IEEE.
PY - 2021
Y1 - 2021
N2 - Most modern software systems are continuously evolving, with changes frequently taking place in the core components or the execution context. These changes can adversely introduce regression faults, causing previously working functions to fail. Regression testing is essential for maintaining the quality of evolving complex software, but it can be overly time-consuming when the size of the test suite is large, or the execution of the test cases takes a long time. There are extensive research studies on selective regression testing aiming at minimizing the size of the regression test suite while maximizing the detection of the regression faults. However, most of the existing techniques focus on the regression faults caused by the code changes, the impact of the context changes on the non-modified software has barely been explored. This paper presents a context-aware regression test selection (CARTS) approach that not only accounts for the modification of code but also changes in the execution context, including libraries, external APIs, and databases. After a change, CARTS uses the program invariants denoted in the pre- and postconditions of a function to determine if the function is affected by the change and selects all the test cases that executed the modified code as well as the non-modified functions whose preconditions are affected by the change. To evaluate the effectiveness of our approach, we conducted empirical studies on multi-release open-source software and case studies on real-world systems that have ongoing changes in code as well as in the execution context. The results of our controlled experiments show that with an average of 32.5% of the regression test cases, CARTS selected all the fault-revealing test cases. In the case studies, all the fault-revealing test cases were selected by using an average of 25.3% of the regression test suite. These results suggest that CARTS can be effective for selecting fault-revealing test cases for both code and execution context changes.
AB - Most modern software systems are continuously evolving, with changes frequently taking place in the core components or the execution context. These changes can adversely introduce regression faults, causing previously working functions to fail. Regression testing is essential for maintaining the quality of evolving complex software, but it can be overly time-consuming when the size of the test suite is large, or the execution of the test cases takes a long time. There are extensive research studies on selective regression testing aiming at minimizing the size of the regression test suite while maximizing the detection of the regression faults. However, most of the existing techniques focus on the regression faults caused by the code changes, the impact of the context changes on the non-modified software has barely been explored. This paper presents a context-aware regression test selection (CARTS) approach that not only accounts for the modification of code but also changes in the execution context, including libraries, external APIs, and databases. After a change, CARTS uses the program invariants denoted in the pre- and postconditions of a function to determine if the function is affected by the change and selects all the test cases that executed the modified code as well as the non-modified functions whose preconditions are affected by the change. To evaluate the effectiveness of our approach, we conducted empirical studies on multi-release open-source software and case studies on real-world systems that have ongoing changes in code as well as in the execution context. The results of our controlled experiments show that with an average of 32.5% of the regression test cases, CARTS selected all the fault-revealing test cases. In the case studies, all the fault-revealing test cases were selected by using an average of 25.3% of the regression test suite. These results suggest that CARTS can be effective for selecting fault-revealing test cases for both code and execution context changes.
KW - Selective regression testing
KW - program invariants
UR - https://www.scopus.com/pages/publications/85126229025
U2 - 10.1109/APSEC53868.2021.00050
DO - 10.1109/APSEC53868.2021.00050
M3 - Conference contribution
T3 - Proceedings - Asia-Pacific Software Engineering Conference, APSEC
SP - 431
EP - 440
BT - Proceedings - 2021 28th Asia-Pacific Software Engineering Conference, APSEC 2021
PB - IEEE Computer Society
T2 - 28th Asia-Pacific Software Engineering Conference, APSEC 2021
Y2 - 6 December 2021 through 9 December 2021
ER -