Study of the relationship of bug consistency with respect to performance of spectra metrics

Hua Jie Lee; Lee Naish; Kotagiri Ramamohanarao

doi:10.1109/ICCSIT.2009.5234512

Study of the relationship of bug consistency with respect to performance of spectra metrics

Hua Jie Lee, Lee Naish, Kotagiri Ramamohanarao

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

31 Citations (Scopus)

Abstract

In practice, manual debugging to locate bugs is a daunting and time-consuming task. By using software fault localization, we can reduce this time substantially. The technique of software fault localization can be performed using execution profiles of the software under several test inputs. Such profiles, known as program spectra, consist of the coverage of correct and incorrect executions statement from a given test suite. We have performed a systematic evaluation of several metrics that make use of measurement obtained from program spectra on Siemens Test Suite. In this paper, we discuss how the effectiveness of various metrics degrade in determining buggy statements as the bug consistency (error detection accuracy, q_e) of a statement approaches zero. Bug consistency of a statement refers to the ratio of the number of failed tests executing the statement over the total number of tests executing the statement. We proposed Effect(M) as to measure the effectiveness of these metrics as q_e value varies. We also demonstrate that the q_e (previously not considered as a metric), is just as effective as some of the metrics proposed. We also formally prove that q_e is identical to the metric that Tarantula system uses for bug localization.

Original language	English
Title of host publication	Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009
Place of Publication	Piscataway, NJ
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	501-508
Number of pages	8
ISBN (Print)	9781424445196
DOIs	https://doi.org/10.1109/ICCSIT.2009.5234512
Publication status	Published - 2009
Externally published	Yes
Event	2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009 - Beijing, China Duration: 8 Aug 2009 → 11 Aug 2009

Other

Other	2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009
Country/Territory	China
City	Beijing
Period	8/08/09 → 11/08/09

Access to Document

10.1109/ICCSIT.2009.5234512

Cite this

Lee, H. J., Naish, L., & Ramamohanarao, K. (2009). Study of the relationship of bug consistency with respect to performance of spectra metrics. In Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009 (pp. 501-508). Article 5234512 Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICCSIT.2009.5234512

@inproceedings{d4b8e009def745db933642aa0ea74628,

title = "Study of the relationship of bug consistency with respect to performance of spectra metrics",

abstract = "In practice, manual debugging to locate bugs is a daunting and time-consuming task. By using software fault localization, we can reduce this time substantially. The technique of software fault localization can be performed using execution profiles of the software under several test inputs. Such profiles, known as program spectra, consist of the coverage of correct and incorrect executions statement from a given test suite. We have performed a systematic evaluation of several metrics that make use of measurement obtained from program spectra on Siemens Test Suite. In this paper, we discuss how the effectiveness of various metrics degrade in determining buggy statements as the bug consistency (error detection accuracy, qe) of a statement approaches zero. Bug consistency of a statement refers to the ratio of the number of failed tests executing the statement over the total number of tests executing the statement. We proposed Effect(M) as to measure the effectiveness of these metrics as qe value varies. We also demonstrate that the qe (previously not considered as a metric), is just as effective as some of the metrics proposed. We also formally prove that qe is identical to the metric that Tarantula system uses for bug localization.",

author = "Lee, {Hua Jie} and Lee Naish and Kotagiri Ramamohanarao",

year = "2009",

doi = "10.1109/ICCSIT.2009.5234512",

language = "English",

isbn = "9781424445196",

pages = "501--508",

booktitle = "Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

note = "2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009 ; Conference date: 08-08-2009 Through 11-08-2009",

}

Lee, HJ, Naish, L & Ramamohanarao, K 2009, Study of the relationship of bug consistency with respect to performance of spectra metrics. in Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009., 5234512, Institute of Electrical and Electronics Engineers (IEEE), Piscataway, NJ, pp. 501-508, 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009, Beijing, China, 8/08/09. https://doi.org/10.1109/ICCSIT.2009.5234512

Study of the relationship of bug consistency with respect to performance of spectra metrics. / Lee, Hua Jie; Naish, Lee; Ramamohanarao, Kotagiri.
Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE), 2009. p. 501-508 5234512.

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Study of the relationship of bug consistency with respect to performance of spectra metrics

AU - Lee, Hua Jie

AU - Naish, Lee

AU - Ramamohanarao, Kotagiri

PY - 2009

Y1 - 2009

N2 - In practice, manual debugging to locate bugs is a daunting and time-consuming task. By using software fault localization, we can reduce this time substantially. The technique of software fault localization can be performed using execution profiles of the software under several test inputs. Such profiles, known as program spectra, consist of the coverage of correct and incorrect executions statement from a given test suite. We have performed a systematic evaluation of several metrics that make use of measurement obtained from program spectra on Siemens Test Suite. In this paper, we discuss how the effectiveness of various metrics degrade in determining buggy statements as the bug consistency (error detection accuracy, qe) of a statement approaches zero. Bug consistency of a statement refers to the ratio of the number of failed tests executing the statement over the total number of tests executing the statement. We proposed Effect(M) as to measure the effectiveness of these metrics as qe value varies. We also demonstrate that the qe (previously not considered as a metric), is just as effective as some of the metrics proposed. We also formally prove that qe is identical to the metric that Tarantula system uses for bug localization.

AB - In practice, manual debugging to locate bugs is a daunting and time-consuming task. By using software fault localization, we can reduce this time substantially. The technique of software fault localization can be performed using execution profiles of the software under several test inputs. Such profiles, known as program spectra, consist of the coverage of correct and incorrect executions statement from a given test suite. We have performed a systematic evaluation of several metrics that make use of measurement obtained from program spectra on Siemens Test Suite. In this paper, we discuss how the effectiveness of various metrics degrade in determining buggy statements as the bug consistency (error detection accuracy, qe) of a statement approaches zero. Bug consistency of a statement refers to the ratio of the number of failed tests executing the statement over the total number of tests executing the statement. We proposed Effect(M) as to measure the effectiveness of these metrics as qe value varies. We also demonstrate that the qe (previously not considered as a metric), is just as effective as some of the metrics proposed. We also formally prove that qe is identical to the metric that Tarantula system uses for bug localization.

UR - http://www.scopus.com/inward/record.url?scp=70449123960&partnerID=8YFLogxK

U2 - 10.1109/ICCSIT.2009.5234512

DO - 10.1109/ICCSIT.2009.5234512

M3 - Conference proceeding contribution

AN - SCOPUS:70449123960

SN - 9781424445196

SP - 501

EP - 508

BT - Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Piscataway, NJ

T2 - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009

Y2 - 8 August 2009 through 11 August 2009

ER -

Lee HJ, Naish L, Ramamohanarao K. Study of the relationship of bug consistency with respect to performance of spectra metrics. In Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009. Piscataway, NJ: Institute of Electrical and Electronics Engineers (IEEE). 2009. p. 501-508. 5234512 doi: 10.1109/ICCSIT.2009.5234512

Study of the relationship of bug consistency with respect to performance of spectra metrics

Abstract

Other

Access to Document

Other files and links

Fingerprint

Cite this