Verification of microservices using metamorphic testing

Gang Luo; Xi Zheng; Huai Liu; Rongbin Xu; Dinesh Nagumothu; Ranjith Janapareddi; Er Zhuang; Xiao Liu

doi:10.1007/978-3-030-38991-8_10

Verification of microservices using metamorphic testing

Gang Luo, Xi Zheng, Huai Liu, Rongbin Xu, Dinesh Nagumothu, Ranjith Janapareddi, Er Zhuang, Xiao Liu

School of Computing

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

4 Citations (Scopus)

Abstract

Microservices architecture is drawing more and more attention recently. By dividing the monolithic application into different services, microservices-based applications are more flexible, scalable and portable than traditional applications. However, the unique characteristics of Microservices architecture have also brought significant challenges for software verification. One major challenge is the oracle problem: in the testing of microservices, it is often very difficult to verify the test result given a test input, due to the features of wide distribution, heterogeneity, frequent changes, and numerous runtime behaviors. To tackle such a challenge, in this paper, we investigate how to apply metamorphic testing into the verification of microservices-based applications, which is a simple yet effective approach to oracle problem. Empirical studies are conducted to evaluate the performance of metamorphic testing based on real-world microservice applications, against the baseline random testing technique with a complete oracle. The results show that in the absence of oracles, metamorphic testing can deliver relatively high failure-detection effectiveness. Our work demonstrates that metamorphic testing is both applicable and effective in addressing the oracle problem for the verification of microservices, similar to many other application domains.

Original language	English
Title of host publication	Algorithms and architectures for parallel processing
Subtitle of host publication	19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I
Editors	Sheng Wen, Albert Zomaya, Laurence T. Yang
Place of Publication	Cham: Switzerland
Publisher	Springer, Springer Nature
Pages	138-152
Number of pages	15
ISBN (Electronic)	9783030389918
ISBN (Print)	9783030389901
DOIs	https://doi.org/10.1007/978-3-030-38991-8_10
Publication status	Published - 2020
Event	International Conference on Algorithms and Architectures for Parallel Processing (19th : 2019) - Melbourne, Australia Duration: 9 Dec 2019 → 11 Dec 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11944 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	International Conference on Algorithms and Architectures for Parallel Processing (19th : 2019)
Abbreviated title	ICA3PP 2019
Country/Territory	Australia
City	Melbourne
Period	9/12/19 → 11/12/19

Keywords

Automatic test case generation
Microservice
Metamorphic Testing
Mutation testing
Software verification

Access to Document

10.1007/978-3-030-38991-8_10

Cite this

Luo, G., Zheng, X., Liu, H., Xu, R., Nagumothu, D., Janapareddi, R., Zhuang, E., & Liu, X. (2020). Verification of microservices using metamorphic testing. In S. Wen, A. Zomaya, & L. T. Yang (Eds.), Algorithms and architectures for parallel processing: 19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I (pp. 138-152). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11944 LNCS). Springer, Springer Nature. https://doi.org/10.1007/978-3-030-38991-8_10

Luo, Gang ; Zheng, Xi ; Liu, Huai et al. / Verification of microservices using metamorphic testing. Algorithms and architectures for parallel processing: 19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I. editor / Sheng Wen ; Albert Zomaya ; Laurence T. Yang. Cham: Switzerland : Springer, Springer Nature, 2020. pp. 138-152 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{0e4bb32840964d198bbc99d730cb4085,

title = "Verification of microservices using metamorphic testing",

abstract = "Microservices architecture is drawing more and more attention recently. By dividing the monolithic application into different services, microservices-based applications are more flexible, scalable and portable than traditional applications. However, the unique characteristics of Microservices architecture have also brought significant challenges for software verification. One major challenge is the oracle problem: in the testing of microservices, it is often very difficult to verify the test result given a test input, due to the features of wide distribution, heterogeneity, frequent changes, and numerous runtime behaviors. To tackle such a challenge, in this paper, we investigate how to apply metamorphic testing into the verification of microservices-based applications, which is a simple yet effective approach to oracle problem. Empirical studies are conducted to evaluate the performance of metamorphic testing based on real-world microservice applications, against the baseline random testing technique with a complete oracle. The results show that in the absence of oracles, metamorphic testing can deliver relatively high failure-detection effectiveness. Our work demonstrates that metamorphic testing is both applicable and effective in addressing the oracle problem for the verification of microservices, similar to many other application domains.",

keywords = "Automatic test case generation, Microservice, Metamorphic Testing, Mutation testing, Software verification",

author = "Gang Luo and Xi Zheng and Huai Liu and Rongbin Xu and Dinesh Nagumothu and Ranjith Janapareddi and Er Zhuang and Xiao Liu",

year = "2020",

doi = "10.1007/978-3-030-38991-8_10",

language = "English",

isbn = "9783030389901",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer, Springer Nature",

pages = "138--152",

editor = "Sheng Wen and Albert Zomaya and Yang, {Laurence T.}",

booktitle = "Algorithms and architectures for parallel processing",

address = "United States",

note = "International Conference on Algorithms and<br/>Architectures for Parallel Processing (19th : 2019), ICA3PP 2019 ; Conference date: 09-12-2019 Through 11-12-2019",

}

Luo, G, Zheng, X, Liu, H, Xu, R, Nagumothu, D, Janapareddi, R, Zhuang, E & Liu, X 2020, Verification of microservices using metamorphic testing. in S Wen, A Zomaya & LT Yang (eds), Algorithms and architectures for parallel processing: 19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11944 LNCS, Springer, Springer Nature, Cham: Switzerland, pp. 138-152, International Conference on Algorithms and
Architectures for Parallel Processing (19th : 2019), Melbourne, Victoria, Australia, 9/12/19. https://doi.org/10.1007/978-3-030-38991-8_10

Verification of microservices using metamorphic testing. / Luo, Gang; Zheng, Xi; Liu, Huai et al.
Algorithms and architectures for parallel processing: 19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I. ed. / Sheng Wen; Albert Zomaya; Laurence T. Yang. Cham: Switzerland: Springer, Springer Nature, 2020. p. 138-152 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11944 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceeding contribution › peer-review

TY - GEN

T1 - Verification of microservices using metamorphic testing

AU - Luo, Gang

AU - Zheng, Xi

AU - Liu, Huai

AU - Xu, Rongbin

AU - Nagumothu, Dinesh

AU - Janapareddi, Ranjith

AU - Zhuang, Er

AU - Liu, Xiao

PY - 2020

Y1 - 2020

N2 - Microservices architecture is drawing more and more attention recently. By dividing the monolithic application into different services, microservices-based applications are more flexible, scalable and portable than traditional applications. However, the unique characteristics of Microservices architecture have also brought significant challenges for software verification. One major challenge is the oracle problem: in the testing of microservices, it is often very difficult to verify the test result given a test input, due to the features of wide distribution, heterogeneity, frequent changes, and numerous runtime behaviors. To tackle such a challenge, in this paper, we investigate how to apply metamorphic testing into the verification of microservices-based applications, which is a simple yet effective approach to oracle problem. Empirical studies are conducted to evaluate the performance of metamorphic testing based on real-world microservice applications, against the baseline random testing technique with a complete oracle. The results show that in the absence of oracles, metamorphic testing can deliver relatively high failure-detection effectiveness. Our work demonstrates that metamorphic testing is both applicable and effective in addressing the oracle problem for the verification of microservices, similar to many other application domains.

AB - Microservices architecture is drawing more and more attention recently. By dividing the monolithic application into different services, microservices-based applications are more flexible, scalable and portable than traditional applications. However, the unique characteristics of Microservices architecture have also brought significant challenges for software verification. One major challenge is the oracle problem: in the testing of microservices, it is often very difficult to verify the test result given a test input, due to the features of wide distribution, heterogeneity, frequent changes, and numerous runtime behaviors. To tackle such a challenge, in this paper, we investigate how to apply metamorphic testing into the verification of microservices-based applications, which is a simple yet effective approach to oracle problem. Empirical studies are conducted to evaluate the performance of metamorphic testing based on real-world microservice applications, against the baseline random testing technique with a complete oracle. The results show that in the absence of oracles, metamorphic testing can deliver relatively high failure-detection effectiveness. Our work demonstrates that metamorphic testing is both applicable and effective in addressing the oracle problem for the verification of microservices, similar to many other application domains.

KW - Automatic test case generation

KW - Microservice

KW - Metamorphic Testing

KW - Mutation testing

KW - Software verification

UR - http://www.scopus.com/inward/record.url?scp=85082142385&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-38991-8_10

DO - 10.1007/978-3-030-38991-8_10

M3 - Conference proceeding contribution

SN - 9783030389901

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 138

EP - 152

BT - Algorithms and architectures for parallel processing

A2 - Wen, Sheng

A2 - Zomaya, Albert

A2 - Yang, Laurence T.

PB - Springer, Springer Nature

CY - Cham: Switzerland

T2 - International Conference on Algorithms and<br/>Architectures for Parallel Processing (19th : 2019)

Y2 - 9 December 2019 through 11 December 2019

ER -

Luo G, Zheng X, Liu H, Xu R, Nagumothu D, Janapareddi R et al. Verification of microservices using metamorphic testing. In Wen S, Zomaya A, Yang LT, editors, Algorithms and architectures for parallel processing: 19th International Conference, ICA3PP 2019 Melbourne, VIC, Australia, December 9–11, 2019 Proceedings, Part I. Cham: Switzerland: Springer, Springer Nature. 2020. p. 138-152. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-38991-8_10

Verification of microservices using metamorphic testing

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this