Scenario-based test reduction and prioritization for multi-module autonomous driving systems

Yao Deng, Xi Zheng*, Mengshi Zhang, Guannan Lou, Tianyi Zhang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference proceeding contributionpeer-review

12 Citations (Scopus)

Abstract

When developing autonomous driving systems (ADS), developers often need to replay previously collected driving recordings to check the correctness of newly introduced changes to the system. However, simply replaying the entire recording is not necessary given the high redundancy of driving scenes in a recording (e.g., keeping the same lane for 10 minutes on a highway). In this pa- per, we propose a novel test reduction and prioritization approach for multi-module ADS. First, our approach automatically encodes frames in a driving recording to feature vectors based on a driving scene schema. Then, the given recording is sliced into segments based on the similarity of consecutive vectors. Lengthy segments are truncated to reduce the length of a recording and redundant segments with the same vector are removed. The remaining seg- ments are prioritized based on both the coverage and the rarity of driving scenes. We implemented this approach on an industry- level, multi-module ADS called Apollo and evaluated it on three road maps in various regression settings. The results show that our approach significantly reduced the original recordings by over 34% while keeping comparable test effectiveness, identifying almost all injected faults. Furthermore, our test prioritization method achieves about 22% to 39% and 41% to 53% improvements over three baselines in terms of both the average percentage of faults detected (APFD) and TOP-K.

Original languageEnglish
Title of host publicationESEC/FSE 2022 - Proceedings of the 30th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering
EditorsAbhik Roychoudhury, Cristian Cadar, Miryung Kim
Place of PublicationNew York, NY
PublisherAssociation for Computing Machinery, Inc
Pages82-93
Number of pages12
ISBN (Electronic)9781450394130
DOIs
Publication statusPublished - 9 Nov 2022
Event30th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022 - Singapore, Singapore
Duration: 14 Nov 202218 Nov 2022

Publication series

NameESEC/FSE 2022 - Proceedings of the 30th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Conference

Conference30th ACM Joint Meeting European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/FSE 2022
Country/TerritorySingapore
CitySingapore
Period14/11/2218/11/22

Keywords

  • Autonomous Driving
  • Regression Testing
  • Test Prioritization
  • Testing Reduction

Fingerprint

Dive into the research topics of 'Scenario-based test reduction and prioritization for multi-module autonomous driving systems'. Together they form a unique fingerprint.

Cite this