Projects per year
Abstract
Deep Neural Network based models are widely used for perception and control in autonomous driving. Recent work leverages metamorphic testing to improve defect detection but is limited to using only an equality-based metamorphic relation. Thus, it does not provide sufficient expressiveness for users to define custom metamorphic relations nor means to automatically generate meaningful inputs based on such expressive metamorphic relations that reflect real-world traffic behaviors. In this paper, we preliminarily design and evaluate a declarative Behaviour-Driven Development (BDD)-based metamorphic testing framework BMT, which enables domain experts to specify custom traffic behaviors—a car shall decelerate by x% when a bicycle is in front, etc. It then automatically translates a human-written behavior to a corresponding metamorphic relation and synthesizes meaningful test inputs using a variety of image and graphics processing techniques. Our preliminary evaluation shows that BMT can detect a significant number of erroneous predictions of three driving models for speed predictions. These detected erroneous predictions are manually examined and confirmed by six human judges as meaningful traffic violations. By automating test generation from custom behaviors, BMT enables experts to easily express domain-specific constraints and finds violations of such constraints.
Original language | English |
---|---|
Title of host publication | Proceedings - 2021 IEEE/ACM 6th International Workshop on Metamorphic Testing, MET 2021 |
Place of Publication | Piscataway, NJ |
Publisher | Institute of Electrical and Electronics Engineers (IEEE) |
Pages | 32-36 |
Number of pages | 5 |
ISBN (Electronic) | 9781665444644 |
DOIs | |
Publication status | Published - Jun 2021 |
Event | 2021 IEEE/ACM 6th International Workshop on Metamorphic Testing - Virtual, Madrid, Spain Duration: 22 May 2021 → 30 May 2021 |
Conference
Conference | 2021 IEEE/ACM 6th International Workshop on Metamorphic Testing |
---|---|
Abbreviated title | MET 2021 |
Country/Territory | Spain |
City | Madrid |
Period | 22/05/21 → 30/05/21 |
Keywords
- Metamorphic Testing
- Autonomous Driving
- Deep Learning Models
- Software Testing and Verification
Fingerprint
Dive into the research topics of 'BMT: behavior driven development-based metamorphic testing for autonomous driving models'. Together they form a unique fingerprint.Projects
- 1 Finished
-
SUT led : Context-aware verification and validation framework for autonomous driving
Chen, T., Vu, H., Liu, H., Zheng, J. & Zhou, Z.
25/02/21 → 24/02/24
Project: Research