DeepTC-Enhancer : improving the readability of automatically generated tests

Roy, Devjeet; Zhang, Ziyi; Ma, Maggie; Arnaoudova, Venera; Panichella, Annibale; Panichella, Sebastiano; Gonzalez, Danielle; Mirakhorli, Mehdi

doi:10.1145/3324884.3416622

Please use this identifier to cite or link to this item: https://doi.org/10.21256/zhaw-22413

Publication type:	Conference paper
Type of review:	Not specified
Title:	DeepTC-Enhancer : improving the readability of automatically generated tests
Authors:	Roy, Devjeet Zhang, Ziyi Ma, Maggie Arnaoudova, Venera Panichella, Annibale Panichella, Sebastiano Gonzalez, Danielle Mirakhorli, Mehdi
et. al:	No
DOI:	10.1145/3324884.3416622 10.21256/zhaw-22413
Proceedings:	Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering
Page(s):	287
Pages to:	298
Conference details:	35th IEEE/ACM International Conference on Automated Software Engineering (ASE), Virtual Event, 21-25 September 2020
Issue Date:	2020
Publisher / Ed. Institution:	Association for Computing Machinery
ISBN:	978-1-4503-6768-4
Language:	English
Subjects:	Software testing; Deep learning; Test case generation; Program comprehension; Empirical study
Subject (DDC):	006: Special computer methods
Abstract:	Automated test case generation tools have been successfully proposed to reduce the amount of human and infrastructure resources required to write and run test cases. However, recent studies demonstrate that the readability of generated tests is very limited due to (i) uninformative identifiers and (ii) lack of proper documentation. Prior studies proposed techniques to improve test readability by either generating natural language summaries or meaningful methods names. While these approaches are shown to improve test readability, they are also affected by two limitations: (1) generated summaries are often perceived as too verbose and redundant by developers, and (2) readable tests require both proper method names but also meaningful identifiers (within-method readability). In this work, we combine template based methods and Deep Learning (DL) approaches to automatically generate test case scenarios (elicited from natural language patterns of test case statements) as well as to train DL models on path-based representations of source code to generate meaningful identifier names. Our approach, called DeepTC-Enhancer, recommends documentation and identifier names with the ultimate goal of enhancing readability of automatically generated test cases. An empirical evaluation with 36 external and internal developers shows that (1) DeepTC-Enhancer outperforms significantly the baseline approach for generating summaries and performs equally with the baseline approach for test case renaming, (2) the transformation proposed by DeepTC-Enhancer results in a significant increase in readability of automatically generated test cases, and (3) there is a significant difference in the feature preferences between external and internal developers.
URI:	https://digitalcollection.zhaw.ch/handle/11475/22413
Fulltext version:	Published version
License (according to publishing contract):	CC BY 4.0: Attribution 4.0 International
Departement:	School of Engineering
Organisational Unit:	Institute of Computer Science (InIT)
Published as part of the ZHAW project:	COSMOS – DevOps for Complex Cyber-physical Systems of Systems
Appears in collections:	Publikationen School of Engineering

Files in This Item:

File	Description	Size	Format
2021_Roy-etal_DeepTC-Enhancer_ASE.pdf		1.17 MB	Adobe PDF	View/Open

Show full item record

Roy, D., Zhang, Z., Ma, M., Arnaoudova, V., Panichella, A., Panichella, S., Gonzalez, D., & Mirakhorli, M. (2020). DeepTC-Enhancer : improving the readability of automatically generated tests [Conference paper]. Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, 287–298. https://doi.org/10.1145/3324884.3416622

Roy, D. et al. (2020) ‘DeepTC-Enhancer : improving the readability of automatically generated tests’, in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. Association for Computing Machinery, pp. 287–298. Available at: https://doi.org/10.1145/3324884.3416622.

D. Roy et al., “DeepTC-Enhancer : improving the readability of automatically generated tests,” in Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, 2020, pp. 287–298. doi: 10.1145/3324884.3416622.

ROY, Devjeet, Ziyi ZHANG, Maggie MA, Venera ARNAOUDOVA, Annibale PANICHELLA, Sebastiano PANICHELLA, Danielle GONZALEZ und Mehdi MIRAKHORLI, 2020. DeepTC-Enhancer : improving the readability of automatically generated tests. In: Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering. Conference paper. Association for Computing Machinery. 2020. S. 287–298. ISBN 978-1-4503-6768-4

Roy, Devjeet, Ziyi Zhang, Maggie Ma, Venera Arnaoudova, Annibale Panichella, Sebastiano Panichella, Danielle Gonzalez, and Mehdi Mirakhorli. 2020. “DeepTC-Enhancer : Improving the Readability of Automatically Generated Tests.” Conference paper. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, 287–98. Association for Computing Machinery. https://doi.org/10.1145/3324884.3416622.

Roy, Devjeet, et al. “DeepTC-Enhancer : Improving the Readability of Automatically Generated Tests.” Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering, Association for Computing Machinery, 2020, pp. 287–98, https://doi.org/10.1145/3324884.3416622.