| Description |
In our ESEC/FSE-17 paper titled A Feasibility Study of Using Automated Program Repair for Introductory Programming Assignments, we apply four state-of-the-art automated program repair (APR) tools to student programs collected from an Introductory C Programming course (CS-101) offered in Indian Institute of Technology Kanpur (IIT-K). To overcome the low repair rate of APR tools (due to that student programs are often severely incorrect), we introduce a new repair policy and strategy tailored to programming tutoring (described in Section 6 of our paper), which is implemented in our toolchain. In this repository, we share the artifacts we used in our study. Our artifacts consist of (1) dataset containing student programs, (2) toolchain, and (3) user study materials (we conducted a user study with students and teaching assistants to see the feasibility of using APR tools).
|
| Related Publication |
@inproceedings{10.1145/3106237.3106262,
author = {Yi, Jooyong and Ahmed, Umair Z. and Karkare, Amey and Tan, Shin Hwei and Roychoudhury, Abhik},
title = {A feasibility study of using automated program repair for introductory programming assignments},
year = {2017},
isbn = {9781450351058},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3106237.3106262},
doi = {10.1145/3106237.3106262},
abstract = {Despite the fact an intelligent tutoring system for programming (ITSP) education has long attracted interest, its widespread use has been hindered by the difficulty of generating personalized feedback automatically. Meanwhile, automated program repair (APR) is an emerging new technology that automatically fixes software bugs, and it has been shown that APR can fix the bugs of large real-world software. In this paper, we study the feasibility of marrying intelligent programming tutoring and APR. We perform our feasibility study with four state-of-the-art APR tools (GenProg, AE, Angelix, and Prophet), and 661 programs written by the students taking an introductory programming course. We found that when APR tools are used out of the box, only about 30\% of the programs in our dataset are repaired. This low repair rate is largely due to the student programs often being significantly incorrect - in contrast, professional software for which APR was successfully applied typically fails only a small portion of tests. To bridge this gap, we adopt in APR a new repair policy akin to the hint generation policy employed in the existing ITSP. This new repair policy admits partial repairs that address part of failing tests, which results in 84\% improvement of repair rate. We also performed a user study with 263 novice students and 37 graders, and identified an understudied problem; while novice students do not seem to know how to effectively make use of generated repairs as hints, the graders do seem to gain benefits from repairs.},
booktitle = {Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering},
pages = {740–751},
numpages = {12},
keywords = {Intelligent Tutoring System, Automated Program Repair},
location = {Paderborn, Germany},
series = {ESEC/FSE 2017}
}
|