| Description |
In this data repository we store the data for the paper Discovering and quantifying misconceptions in formal methods using intelligent tutoring systems. The paper describes a quantitative study to analyze candidates for misconceptions at modeling with propositional logic, modal logic, and first-order logic. For this, we use data from the intelligent tutoring system Iltis (https://iltis.cs.tu-dortmund.de).
This repository includes the following data:
for each type of logic, which statements were to be modeled and which operators they contained (see this page),
for each statement, its difficulty and popularity (see this page), and
the data for the analyses discussed in the paper, that is the difficulty of single linguistic operators and typical mistakes (see this page and this page). |
| Related Publication |
"@inproceedings{10.1145/3545945.3569806,
author = {Schmellenkamp, Marko and Latys, Alexandra and Zeume, Thomas},
title = {Discovering and Quantifying Misconceptions in Formal Methods Using Intelligent Tutoring Systems},
year = {2023},
isbn = {9781450394314},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3545945.3569806},
doi = {10.1145/3545945.3569806},
abstract = {In this paper we advocate the study of misconceptions in the formal methods domain by integrating quantitative and qualitative methods. In this domain, so far, misconceptions have mostly been studied with qualitative methods, typically via interviews with less than 20 subjects. We discuss workflows for (1) determining the commonness of qualitatively established misconceptions by quantitative means; and for (2) the initial discovery of misconceptions by quantitative methods followed by qualitative assessments.Parts of these workflows are then applied to a data set for exercises on logical modeling from the intelligent tutoring system Iltis with > 250 data points for many of the exercises. We analyze the data in order to (1) determine the commonness of qualitativelyidentified misconceptions on modeling in propositional logic; and to (2) discover typical mistakes in modeling in propositional logic, modal logic, and first-order logic.},
booktitle = {Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1},
pages = {465–471},
numpages = {7},
keywords = {formal methods, first-order logic, propositional logic, modal logic, misconceptions, intelligent tutoring system},
location = {Toronto ON, Canada},
series = {SIGCSE 2023}
}" |