Generating comprehension checking problems using mutation analysis compensating for shakyo-style learning defects: A case study on binary operators with evaluating its effectiveness

When learning some programming language it is often the case that novice programmers input some sample code and check the execution results. This is called Shakyo-Style learning, which tends to be a monotonous task without thinking and thus results in insufficient understanding of the syntax and semantics of the language. In this study, we present a new learning method with making minute changes to source code. The changes are made by the techniques in mutation test. In this work we change some binary operators in the original source code and let novice programmers write the standard output of the generated code. Based on the method, we developed a tool and evaluated it by an experiment, which shows that the group with learning by the tool in addition to Shakyo-Style learning improved learning effectiveness, compared with the group with solving exercises similar to problems in the examinations in the experiments. Furthermore, by the results of questionnaires, we found that our method improves the motivation of novice programmers. We also report the implementation of the tool.