Testing students' ability to use derivatives, integrals, and vectors in a purely mathematical context and in a physical context

In this article, we discuss the development and the administration of a multiple-choice test, which we named Test of Calculus and Vectors in Mathematics and Physics (TCV-MP), aimed at comparing students' ability to answer questions on derivatives, integrals, and vectors in a purely mathematical context and in the context of physics. The comparison between the two contexts was achieved by using parallel (isomorphic) questions in mathematics and physics. The final version of the test contains 34 items (17 in a purely mathematical context and 17 in the context of physics) involving different representations (graphs, words, numbers, and formal expressions) of the concepts covered by the test. The test was administered in Spring 2018 to 1252 first-year students enrolled in 23 different degree programs of the School of Science and the School of Engineering of the University of Padua. We assessed the validity, reliability, and discriminatory power of the test both as a whole and at the single-item level, obtaining values within the desired ranges. The analysis of students' answers to individual items and the comparison between parallel mathematics and physics items provides insights into the factors that affect students' ability to use derivatives, integrals, and vectors in the context of introductory physics. We believe that the instrument we have developed can be useful not only for research purposes, but also for instructors and for students.