As professional educators we produce a large number of assessments for our students to complete. These assessments or exams are often evaluated informally based upon student feedback and simple measures such as average score on a question. This paper highlights another more rigorous approach to item evaluation, and presents an evaluation of several items from an assessment as examples of the type of information that Item Response Theory can provide.

We propose a variety of models for incorporating learning over time into the cognitive assessment modeling framework. In two of the models, we use Item Response Theory (IRT; VanDerLinden and Hambleton 1997) where we assume that a continuous latent parameter measures a student’s general proficiency in the area of interest. In the other two models, we use Cognitive Diagnosis Models (CDMs; Rupp and Templin 2008) where we estimate whether students possess a set of skills as the latent student parameter. In all four models, we assume that students take multiple exams in the same content area over a period of time and that at each time point, we are interested in tracking their learning. Therefore, the models consider what the students knew at the previous time point when estimating their current knowledge. With this information, we believe we can make better predictions about end of year, high-stakes exam scores and inform teachers of areas where students are struggling. We may also be able to compare different methods of teaching to find ones that most promote learning and make some statements about the true rate and variability with which students learn which could help teachers, researchers, and policy makers set more realistic goals for students. Each model is discussed both empirically and mathematically. In a simulation study of one model, the parameters describing student learning were recovered with 94.6% accuracy.

Several security protocols require a human to compare two hash values to ensure successful completion. When the hash values are represented as long sequences of numbers, humans may make a mistake or require significant time and patience to accurately compare the hash values. To improve usability during comparison, a number of researchers have proposed various hash representations that use words, sentences, or images rather than numbers. This is the first work to perform a comparative study of these hash comparison schemes to determine which scheme allows the fastest and most accurate comparison. To evaluate the schemes, we performed an online user study with more than 400 participants. Our findings indicate that only a small number of schemes allow quick and accurate comparison across a wide range of subjects from varying backgrounds.