Although large-scale assessments (LSA) of school achievement claim to measure domain-specific achievement, they have been criticized for primarily measuring domain-general abilities. Numerous studies provide evidence that LSA of mathematical achievement as well as verbal achievement cover both general cognitive abilities (GCA) and domain-specific achievement dimensions. We extend previous research by analyzing a standards-oriented and literacy-oriented LSA in the domain of science to determine the relation of these two assessment types with domain-general abilities. While literacy-oriented assessments focus on the knowledge and skills students need to meet the demands of modern societies, standards-oriented assessments focus on national educational standards and curricula. A sample of 1722 students worked on three assessments: (a) the PISA scientific literacy assessment; (b) a standards-oriented assessment based on the German National Educational Standards in biology, chemistry, and physics developed by the Institute for Educational Quality Improvement (IQB); and (c) a GCA test. Comparisons of competing structural models showed that models differentiating between domain-specific achievement and GCA best represented the structure of the assessments. Furthermore, standards-oriented and literacy-oriented LSAs in science shared common variance with GCA but also comprised specific variance. In addition to a factor representing students' GCA, we identified a science literacy-oriented and two standards-oriented factors. Relations with school grades in various STEM and non-STEM subjects were mixed and only partly provided evidence for the specificity of science LSAs. Our findings are important for understanding and interpreting results of LSAs in the contexts of GCA and science. We discuss our outcomes with respect to educational monitoring practices.

Evidence on the interrelation of intelligence development and the development of domain-specific academic achievement is still inconclusive. We investigated the longitudinal relation between these 2 constructs in the domains mathematics and reading. Data from 6 large adolescent student samples (Ntotal = 24,828) from 4 longitudinal studies were analyzed using an integrated approach. Continuous time models corroborate the assumption that intelligence and academic achievement are reciprocally related over the course of 9 months, a time period that approximates the length of a school year. Reciprocal relations were observed regardless of the achievement indicator employed (standardized test score or report card grade) and the academic domain. Multigroup analyses demonstrated that the strengths of associations between intelligence and the indicators of academic achievement was robust across sexes. Our rigorous tests of the interrelation between intelligence and academic achievement underscore the importance of adolescents' learning opportunities not only for achievement in academic domains, but for intelligence development more generally.

This study focuses on a topic with a long tradition in educational psychology. In a large data set with several achievement measures we investigated the effects of intelligence and motivation on academic achievement in three domains, namely, German, mathematics, and English, using three different achievement measures (standardized tests, grades, and final written exams) in a sample of upper secondary students (N = 3,775; Grade 13; 54.8 % female; age M = 19.92 years) in Germany. Furthermore, we focused on grade point average (GPA) as a general achievement indicator at the end of upper secondary school. First, we aimed to replicate previous results on the predictive power of intelligence and motivation for achievement. Second, we aimed to extend the large body of existing research by adding final written exams - school-based performance tests - as an additional measure. Our findings indicate that motivation had stronger effects on achievement than intelligence did. This was particularly true for the domain-specific achievement measures. Motivation had the strongest effects on grades, followed by final exams. The effects of intelligence were comparatively stronger for standardized achievement tests. Overall, the findings suggest that both intelligence and motivation are important predictors of achievement and that this is true for all kinds of achievement measures.

One of the most powerful determinants of course selection in upper secondary level is undoubtedly students’ self-concept. Students with a high self-concept in a domain are more likely to select a course in that domain. However, according to the dimensional comparison theory, the formation of self-concept includes comparison processes with self-concepts in other domains. Regarding gender, females are less likely to choose physics and are more likely to have lower STEM self-concepts as well as lower aspirations toward STEM careers than males. In Germany, students in Grade 10 choose specific academic tracks to attend during upper secondary school. The academic track choice goes in hand with choosing advanced courses. This choice entails the decision about whether to pursue STEM subjects. We adopted the person-centered approach of latent profile analysis (LPA) to investigate the patterns of students’ self-concepts across the five domains, math, biology, reading, English, and physics. Furthermore, we investigated how those patterns influence educational choices regarding science subjects in upper secondary school in Germany. Based on a sample of 1,658 students, we tested whether the distinct profiles of self-concept in different domains in Grade 8 predicted gendered science course selection in Grade 10 as well as career aspirations in science. LPAs yielded four distinct profiles of self-concept that differed in level and shape: high math, high verbal, low overall, and high overall. These profiles were equivalent across gender. Gender differences were manifested in the relative distribution across the four profiles: females were more present in the low overall and high verbal-related self-concept profiles and males in the overall high and high math-related self-concept profiles. The profiles differed regarding abilities, choice of science course in upper secondary level, and science career aspirations.

This study focuses on a topic with a long tradition in educational psychology. In a large data set with several achievement measures we investigated the effects of intelligence and motivation on academic achievement in three domains, namely, German, mathematics, and English, using three different achievement measures (standardized tests, grades, and final written exams) in a sample of upper secondary students (N = 3 775; Grade 13; 54.8 % female; age M = 19.92 years) in Germany. Furthermore, we focused on grade point average (GPA) as a general achievement indicator at the end of upper secondary school. First, we aimed to replicate previous results on the predictive power of intelligence and motivation for achievement. Second, we aimed to extend the large body of existing research by adding final written exams - school-based performance tests - as an additional measure. Our findings indicate that motivation had stronger effects on achievement than intelligence did. This was particularly true for the domain-specific achievement measures. Motivation had the strongest effects on grades, followed by final exams. The effects of intelligence were comparatively stronger for standardized achievement tests. Overall, the findings suggest that both intelligence and motivation are important predictors of achievement and that this is true for all kinds of achievement measures.

For Instruction, teachers often rely on prefabricated material that may include irrelevant information. However, graphs can place a heavy burden on the cognitive system if their complexity is not suitable for a given task. In this study, we compared bar graphs showing task-irrelevant data points or task-irrelevant data series with a control condition using a within-subject design and eye tracking methodology. Data were analyzed using linear mixed-effects models. Results show that task-irrelevant data significantly elevated processing time, error rate and cognitive load. Even though perceptual grouping by color was expected to aid the process when a task irrelevant data series was included in a graph, effects were strongest in this condition. Analyses of attention distribution using eye tracking measures revealed that task processing differed qualitatively between the conditions, yielding important implications for instruction.

The study focuses on integration aids (i.e., signals) and their effect on how students process different types of graphical representations (representational pictures vs. organizational pictures vs. diagrams) in standardized multiple-choice items assessing science achievement. Based on text-picture integration theories each type of pictorial representation hold different cognitive requirements concerning integration processes of two representations. Further, depending on type of representation not every picture is needed to answer an item correctly.Students from fifth sixth grade (N = 60) work through 12 multiple choice items while their eye movements were recorded. Results showed that students achieved higher test scores when items were presented in an integrated format than in a non-integrated format, however, this was only true for diagrams. Eye movement data revealed that students looked longer on the graphical representations in items presented in the integrated format condition compared to the non-integrated format condition. Furthermore, relations between looking at the diagrams and achievement in the integrated format emerged. (C) 2017 The Authors. Published by Elsevier Ltd.

This study investigates the interplay between general cognitive abilities and domain-specific mathematical ability by applying three different models. The first model is a g factor model, the second model reflects two correlated factors, and the third model is a nested-factor model that comprises a g factor, a domain-specific mathematical factor and cognitive ability. Furthermore, we analyzed the relation between student characteristics such as self-concept in mathematics, gender, SES, and grade in mathematics. Using data from six different German LSAs of the three cohorts, Grade 5, 9 and 13 (sample sizes range from NGrade5=730 to NGrade9=3893), and two different frameworks (i.e., literacy and curricular), we confirmed that LSAs do test mathematical ability beyond g. The two cognitive factors correlated differently with the student characteristics considered; this calls for future longitudinal approaches to the issue.

Teaching quality often is assumed to be a personal and stable characteristic of teachers. Whether this is true has scarcely been investigated empirically. In this study the extent to which value-added scores of teachers teaching German and English as a foreign language (EFL) to the same class remain consistent across subjects was investigated. Then, the consistency of two teaching quality dimensions—classroom management and motivational support—across subjects was explored. A sample consisting of 25 classes with 548 students to whom German and EFL were taught by the same teacher was analyzed using multivariate multilevel models and generalizability theory. The results showed that the value-added scores were highly correlated across subjects. While there was hardly any subject-dependent variance in classroom management, there was substantial subject-dependent variance in motivational support. The results indicate that it is important to conduct further studies on the situational and contextual factors that might influence teaching quality to gain a more comprehensive picture regarding the consistency of teaching quality across various conditions.

Solving test items might require abilities in test-takers other than the construct the test was designed to assess. Item and student characteristics such as item format or reading comprehension can impact the test result. This experiment is based on cognitive theories of text and picture comprehension. It examines whether integration aids, which relate pictorial representations to the corresponding textual representations in item stimuli, affect performance in a science test. The results show that items containing referential connections between both representations and highlighting associated information are easier to solve than non-integrated items (i.e., items without aids). However, this is only true for information-complementary representations, not for information-equivalent representations. Furthermore, an effect of reading comprehension on students' test performance observed when complementary information was presented in a non-integrated format was absent in the integrated format condition.

Pictures are often used in standardized educational large-scale assessment (LSA), but their impact on test parameters has received little attention up until now. Even less is known about pictures’ affective effects on students in testing (i.e., test-taking pleasure and motivation). However, such knowledge is crucial for a focused application of multiple representations in LSA. Therefore, this study investigated how adding representational pictures (RPs) to text-based item stems affects (1) item difficulty and (2) students’ test-taking pleasure. An experimental study with N = 305 schoolchildren was conducted, using 48 manipulated parallel science items (text-only vs. text-picture) in a rotated multimatrix design to realize within-subject measures. Students’ general cognitive abilities, reading abilities, and background variables were assessed to consider potential interactions between RPs’ effects and students’ performance. Students also rated their item-solving pleasure for each item. Results from item-response theory (IRT) model comparisons showed that RPs only reduced item difficulty when pictures visualized information mandatory for solving the task, while RPs substantially enhanced students’ test-taking pleasure even when they visualized optional context information. Overall, our findings suggest that RPs have a positive cognitive and affective influence on students’ performance in LSA (i.e., multimedia effect in testing) and should be considered more frequently.

Research on graph comprehension suggests that point differences are easier to read in bar graphs, while trends are easier to read in line graphs. But are graph readers able to detect and use the most suited graph type for a given task? In this study, we applied a dual repre-sentation paradigm and eye tracking methodology to determine graph readers’ preferential processing of bar and line graphs while solving both point difference and trend tasks. Data were analyzed using linear mixed-effects models. Results show that participants shifted their graph preference depending on the task type and refined their preference over the course of the graph task. Implications for future research are discussed.

SummaryAlthough pictures are often added to text in items of educational tests, little is known about their influence on item solving. Therefore, we conducted an experiment in which we examined how pictures affected item solving. A total of N = 158 fourth‐grade students completed a physics knowledge test under one of six experimental conditions. The experimental conditions varied according to whether or not pictures were presented in the stem and in the answer options of the test items. The results showed that pictures in the stem and in the answer options increased the correctness with which students responded to the test items. This was particularly true for test items that required the application of relationships. In addition, response time was reduced when pictures were added to the answer options of the test items. Hence, pictures are an important feature of test items that produce changes in item processing. Copyright © 2011 John Wiley & Sons, Ltd.