Argumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning-a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the measurement of knowledge and understanding to the measurement of performance and knowledge in use. Performance tasks requiring argumentation must capture the many ways students can construct and evaluate arguments in science, yet such tasks are both expensive and resource-intensive to score. In this study we explore how machine learning text classification techniques can be applied to develop efficient, valid, and accurate constructed-response measures of students' competency with written scientific argumentation that are aligned with a validated argumentation learning progression. Data come from 933 middle school students in the San Francisco Bay Area and are based on three sets of argumentation items in three different science contexts. The findings demonstrate that we have been able to develop computer scoring models that can achieve substantial to almost perfect agreement between human-assigned and computer-predicted scores. Model performance was slightly weaker for harder items targeting higher levels of the learning progression, largely due to the linguistic complexity of these responses and the sparsity of higher-level responses in the training data set. Comparing the efficacy of different scoring approaches revealed that breaking down students' arguments into multiple components (e.g., the presence of an accurate claim or providing sufficient evidence), developing computer models for each component, and combining scores from these analytic components into a holistic score produced better results than holistic scoring approaches. However, this analytical approach was found to be differentially biased when scoring responses from English learners (EL) students as compared to responses from non-EL students on some items. Differences in the severity between human and computer scores for EL between these approaches are explored, and potential sources of bias in automated scoring are discussed.

Background: Meningiomas are the most common intracranial tumor with surgery, dural margin treatment, and radiotherapy as cornerstones of therapy. Response to treatment continues to be highly heterogeneous even across tumors of the same grade. Methods: Using a cohort of 2490 meningiomas in addition to 100 cases from the prospective RTOG-0539 phase II clinical trial, we define molecular biomarkers of response across multiple different, recently defined molecular classifications and use propensity score matching to mimic a randomized controlled trial to evaluate the role of extent of resection, dural marginal resection, and adjuvant radiotherapy on clinical outcome. Results: Gross tumor resection led to improved progression-free-survival (PFS) across all molecular groups (MG) and improved overall survival in proliferative meningiomas (HR 0.52, 95%CI 0.30-0.93). Dural margin treatment (Simpson grade 1/2) improved PFS versus complete tumor removal alone (Simpson 3). MG reliably predicted response to radiotherapy, including in the RTOG-0539 cohort. A molecular model developed using clinical trial cases discriminated response to radiotherapy better than standard of care grading in multiple cohorts (ΔAUC 0.12, 95%CI 0.10-0.14). Conclusions: We elucidate biological and molecular classifications of meningioma that influence response to surgery and radiotherapy in addition to introducing a novel molecular-based prediction model of response to radiation to guide treatment decisions.

Background: Meningiomas have been demonstrated to have significant heterogeneity between patients and even within each WHO grade, making prognostication challenging with current standard of care classifications. We previously developed a DNA methylation-based predictor (PMID 31158293) of meningioma recurrence risk following surgery and validated this in retrospective cohorts. For this study we utilize prospectively collected samples from multiple institutions enriched for biologically aggressive meningiomas to confirm the utility of our predictor in prognosticating meningioma patients and informing selection for adjuvant radiotherapy (RT).

Baseline surveys of offshore pelagic fishes in the eastern Chukchi Sea in 2012 and 2013 found that age-0 Arctic cod (Boreogadus saida) dominated the pelagic fish community in summer, with relatively few adults present in the region. Since this time, drastic changes in the ocean-atmosphere-ice feedback loop have led to continued warming, further reducing ice cover, and increased northward transport has led to an increase in Pacific-origin waters on the Chukchi shelf in summer. To examine potential bottom-up effects of these environmental changes on pelagic fishes in this rapidly changing environment, we extended a time series of large-scale acoustic-trawl surveys with additional surveys in 2017 and 2019. Age-0 Arctic cod were the most abundant pelagic fish in all four survey years, comprising 68–93% of fish abundance. However, age-0 walleye pollock (Gadus chalcogrammus), which were scarce (<0.1% of fishes) and confined to the southern Chukchi in 2012 and 2013, were present in high abundance (>21% of fish abundance) throughout the Chukchi shelf in 2017 and 2019. Age-0 Arctic cod were substantially more abundant in 2017 than in other years, possibly due to increased survivorship of larvae under warm conditions. Unlike in 2017, Arctic cod and pollock were spatially separated in 2019 due to enhanced transport, with Arctic cod primarily present in the northeastern portion of the survey area, which was characterized by cool surface and bottom temperatures. The substantial increase in abundance of age-0 pollock in recent years suggests that environmental conditions now allow this species to extend its northern range into the southern and central Chukchi Sea, at least on a seasonal basis. The changes in abundance and species composition of pelagic fishes in the 2012–2019 time series are tightly coupled to recent changes in sea ice, temperature, and the increasing transport of Bering Sea waters through Bering Strait into the Chukchi Sea. Given that the environment is expected to experience further warming and increased transport, these northward shifts in species distribution are likely to persist in the future.

This chapter details the role that Rasch measurement played in the development of an assessment instrument that can be used to measure the complex science learning described in the Next Generation Science Standards during a large-scale efficacy study. Four model-based reasoning (MBR) tasks were developed and tested along with content-focused (CF) items with high-school biology students, high-school biology teachers, and crowd-sourced adults. Rasch modeling was used to investigate the relative difficulties of the items within the tasks, explore the relationship between performance on the MBR tasks and the CF items, and compare the performance of students, teachers, and adults.

Recent summer surveys of the northeastern Chukchi Sea found pelagic fishes were dominated by large numbers of age-0 Arctic cod (Boreogadus saida, Gadidae) and walleye pollock (Gadus chalcogrammus, Gadidae), while adult fishes were comparatively scarce. The source and fate of these young fishes remain unclear, as sampling in this region is impeded by seasonal ice cover much of the year. Seafloor-mounted echosounders were deployed at three locations in the northeastern Chukchi Sea from 2017 to 2019 to determine the movement and seasonal variability of these age-0 gadids. These observations indicated that the abundance of pelagic fishes and community composition on the Chukchi Sea shelf were highly variable on seasonal time scales, with few fish present in winter. Tracking indicated that fish movements were strongly correlated with local currents. Fishes were primarily displaced to the northeast in summer and fall, with periodic reversals towards the southwest driven by changes in regional wind patterns. The flux of fishes past the moorings indicated that the prevailing northward currents transport a large proportion of the age-0 pelagic fishes present on the Chukchi shelf in summer to the northeast by fall, leading to relatively low abundances of age-1+fishes in this environment.

This study uses Many-Facet Rasch Measurement (MFRM) to examine the extent to which computer scoring models for assessing students’ argumentation in science might be more or less severe when scoring students who have been designated as English Learner (EL) students than humans scoring the same data. We found that while no one machine scoring approach produced significant bias, performance on certain items demonstrated that one machine model had significant potential to widen performance gaps.

Pelagic trawls are one of the primary methods of sampling midwater fishes. However, these trawls are species- and size-selective, and small fish can escape through trawl meshes. This can introduce uncertainty and bias into survey abundance estimates if not accounted for. The small, abundant pelagic fishes of the Alaska Arctic are challenging to sample with trawls as they are sufficiently motile to avoid small fine-mesh trawls but are also small enough to escape through the meshes of trawls designed to capture larger fishes. A pelagic herring trawl equipped with a fine-mesh codend liner was used to quantify the size and species composition of pelagic fishes during a baseline acoustic-trawl survey of the Chukchi Shelf. Subsequent experiments with recapture nets attached to the outside of the trawl netting suggested that escapement of small fishes was substantial, particularly in the aft net section. Thus, the trawl was further modified by reducing the taper in the aft net section and adding a small-mesh section in front of the codend to potentially reduce escapement. Further use of recapture nets during two subsequent acoustic-trawl surveys confirmed that this trawl modification substantially increased retention of small fishes and resulted in less size selectivity. These improvements will reduce biases in estimates of abundance, size, and species composition of pelagic Arctic fishes. This work highlights the importance of quantifying escapement from survey trawls and demonstrates that escapement estimates can guide successful trawl modifications.

Fin Balaenoptera physalus and humpback Megaptera novaeangliae whales share foraging areas and may compete for the same prey, but little is known about the extent to which they partition prey resources. Visual cetacean surveys and simultaneous acoustic-trawl surveys of prey were conducted around 2 submarine canyons off Kodiak Island, Alaska, in 2004 and 2006. Statistical models were used to examine the associations between sightings of fin and humpback whales and measures of their potential prey and environment. Observations and models indicate that fin whales were disproportionately abundant in areas with the highest observed euphausiid concentrations, while humpback whales were abundant at lower euphausiid concentrations and in areas where juvenile walleye pollock were abundant. Fin whales were abundant in the areas where euphausiid biomass was deepest and in the deepest areas surveyed (>150 m depth). In contrast, humpback whales primarily occurred in shallower areas and near more shallowly distributed euphausiids. The different depth and prey affinities of fin and humpback whales suggest niche and habitat partitioning between these 2 co-occurring species. Abundance models built using acoustic estimates of prey density are a useful tool to further understanding of the abundance, distribution, and behavior of these animals.

Estimating and monitoring the construct-irrelevant variance (CIV) is of significant importance to validity, especially for constructed response assessments with rich contextualized information. To examine CIV in contextualized constructed response assessments, we developed a framework including a model accounting for CIV and a measurement that could differentiate the CIV. Specifically, the model includes CIV due to three factors: the variability of assessment item scenarios, judging severity, and rater scoring sensitivity to the scenarios in tasks. We proposed using the many-facet Rasch measurement (MFRM) to examine the CIV because this measurement model can compare different CIV factors on a shared scale. To demonstrate how to apply this framework, we applied the framework to a video-based science teacher pedagogical content knowledge (PCK) assessment, including two tasks, each with three scenarios. Results for task I, which assessed teachers’ analysis of student thinking , indicate that the CIV due to the variability of the scenarios was substantial, while the CIV due to judging severity and rater scoring sensitivity of the scenarios in teacher responses was not. For task II, which assessed teachers’ analysis of responsive teaching , results showed that the CIV due to the three proposed factors was all substantial. We discuss the conceptual and methodological contributions, and how the results inform item development.

Recent summer surveys of the Chukchi Sea determined that pelagic fishes were dominated by large numbers of age-0 Arctic cod and walleye pollock, while adult fishes are comparatively scarce. Modeling based on regional currents indicates that these age-0 fishes are likely advected to the north in fall; however, the source and fate of these fishes remains unclear as this region is seasonally ice-covered. To determine the movement and variability of this age-0 gadid population, bottom-moored multifrequency echosounders were deployed at three locations in the northeastern Chukchi Sea from 2017-2019. These observations indicate that the abundance and composition of the pelagic community on the Chukchi Sea shelf is highly variable over seasonal time scales. Fish abundance was very low in winter, increased in May, and reached peak abundance in late summer. Target strength and diel vertical migration of fishes increased in summer, indicating that this is a key period for growth. Age-0 gadids were displaced to the northeast, consistent with the dominant advection on the shelf. Fish speeds and headings were strongly correlated with local currents, providing evidence that these small age-0 fishes are primarily being passively transported and behavior plays a limited role in population distribution.

Many rockfishes (Sebastes spp.) inhabit rugged areas of seafloor that are inaccessible to survey trawl gear.Their utilization of such habitat makes estimation of their abundance difficult.Furthermore, it is often difficult to assess whether habitat is trawlable or untrawlable and to estimate the spatial extent of both habitat types.To help determine trawlability for the continental shelf in the Gulf of Alaska, we used multibeam sonar data collected in the area during 2011, 2013, and 2015.These data were used to derive 3 characteristics of the seafloor: oblique incidence backscatter strength (S b oblique), seafloor ruggedness, and bathymetric position index.Habitat type was categorized as trawlable or untrawlable through analysis of video from deployed drift cameras.We tested the effectiveness of the use of these seafloor characteristics in prediction of habitat trawlability with 4 types of models: generalized linear model, generalized additive model, boosted regression tree, and random forest.All 4 models perform moderately well at predicting trawlability across the shelf, and results from all of them indicate that S b oblique is the most important characteristic in discriminating between trawlable and untrawlable habitat.These results indicate that multibeam sonar data can help determine habitat type, information that in turn can help improve habitat-specific estimates of biomass of marine fish species.

Fishery-independent surveys, such as bottom trawl surveys, provide time-series abundance estimates, which inform many modern stock assessments. Area-swept biomass estimates from trawl surveys assume that fish densities do not differ between trawlable (T) and untrawlable (UT) areas. Bias and imprecision in the biomass estimates can occur when this assumption is not met. Thus, reliable estimates are needed for both the extent of T and UT habitat types in the surveyed area, and the relative densities of the fish species in the two habitat types to accurately assess groundfish populations. Acoustics and stereo-camera survey tools were used in the present study to determine the extent of T and UT habitat within 25-km(2) bottom trawl survey grid cells historically designated as T/UT. Splitbeam acoustics were used to compare the abundance of rockfishes (Sebastes spp.) between the T/UT grid cell areas. Acoustic data were collected along uniformly spaced transects within 52 T and 43 UT grid cells throughout the Gulf of Alaska during summers 2013, 2015, and 2017. The acoustic backscatter attributed to rockfishes in UT grid cells was approximately three times that in T cells, and the percentages available to the bottom trawl survey were 40 % for harlequin rockfish, 43 % for northern rockfish, 51 % for dusky rockfish, and 98 % for Pacific ocean perch (POP). These findings allowed for estimation of the trawl catchability coefficient (q; a scaler between estimates of the area-swept survey abundance and actual abundance) of 0.46 for harlequin rockfish, 0.50 for northern rockfish, and 0.64 for dusky rockfish, and 1.15 for POP. These values could be used to inform the relationship between trawl survey estimated and actual abundances of rockfishes to improve the accuracy of stock assessments for these species.

Machine learning has been frequently employed to automatically score constructed response assessments. However, there is a lack of evidence of how this predictive scoring approach might be compromised by construct-irrelevant variance (CIV), which is a threat to test validity. In this study, we evaluated machine scores and human scores with regard to potential CIV. We developed two assessment tasks targeting science teacher pedagogical content knowledge (PCK); each task contains three video-based constructed response questions. 187 in-service science teachers watched the videos with each had a given classroom teaching scenario and then responded to the constructed-response items. Three human experts rated the responses and the human-consent scores were used to develop machine learning algorithms to predict ratings of the responses. Including the machine as another independent rater, along with the three human raters, we employed the many-facet Rasch measurement model to examine CIV due to three sources: variability of scenarios, rater severity, and rater sensitivity of the scenarios. Results indicate that variability of scenarios impacts teachers’ performance, but the impact significantly depends on the construct of interest; for each assessment task, the machine is always the most severe rater, compared to the three human raters. However, the machine is less sensitive than the human raters to the task scenarios. This means the machine scoring is more consistent and stable across scenarios within each of the two tasks.

The highly productive northern Bering and Chukchi marine shelf ecosystem has long been dominated by strong seasonality in sea-ice and water temperatures. Extremely warm conditions from 2017 into 2019—including loss of ice cover across portions of the region in all three winters—were a marked change even from other recent warm years. Biological indicators suggest that this change of state could alter ecosystem structure and function. Here, we report observations of key physical drivers, biological responses and consequences for humans, including subsistence hunting, commercial fishing and industrial shipping. We consider whether observed state changes are indicative of future norms, whether an ecosystem transformation is already underway and, if so, whether shifts are synchronously functional and system wide or reveal a slower cascade of changes from the physical environment through the food web to human society. Understanding of this observed process of ecosystem reorganization may shed light on transformations occurring elsewhere. Exceptionally warm years in 2017–2019 have caused changes in the physical and biological characteristics of the Pacific Arctic Ocean. What these changes mean for the ecosystem and societal consequences will depend on if they are evidence of a transformation or anomalies in the system.

Author(s): Lee, HS; McNamara, D; Bracey, ZB; Wilson, C; Osborne, J; Haudek, KC; Liu, OL; Pallant, A; Gerard, L; Linn, MC; Sherin, B | Abstract: Rapid advancements in computing have enabled automatic analyses of written texts created in educational settings. The purpose of this symposium is to survey several applications of computerized text analyses used in the research and development of productive learning environments. Four featured research projects have developed or been working on (1) equitable automated scoring models for scientific argumentation for English Language Learners, (2) a real-time, adjustable formative assessment system to promote student revision of uncertainty-infused scientific arguments, (3) a web-based annotation tool to support student revision of scientific essays, and (4) a new research methodology that analyzes teacher-produced text in online professional development courses. These projects will provide unique insights towards assessment and research opportunities associated with a variety of computerized text analysis approaches.

This study tests the influence of a video-based, analysis-of-practice professional development (PD) program on upper-elementary teachers' science content knowledge, pedagogical content knowledge, and teaching practice and on their students' achievement. Using a cluster-randomized experimental design, the study compares the outcomes for teachers in an analysis-of-practice program with those of teachers in a content-deepening program. Mediational analyses explore the relationship between teacher outcomes and student learning. In comparison with the content-deepening PD program, the analysis-of-practice PD program significantly impacted teachers' knowledge and practice. Mediation analyses revealed a strong relationship between teaching practice and student learning. The study advances the field beyond the currently accepted consensus model of effective PD toward an empirically tested model.

In this exploratory study, we attempted to measure potential changes in teacher knowledge and practice as a result of an intervention, as well as trace such changes through a theoretical path of influence that could inform a model of teacher professional knowledge. We created an instrument to measure pedagogical content knowledge (PCK), studied the impact of a two-year professional development intervention, explored the relationships among teacher variables to attempt to validate a model of teacher professional knowledge, and examined the relationship of teacher professional knowledge and classroom practice on student achievement. Teacher professional knowledge and skill was measured in terms of academic content knowledge (ACK), general pedagogical knowledge (GenPK), PCK and teacher practice. Our PCK instrument identified two factors within PCK: PCK-content knowledge and PCK-pedagogical knowledge. Teacher gains existed for all variables. Only GenPK had a significant relationship to teacher practice. ACK was the only variable that explained a substantial portion of student achievement. Our findings provide empirical evidence that we interpret through the lens of the model of teacher professional knowledge and skill, including PCK [Gess-Newsome, J. (2015). A model of teacher professional knowledge and skill including PCK: Results of the thinking from the PCK summit. In A. Berry, P. Friedrichsen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 28–42). London: Routledge Press], highlighting the complexity of measuring teacher professional knowledge and skill.