Higher education research in STEM is a critical and evolving domain. However, it often relies heavily on college student samples, particularly those who have already signaled STEM interest via their majors or extracurricular activities, limiting the generalizability of findings. New digital learning platform based research infrastructures offer advantages that complement traditional recruitment and data collection approaches. We utilized the [Platform] research platform to validate a biographical data (or biodata) measure of STEM and other early academic life experiences, recruiting online adult learners with no specific predisposition toward STEM. This sample spans broader interests and a wider age range than traditional college‐aged STEM students. As with many measure validation studies, our current instrument is relatively long, raising concerns about participant fatigue and data quality. The [Platform] infrastructure enables administration of convergent and discriminant validity measures (vocational interest, STEM interest, and personality traits) at different times, reducing common‐method variance and respondent fatigue. We conducted an exploratory factor analysis of a seven-factor model on online adult learners (n = 467), finding that five factors (general parental support, STEM parental support, teacher support, math extracurricular engagement, and science extracurricular engagement) loaded as expected, while the remaining two were reconceptualized into factors comprising items that capture community STEM engagement and intrinsic hands-on engagement with STEM. We describe our interpretability assessment process and item refinement decisions for future validation efforts. Finally, we highlight additional advantages and considerations for researchers and learning scientists using large-scale platforms for measurement studies.
Over the past half-century, Holland’s RIASEC model has dominated vocational interest research. Although the RIASEC categories effectively capture general occupational themes, their breadth obscures meaningful variability across underlying basic interests , which are more refined interest scales. In this research, we adapted the Comprehensive Assessment of Basic Interests (CABIN; Su et ., 2019) to be used alongside O*NET for assessing interest fit, resulting in the 60-item CABIN-NET that measures 20 basic interest scales organized under RIASEC domains. In Study 1, we refined items for the CABIN-NET using a nationally representative adult sample ( N = 768). In Studies 2 and 3, we collected extensive reliability and validity evidence using longitudinal samples of graduates from four-year universities ( N = 816) and community colleges ( N = 560). Across samples, results consistently showed that basic interest fit had stronger predictive power for career outcomes compared to RIASEC fit. Overall, the CABIN-NET provides a short, reliable measure of both basic interests and RIASEC interests, offering two ways of assessing person-occupation fit and balancing bandwidth and fidelity. The CABIN-NET also offers a new content-based connection between basic interests and occupational knowledge, providing theoretical and practical implications for incorporating basic interests into person-environment fit research and applications.
BACKGROUND:Team-based primary care has become the norm within many large healthcare systems; however, limited guidance exists on how to optimally staff primary care teams in relationship to healthcare. OBJECTIVE:This paper examines the associations between variations in team staffing configurations on primary care access and clinical quality. DESIGN:Observational study linking national Veterans Health Administration (VHA) data from February 2020 on primary care team staffing configurations to data on access to and quality of care the teams delivered. PARTICIPANTS:We examined data from 22,390 primary care personnel assigned to 7050 teams from 1050 VA Medical Centers and Community-Based Outpatient Clinics across the USA. MAIN MEASURES:We used data from VHA's Corporate Data Warehouse. We assessed team-based measures of overall adherence to VHA's national guidelines for front-line clinical team staffing based on achievement of recommended staffing configurations in terms of quantity of staff and diversity of professional roles. To measure staffing sufficiency, we integrated total number of team members (team size) with their full-time equivalents (FTEs). To measure role diversity, we assessed deviation from guidelines using network analysis of staffing data. As outcomes, we used three measures of patient access to primary care and four measures of clinical quality that were prioritized by a prior expert panel. We analyzed associations between predictors and outcomes using random intercept multilevel models, with teams nested within healthcare facility. KEY RESULTS:Variation in team size and FTE reflected lack of adherence to VHA guidelines rather than normal variation. Overall adherence to VHA guidelines was unrelated to access or quality of care delivered. In most cases, teams with higher FTEs exhibited better outcomes. Increased role diversity was associated with decreased secure messaging communication ratios. Teams with more members exhibited improved 2-day post-hospital discharge contact, but reduced access through third next available appointments. CONCLUSIONS:Primary care teams require a minimum amount of FTE staff capacity to deliver high quality and access to healthcare. Future work should examine how these associations vary by specific job role to further optimize staffing configurations.
Artificial Intelligence (AI) is significantly reshaping work settings, influencing the context, conditions, and content of various professional roles. It becomes crucial to assess AI’s effect on academic work. This study explores AI’s application within teaching and research tasks in academia. Specifically, it pursues two Objectives (1) to identify and describe both current and prospective AI systems in higher education, and (2) to characterize the opportunities and risks of integrating AI into academic environments. Interviews were conducted with 28 participants from Portugal, the Netherlands, and the United States. The questions addressed AI’s influence on Ethical Principles and Decent Work Dimensions. Results were analyzed considering the Socio-Technical Systems Approach. Interviews were coded, analyzed for sentiment, and clustered into seven participant profiles based on coding similarities: “Optimists,” “Moderates,” “Dreamers,” “Cautious Skeptics,” “Expansionists,” “Knowledgeable,” and “Strategists.” Findings emphasize the importance of aligning technology and human needs to achieve successful AI integration. They also point to the value of well-defined guidelines, fair funding, and continuous professional development. By illustrating the spectrum of attitudes and readiness levels among academic stakeholders, this study offers key insights for policymakers, administrators, and educators seeking to embrace AI while preserving Ethical Principles and Decent Work standards.
Despite their potential as human proxies, LLMs often fail to generate heterogeneous data with human-like diversity, thereby diminishing their value in advancing social science research. To address this gap, we propose a novel method to incorporate psychological insights into LLM simulation through the Personality Structured Interview (PSI). PSI leverages psychometric scale-development procedures to capture personality-related linguistic information from a formal psychological perspective. To systematically evaluate simulation fidelity, we developed a measurement theory grounded evaluation procedure that considers the latent construct nature of personality and evaluates its reliability, structural validity, and external validity. Results from three experiments demonstrate that PSI effectively improves human-like heterogeneity in LLM-simulated personality data and predicts personality-related behavioral outcomes. We further offer a theoretical framework for designing theory-informed structured interviews to enhance the reliability and effectiveness of LLMs in simulating human-like data for broader psychometric research.
This research developed the Occupational Values Inventory (OVI), a 30-item measure that expands the content coverage of work values by linking into a broad range of O*NET occupational descriptors, including Work Activities and Work Contexts, along with salary data from the Bureau of Labor Statistics. The OVI assesses 11 work values— Interpersonal , Outdoor , Physical , Leadership , Salary , Prestige , Variety , Interest , Work Hours , Knowledge Utilization , and Autonomy —providing an updated and refined coverage of work values relevant to people and occupations in the modern labor market. Study 1 ( N = 768) developed and refined the OVI’s scales using conceptual and psychometric criteria, also reporting initial evidence in support of reliability and validity. Studies 2 and 3 further evaluated the reliability and criterion-related validity of the OVI based on longitudinal samples of recent graduates from universities ( N = 816) and community colleges ( N = 560). OVI scales and profiles were correlated with occupational aspirations and choice; work value fit with O*NET occupations correlated with subjective career outcomes including career choice satisfaction. The OVI can provide objective assessment of work values fit between people and a wide range of occupations, and may be used in a range of vocational research and applied settings.
When assessing adverse impact, the four-fifths rule (a measure of practical significance of the impact ratio) and ZD test (a statistical significance test of the difference in selection proportions) continue to be widely used in practice, despite disadvantages of using these two measures either in isolation or together in a disjointed manner. This study presents a novel approach that improves upon these problems by estimating a Bayesian impact ratio, which reflects a posterior probability distribution of the most probable values of the impact ratio. We examine the Bayesian impact ratio via a simulation that captures a range of selection scenarios with realistically varying parameters (e.g., total applicants, percentage of people from the protected class, and percentages of people hired from both subgroups). Our Bayesian priors follow a hypothetical court case by representing objective, weak, and strong plaintiff- and defendant-oriented assumptions of an adverse impact case. We demonstrate how to interpret the results of the Bayesian impact ratio, consider model sensitivity when evaluating different Bayesian priors, and make conclusions based on small samples, following the legal burden of proof (a preponderance of the evidence). Compared with the four-fifths rule and ZD test, we conclude that the Bayesian impact ratio reflects a more integrated and useful statistical approach when determining the presence of adverse impact with potential for clearer communication of results.
Objective: Sensory behaviors are common clinically relevant features of many neurodevelopmental disorders including Autism. However, existing assessments critically lack granularity in evaluating hypersensitivity and hyposensitivity, particularly in relation to stimulus intensity. We address this gap by developing the Sensory Response Questionnaire Phenotyper (SRQP), a parent questionnaire that comprehensively assesses sensory behaviors across five sensory modalities. Here we report the development of the SRQP, validation against established tools and assessment of its effectiveness in identifying nuanced sensory processing patterns in children with Autism.Methods: A convenience sample of 317 participants (57 with Autism, 260 typically developing) aged 3-17 years were enrolled in a cross-sectional study. The SRQP’s psychometric properties were analyzed using classical test theory (CTT), and item response theory (IRT). Statistical tests were used to evaluate patterns of sensory processing from SRQP results.Results: Data from 271 participants were included. The SRQP demonstrated strong psychometric properties: a receiver operating characteristic (ROC) curve analysis identified 37 as the cut-off value for most accurately distinguishing between typical and atypical responses on the SRQP (sensitivity 0.75, specificity 0.79, AUC 0.87). Total but not modality-specific hyper vs hyposensitivity scores were moderately correlated (ρ = .5). Responses to high-intensity stimuli were not correlated with responses to low-intensity stimuli for auditory, visual, taste, or tactile hyposensitivity (ρ < .4).Conclusion: The SRQP is a new validated parent questionnaire to assess multi-dimensional properties of sensory behaviors. Stimulus intensity is an important parameter of sensory behaviors that should be accounted for in future models.
Differences in employee evaluations due to gender bias may be small in any given rating cycle, but they may accumulate to produce large disparities in the number of women and men promoted to the top of an organization. A highly cited simulation by Martell et al. (1996) demonstrates this cumulative advantage process in a multilevel organization. We replicated this simulation to uncover important details about its operating assumptions, and we extended the simulation to examine a range of variables that may impact the cumulative effects of gender bias. The replication revealed that the male cumulative advantage in the Martell et al. simulation requires (a) decades of typical promotion cycles to produce, (b) constant mean differences in the performance ratings of women and men but equal within-group variances, and (c) attrition that occurs randomly at a low and constant rate. Our extended simulation demonstrates that (a) cumulative effects of gender bias are higher when the attrition rate is lower, (b) gender biases are mitigated when attrition is more strongly associated with good or poor performance, and (c) the cumulative effects of mean gender differences in performance ratings can often be smaller than the cumulative effects of variance differences between gender subgroups. Results suggest that talent development and recognition of high performers might have a greater positive impact on female representation at top levels of a firm than programs aimed at reducing bias in employee evaluations. We encourage additional simulation work that further explores the dynamics of cumulative advantage in employment settings. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Vocational interest assessments are widely used to determine which jobs might be a good fit for people. However, showing a good fit to particular jobs does not necessarily mean that those jobs are available. In this respect, little is known about the alignment between people’s vocational interests and national labor demands. The current study used a national dataset comprising 1.21 million United States residents to investigate this issue empirically. Results revealed three major findings. First, around two-thirds of people were most interested in people-oriented jobs (i.e., artistic, social, or enterprising interests), with the remaining one-third being most interested in things-oriented jobs (i.e., realistic, investigative, or conventional interests). Second, the distribution of people’s interests did not align with U.S. job demands in 2014, 2019, and 2029 (projections), revealing large gaps between interest supply and demand. Notably, the most popular interest among people (artistic) was the least demanded among jobs, whereas the least popular interest among people (conventional) was highly demanded among jobs. Third, interest gaps were generally larger at lower education levels, indicating that higher education can provide more opportunities to achieve interest fit at work. We integrate these findings to discuss implications for individuals, organizations, and career guidance practitioners aimed at better coordinating people’s interests with available jobs to promote individual career success and national workforce readiness.
We examine the language capabilities of language models (LMs) from the critical perspective of human language acquisition. Building on classical language development theories, we propose a three-stage framework to assess the abilities of LMs, ranging from preliminary word understanding to complex grammar and complex logical reasoning. Using this framework, we evaluate the generative capacities of LMs using methods from linguistic research. Results indicate that although recent LMs outperform earlier models in overall performance, their developmental trajectory does not strictly follow the path of human language acquisition. Notably, in generation tasks, LMs are more similar to human performance in areas where information is easier to extract from the corpus, such as average word length, clauses, and auxiliary verbs. Newer LMs did not exhibit significant progress in terms of specific dimensions, such as clauses and auxiliary verbs, where the variation across corpora is relatively limited. Register theory offers a plausible explanation for these observations, suggesting that the linguistic features of the training data have a substantial impact on the models' abilities.
Research on automation and the future of work is a major focus for both academics and practitioners due to technological changes disrupting the labor market and educational pathways. Although recent articles have published projections about the types of tasks and jobs most likely to be automated in the coming years, little attention has been devoted to how different types of vocational interests are susceptible to automation, as well as resulting changes to the match between people's interests and their jobs. In the present article, we provide an integrative review of vocational interests and automation projections within and across jobs. By standardizing and mapping projections to Holland's RIASEC interest model, we found that Investigative (scientific) and Conventional (detail-oriented) interests, including STEM interests, are most susceptible to automation, whereas Social (people-oriented) and Realistic (hands-on) interests are least susceptible. For Artistic and Enterprising interests, some creative work, decision-making, and leadership skills may be affected by automation across a range of jobs. We build on these projections to propose a future research agenda integrating interests, technology, and careers. Specifically, we identify five areas for future research, including using intentional work design to enhance interests, the role of interests in career decisions related to project-based work, changes in people's interests following automation, increased use of basic interests, and the systematic impacts of automation on different groups of people. Overall, this review highlights how vocational interests will remain an important topic with high relevance for career guidance, education, and organizations as the future of work evolves.
Purpose/Objective: Small sample sizes are a common problem in disability research. Here, we show how Bayesian methods can be applied in small sample settings and the advantages that they provide. Method/Design: To illustrate, we provide a Bayesian analysis of employment status (employed vs. unemployed) for those with disability. Specifically, we apply empirically informed priors, based on large-sample (N = 95,593) July 2019 Current Population Survey (CPS) microdata to small subsamples (average n = 26) from July 2021 CPS microdata, defined by six specific difficulties (i.e., hearing, vision, cognitive, ambulatory, independent living, and self-care). We also conduct a sensitivity analysis, to illustrate how various priors (i.e., theory-driven, neutral, noninformative, and skeptical) impact Bayesian results (posterior distributions). Results: Bayesian findings indicate that people with at least one difficulty (especially ambulatory, independent living, and cognitive difficulties) are less likely to be employed than people with no difficulties. Conclusions/Implications: Overall, results suggest that Bayesian analyses allow us to incorporate known information (e.g., previous research and theory) as priors, allowing researchers to learn more from small sample data than when conducting a traditional frequentist analysis.
Abstract Background Team-based primary care (PC) enhances the quality of and access to health care. The Veterans Health Administration (VHA) implements team-based care through Patient Aligned Care Teams (PACTs), consisting of four core members: a primary care provider, registered nurse (RN) care manager, licensed vocational nurse, and scheduling clerk. RNs play a central role: they coordinate patient care, manage operational needs, and serve as a patient point of contact. Currently, it is not known how varying levels of RN staffing on primary care teams impact patient outcomes. Objective This study aims to empirically assess how the stability of RN staffing within team-based primary care affects patient access to care. Methods A retrospective database review using clinical and administrative data from the VHA over 24 months. Participants included 5,897 PC PACTs across 152 VHA healthcare facilities in the United States and its territories. The stability of personnel in the RN role was categorized as: RN continuous churn, RN staffing instability and RN vacancy. All 3 categories were compared to teams with RN stability (i.e., same person in the role for the entire 24-month period). Access measures included: average third-next-available appointment, established patient average wait time in days, urgent care utilization, emergency room utilization, and total inbound-to-outbound PC secure messages ratio. Results RN continuous churn within PACTs had a significant impact on third-next-available appointment (b = 3.70, p < 0.01). However, RN staffing instability and vacancy had no significant relationship with any of the access measures. Several risk adjustment variables, including team full-time equivalency, team stability, relative team size, and average team size, were significantly associated with access to health care. Conclusions Teams are impacted by churn on the team. Adequate staffing and team stability significantly predict patient access primary care services. Healthcare organizations should focus on personnel retention and strategies to mitigate the impact(s) of continuous RN turnover. Future research should examine the relative impact of turnover and stability of other roles (e.g., clerks) and how team members adapt to personnel changes.
Organizations increasingly recognize the importance of including neurodivergent people (e.g., those with attention-deficit/hyperactivity disorder [ADHD], autism, dyslexia) in the workforce. However, research suggests that some selection tools (e.g., measures of conscientiousness) show lower means for those with ADHD, which may carry implications for personnel selection. The three studies reported here address three questions: (1) What is the magnitude of ADHD-based differences in conscientiousness, and are these differences driven by facets with high or low job relevance? (2) Could reframing conscientiousness items within work contexts attenuate group mean differences in conscientiousness? And (3) do work-specific and general conscientiousness measures have different measurement properties for respondents with ADHD? Study 1 surveyed 291 undergraduates, finding those with ADHD scored significantly lower on global conscientiousness and its facets. Study 2 (a mixed-design experiment) had 317 employees complete a work-specific and a decontextualized set of conscientiousness items. Using work-specific conscientiousness items reduced differences in conscientiousness by ADHD status. Study 3 (a between-subjects design, N = 515) experimentally increased the stakes of survey administration to approximate a selection context. Mean differences by ADHD status were present on both work-specific and general items for global conscientiousness and most facets, even under high stakes. However, these results are qualified by findings of measurement noninvariance on general and work-specific conscientiousness facet measures, suggesting scale mean differences by ADHD status may be driven by item content rather than construct-level differences. Together, the findings reinforce a need for ongoing investigation into the implications of using conscientiousness assessments with neurodivergent people. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
Investigating individual differences in human-product interactions can improve the development of usable products for a diverse range of users. This study explores personality and cognitive ability as predictors of usability. A sample of 972 respondents took the Mini-IPIP measure of Big Five personality, a matrix reasoning test for cognitive ability, and the System Usability Scale (SUS) as an assessment of usability across seven commonly used products (Microsoft Word, Google, Amazon, computer, Microsoft Excel, TV remote, microwave). We ran a hierarchical regression analysis, predicting SUS score from demographic variables (gender, age), then individual differences (personality, cognitive ability). The personality traits of emotional stability and openness were statistically and practically significant predictors of SUS across every product, and gender and conscientiousness were significant predictors across many products. Results suggest that the subjective assessment of usability is not impacted by cognitive ability, and that personality traits are small-to-moderate, but consistent, predictors of SUS score.
The importance of data sharing in organizational science is well-acknowledged, yet the field faces hurdles that prevent this, including concerns around privacy, proprietary information, and data integrity. We propose that synthetic data generated using machine learning (ML) could offer one promising solution to surmount at least some of these hurdles. Although this technology has been widely researched in the field of computer science, most organizational scientists are not familiar with it. To address the lack of available information for organizational scientists, we propose a systematic framework for the generation and evaluation of synthetic data. This framework is designed to guide researchers and practitioners through the intricacies of applying ML technologies to create robust, privacy-preserving synthetic data. Additionally, we present two empirical demonstrations using the ML method of generative adversarial networks (GANs) to illustrate the practical application and potential of synthetic data in organizational science. Through this exploration, we aim to furnish the community with a foundational understanding of synthetic data generation and encourage further investigation and adoption of these methodologies. By doing so, we hope to foster scientific advancement by enhancing data-sharing initiatives within the field.
Measuring person-occupation fit serves many important purposes, from helping young people explore majors and careers to helping jobseekers assess fit with available jobs. However, most existing fit measures are limited in that they focus on single individual difference domains without considering how fit may differ across multiple domains. For example, a jobseeker might be highly interested in a job, yet not possess the requisite skills or knowledge to perform the job well. The current research addresses this issue by evaluating an integrative set of person-occupation fit assessments that measure 88 fit dimensions across five domains: vocational interests, work values, knowledge, skills, and personality. These measures were either newly developed or adapted from existing assessments to directly correspond with occupational variables from the Occupational Information Network database. Across three studies with diverse samples, we obtained extensive reliability and validity evidence to evaluate the fit assessments. Results consistently showed that integrating across fit domains led to practical improvements in predictions of relevant outcomes, including career choice and subjective and objective career success. However, some fit measures (i.e., interests and knowledge) were generally more predictive of outcomes than others (i.e., personality), thus warranting greater consideration for use in research and applied contexts. We discuss how our results advance theoretical and practical knowledge concerning the measurement of person-occupation fit in the modern labor market. Moreover, to inspire additional research and applications involving whole-person fit measurement, we made all newly developed fit assessments publicly available, providing guidance for using them with the Occupational Information Network database. (PsycInfo Database Record (c) 2025 APA, all rights reserved).
Research on vocational interests has played an important role in understanding workforce gender disparities. However, current understanding about gender differences in interests is primarily limited to broad RIASEC interest categories that average together differences in narrower interest scales. This study took a refined approach to examine gender differences in 30 basic vocational interests (e.g., medical science, management, social science) using a very large and diverse U.S. sample (N = 1,283,110). Results revealed that gender differences in basic interests are more complex than what can be captured using broad interests alone. There was meaningful variability in the pattern of mean gender differences across basic interests, even those related to the same RIASEC category. Turning to the labor market, we found that gender differences in basic interests showed high convergence with men and women's employment rates in corresponding occupations (r = 0.66). Despite this convergence, there were also discrepancies such that women's actual employment fell short of interest-based predictions in many high-status occupations and in jobs that involve working with tools and machinery. In contrast, fewer men were employed in prosocial occupations than predicted based on their interests. Finally, we examined how gender differences in basic interests varied across intersecting age, ethnicity, and education subgroups. The most striking finding was that gender differences in interests were considerably larger at lower education levels, pointing to specific educational tracks where applied initiatives might have the greatest impact in improving gender representation in the workforce.
An abstract is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.