As artificial intelligence (AI), and particularly Generative AI (GenAI), is increasingly being integrated into education, preparing teachers for the AI era is essential. This study presents the design, implementation, and evaluation of a graduate-level course for in-service STEM teachers; it was designed according to six established principles of effective teacher professional development (PD) and structured around the United Nations Educational, Scientific, and Cultural Organization (UNESCO)'s AI competency framework for teachers (AI-CFT). The course lasted 14 weeks and covered all AI-CFT aspects: human-centered mindset, ethics of AI, AI foundations, AI pedagogy, and AI for PD. A mixed-methods study was used to examine the reflection of AI-CFT aspects in teachers' learning, and their trust and self-efficacy regarding AI use. The findings from two consecutive course offerings (N = 25 and N = 19) revealed significant gains in teachers' self-efficacy and their understanding of the key AI-CFT aspects. Notably, teachers' trust in AI-based educational technology did not increase, while their growing attention to the ethical challenges and limitations of GenAI was in line with the course's emphasis on critical engagement and pedagogical alignment. The research contributes to the emerging area of AI competence development by providing a PD framework that is based on UNESCO's AI-CFT.
Large Language Models (LLMs) are increasingly used in educational settings to enhance assessment and feedback. While prior research has focused primarily on their ability to score responses or model learners’ knowledge, less attention has been given to their use in explaining outputs of machine learning algorithms – a key goal of explainable AI (XAI) in education. In this study, we explore the capacity of ChatGPT to generate natural-language explanations of student knowledge profiles derived from clustering analysis of multi-item chemistry assessments. These explanations are compared to those authored by human experts, with 16 chemistry teachers evaluating both versions in a blind review. While ChatGPT’s explanations were generally preferred for profiles representing simpler student performance patterns, human-authored explanations were favored for more complex profiles requiring nuanced pedagogical reasoning. Our findings highlight the capabilities and limitations of LLMs in generating high-level explanations of algorithmic outputs and suggest that relying on LLMs to analyze multi-item assessment data may actually work against students with more complex knowledge structures.
Nanotechnology involves manipulating matter at the nanoscale (approximately 1–100 nm), where materials often exhibit properties that differ from their bulk forms. Nanomaterials are widely used in medicine, energy, environmental applications, and other fields. Despite its scientific and societal importance, integrating nanoscale science and technology (NST) into secondary education remains challenging. Constraints include limited curricular time, insufficient teacher preparation, and students’ difficulties in understanding scale and size-dependent phenomena. This study describes the development and evaluation of a hybrid Nano Escape Room (Nano EsRm) designed as an informal learning environment for introducing key NST concepts. Grounded in the Six Strands of Science Proficiency framework, the activity integrates gamification, hands-on experiments, digital guidance, and collaborative puzzles. It addresses central nanoscale ideas, including size and scale, size-dependent properties, characterization methods, graphene structure, and real-world applications. Data from 176 students and 49 teachers indicate high student interest and perceived learning gains, with moderate support for conceptual understanding. Teachers evaluated the experience positively, identified the targeted nanoscale concepts, and expressed strong willingness to implement the activity in their classrooms. The findings suggest that structured, game-based informal learning environments provide an accessible approach for introducing complex nanoscale concepts in secondary education.
The rapid adoption of large language models (LLMs) in education raises profound challenges for assessment design. To adapt assessments to the presence of LLM-based tools, it is crucial to characterize the strengths and weaknesses of LLMs in a generalizable, valid and reliable manner. However, current LLM evaluations often rely on descriptive statistics derived from benchmarks, and little research applies theory-grounded measurement methods to characterize LLM capabilities relative to human learners in ways that directly support assessment design. Here, by combining educational data mining and psychometric theory, we introduce a statistically principled approach for identifying items on which humans and LLMs show systematic response differences, pinpointing where assessments may be most vulnerable to AI misuse, and which task dimensions make problems particularly easy or difficult for generative AI. The method is based on Differential Item Functioning (DIF) analysis – traditionally used to detect bias across demographic groups – together with negative control analysis and item-total correlation discrimination analysis. It is evaluated on responses from human learners and six leading chatbots (ChatGPT-4o & 5.2, Gemini 1.5 & 3 Pro, Claude 3.5 & 4.5 Sonnet) to two instruments: a high school chemistry diagnostic test and a university entrance exam. Subject-matter experts then analyzed DIF-flagged items to characterize task dimensions associated with chatbot over- or under-performance. Results show that DIF-informed analytics provide a robust framework for understanding where LLM and human capabilities diverge, and highlight their value for improving the design of valid, reliable, and fair assessment in the AI era.
Differentiated instruction (DI) is widely promoted for addressing learner diversity in science education, yet evidence of its motivational and achievement impact in secondary chemistry remains limited. This study examined the effects of DI on students’ motivational appraisals and learning outcomes in high school chemistry across culturally diverse contexts. DI was implemented through customized pedagogical kits, a tiered tool that diagnoses students’ chemistry misconceptions and assigns matched small-group activities instead of whole-class instruction. A total of 205 students from Arab and Jewish educational sectors in Israel participated in a pre-/post-intervention following a three-tier response to intervention model. Data were collected through validated questionnaires and chemistry achievement tasks and analyzed using hierarchical regression and structural equation modelling (SEM). Results showed significant improvements in achievement, self-efficacy, and attitudes toward DI and non-frontal methods. SEM demonstrated excellent model fit (comparative fit index [CFI] = .997, root mean square error of approximation [RMSEA] = .028), with attitudes mediating contextual influences on achievement. SEM demonstrated excellent model fit (CFI = .997, RMSEA = .028), with attitudes mediating the relationship between contextual factors and achievement. Multi-group analysis indicated partial non-invariance by sector in the Israeli context: the path from attitudes to achievement was stronger among Arab students than among Jewish students. Full invariance was found by gender. These findings position DI as an effective, culturally responsive pedagogy with implications for chemistry-specific differentiation. The paper also discusses the implications of these findings for classroom instruction and for policymakers seeking to reduce sector-based educational inequities.
Many educational programs seek to promote students' pro-environmental attitudes and behaviors, yet few are explicitly designed or examined in light of evidence-based design features known to support such change. This study presents and examines the validity of an evaluation model, the Environmental Attitudes and Behavior Model, through its application to an independently developed intervention program targeting ninth-grade students' attitudes and behaviors related to SDG 13 (Climate Action). The model is grounded in eight design features identified in the literature as effective in fostering pro-environmental attitudes and behaviors. Using a mixed-methods approach, we examined pre-post changes in the attitudes and behaviors of 23 teachers and 127 students through questionnaires, alongside interviews with six teachers and eight student groups. The findings indicate improvements in students' environmental attitudes and behaviors, as well as in their perceived importance of learning about these issues. Analysis of interview data further examined the presence of the model's design features in the intervention, leading to the suggestion of an additional design feature and the refinement of the model into a nine-feature framework. We recommend that evaluations of environmental education programs adopt a dual approach that combines assessment of learning outcomes with analysis of the presence of evidence-based design features that promote pro-environmental attitudes and behavior.
As Large Language Models (LLMs) become increasingly prevalent in science education, it is important to understand their capabilities compared to human learners with respect to authentic learning tasks. Such understanding is crucial for designing AI-resilient assessments and developing AI tutors that can guide students in problem solving. Using standardized assessments as benchmarks allows these comparisons to be based on widely accepted educational criteria. To date, most educational benchmarks have been developed and evaluated in English, with other languages receiving far less attention. The present study addresses this gap by introducing the first Hebrew science education benchmark, based on the national high-school matriculation exam in chemistry. We evaluated three LLMs – ChatGPT 4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro – on 120 multiple-choice questions and compared their performance to that of over 139,000 high-school students. We found that all three LLMs significantly underperformed relative to human learners. To investigate characteristics that render questions more challenging for LLMs, we conducted a regression analysis and found that visual elements and multi-step reasoning tasks negatively impacted their performance. Finally, chemistry education experts analyzed the items that were most difficult for LLMs and characterized their domain-specific failures. This study makes three contributions: (1) it extends LLM evaluation to an underrepresented linguistic context; (2) it advances the methodological landscape of LLM benchmarking by directly comparing multiple models with human students on authentic, curriculum-aligned national examinations; and (3) it provides a mixed-methods analysis of LLM performance, offering a more educationally grounded characterization of current model capabilities.
The “We Have Chemistry!” national projects competition was originally developed in 2008 as a non-formal, engaging learning environment to increase high-school students’ motivation to study chemistry. Participants submit chemistry-related projects under STEM/STEAM categories of their choice. A previous study demonstrated that participation might increase students’ motivation to learn chemistry. This retrospective study focuses on gaining insights into the benefits of this long-running educational program, elucidating possible underlying mechanisms by which participation impacts students’ motivation in relation to competition format and the development of interest. Forty-three competition ex-participants participated in this study, which followed a mixed-methods approach. Causation coding qualitative analysis of narrative data revealed three plausible motivational mechanisms related to the competition format and its components: revealing new contexts for chemistry, engaging in self-directed learning, and targeting personal interest, which were all supported by quantitative data analysis results. Our findings also suggest that for some students, the competition format might facilitate the further development of a personal interest in chemistry that they inherently possess. The combination of contextualized and personalized learning seems to give rise to non-traditional learning processes that target interest, nourish intellectual and emotional needs, and are connected to the motivation to learn chemistry. Results also highlight the teachers’ role in promoting student enrollment. The implications of this research apply to the design of other educational programs that aim to increase adolescents’ motivation to learn science: they support both the practice of combining non-formal and traditional school settings and the practice of designing learning environments that address student heterogeneity.
Behavioral science research highlights that knowledge alone does not ensure behavior change, since decision-making is influenced by cognitive, emotional, and social dimensions. To address the knowledge-behavior gap and foster knowledge-based decision-making and agency in high-school chemistry students during the COVID-19 pandemic, we developed a nano-chemistry learning unit contextualized in public health. It was designed using neuropedagogical instructional principles to support conceptual and personal meaning-making processes by considering the cognitive, affective, and social aspects of learning. After its implementation in high-school chemistry classrooms (10th-12th grades), the unit was evaluated through structured interviews conducted with six students 8–12 months after its completion. Students found the learning experience engaging, interactive, and much different from traditional lessons. They highlighted chemistry’s relevance to real-world challenges, particularly COVID-19. Retention was evident; students accurately recalled nano-chemistry concepts, visual representations, and mechanistic explanations related to mask functionality months after the intervention. Students’ perceptions shifted from viewing mask-wearing as mere compliance to recognizing its role in preventing infection. Some students proactively shared their knowledge. Findings from this evaluation suggest that neuropedagogy-based strategies that integrate active learning and contextualization might be a suitable approach to foster agency and participation. Further research should explore the role in supporting meaningful educational outcomes.
This study explored the impact of authentic out-of-school learning on students' beliefs about their science learning efficacy and career aspirations. The learning activity, designed following an authentic learning framework, was led by research scientists. We examined how students' emotions, induced in an authentic scientific activity, mediated the connection between the perceived authenticity and the self-efficacy/aspiration beliefs. Data were gathered from 177 secondary science students participating in an out-of-school activity using a scanning electron microscope (SEM). Three questionnaires were applied: (1) Perceived authenticity (Post, 7 items, Likert scale); (2) Semantic differential emotion questionnaire (SDEQ) (Pre-Post, 5 items); and (3) Beliefs questionnaire in two parts: Self-efficacy and science aspirations (pre-post, 7 + 5 items, Likert scale). The collected data were integrated into quantitative models of affect with authenticity as an independent variable, the differences in the pre-post belief structures as the dependent variable, and the emotions as mediators. Multiple regression analyses were performed to develop the models by evaluating the size and significance of the relationships between the variables. The results indicated the perceived authenticity significantly predicted both self-efficacy and the career aspiration pre-post differences. However, emotions behaved as a mediating variable only for self-efficacy growth. An additional model evaluated the connection between students' emotions learning science in school and their experience of authenticity and emotions in the out-of-school activity. The study contributes to the literature by revealing underlying affective mechanisms related to out-of-school authentic science activities and suggesting theoretical and empirical justifications.
This paper discusses the ethical considerations surrounding generative artificial intelligence (GenAI) in chemistry education, aiming to guide teachers toward responsible AI integration. GenAI, driven by advanced AI models like Large Language Models, has shown substantial potential in generating educational content. However, this technology's rapid rise has brought forth ethical concerns regarding general and educational use that require careful attention from educators. The UNESCO framework on GenAI in education provides a comprehensive guide to controversies around generative AI and ethical educational considerations, emphasizing human agency, inclusion, equity, and cultural diversity. Ethical issues include digital poverty, lack of national regulatory adaptation, use of content without consent, unexplainable models used to generate outputs, AI-generated content polluting the internet, lack of understanding of the real world, reducing diversity of opinions, and further marginalizing already marginalized voices and generating deep fakes. The paper delves into these eight controversies, presenting relevant examples from chemistry education to stress the need to evaluate AI-generated content critically. The paper emphasizes the importance of relating these considerations to chemistry teachers' content and pedagogical knowledge and argues that responsible AI usage in education must integrate these insights to prevent the propagation of biases and inaccuracies. The conclusion stresses the necessity for comprehensive teacher training to effectively and ethically employ GenAI in educational practices.
Self-regulated learning (SRL) can be defined as the ability of learners to act independently and actively manage their own learning process. This skill becomes especially important in online environments, which allow learners to decide where and how to study. Most research on SRL has focused on students; few studies have addressed teachers’ SRL as learners, and only a handful has done so in the context of online learning. A better understanding of teachers’ SRL is essential since teachers are expected to support the development of their students’ SRL abilities. This study contributes to bridging this gap by examining how online learning patterns reflect the self-regulated learning of teachers as learners in an online professional development (PD) course on nanotechnology. The study applies a mixed methods approach that combines the qualitative analysis of interviews with teacher learners and a personal summary of their learning process represented in four vignettes as well as quantitative log-file analysis to identify teachers’ learning patterns. The patterns identified are interval learning, on-track learning, skipping difficult parts, concentrated learning toward the end of the course (i.e., “bingeing”), and watching together. These patterns indirectly shed light on teachers’ SRL skills, especially their time management and task strategies, demonstrating that there is no one-size-fits-all approach to learning. The study highlights the need for a holistic approach, provides deeper insights into teachers’ learning experiences, and helps design future online PD courses.
Inclusion of a diverse group of students, both regular learners and learners with special needs in chemistry classrooms is an important goal of chemistry educators. However, alternative conceptions in chemistry among high-school students can be a barrier for completing the learning process in the classroom, especially in a heterogeneous class. This study aimed to examine differentiated instruction (DI) in a chemistry classroom. We evaluated how customized pedagogical kits (CPKs) for DI, which aim to overcome alternative conceptions found during chemistry instruction, affected students and teachers. This paper presents the findings of a mixed-method study that was conducted with 9 high-school chemistry teachers, and 551 chemistry students. We used a pre-post questionnaire to investigate the impact of CPKs on teachers' and students' self-efficacy beliefs and attitudes towards chemistry and differentiated instruction, in addition to students' achievements. The findings indicated the significantly higher averages of self-efficacy beliefs and attitudes towards DI in chemistry among teachers and high-school students, in addition to the significantly higher performance of students in chemistry tasks after implementing CPKs in classrooms. Being aware of the limitations of DI, we discussed customized pedagogical kits as a means that can support better inclusion in chemistry education.
The rise of digital technologies since the second half of the 20th century has transformed every aspect of our lives and has had an ongoing effect even on one of the most conservative fields, education, including chemistry education. During the Covid-19 pandemic, chemistry teachers around the world were forced to teach remotely. This situation provided the authors with an opportunity to investigate how chemistry teachers integrate technology into their teaching, compared with how the research literature suggests that it is done. The theoretical framework used in this explorative qualitative study involves chemistry teachers' technological, pedagogical, and content knowledge (TPACK). In particular, the study focused on different modes of technology integration (MOTIs) in chemistry teaching, which is a part of the teachers' TPACK. In the first stage, five expert chemistry teachers were interviewed so that they could share their extensive experience with technology during online chemistry teaching. Analysis of their interviews revealed that the teachers applied 7 MOTIs in their chemistry teaching. Of these MOTIs, 4 were reported in the chemistry teaching literature: (1) using digital tools for visualization, (2) using open digital databases, (3) using computational methods, and (4) using virtual laboratories and videos of chemical experiments. In addition, the interviews revealed three new MOTIs in chemistry teaching not previously reported: (5) supporting multi-level representations, (6) enabling outreach of chemistry research, and (7) presenting chemistry in everyday life phenomena. In the second research stage, we collected the perspectives of other chemistry teachers (N = 22) regarding the 7 MOTIs. This stage enabled us to validate the findings of the first stage on a wider population and provided data to rate the importance of the seven different MOTIs according to the teachers. We wish to stress that understanding the MOTIs will not only enrich teachers' theoretical knowledge base regarding integrating technology into chemistry teaching-it will also contribute to chemistry teachers' preparation and professional development programs.
Integrating generative artificial intelligence (GenAI) in pre-service teachers’ education programs offers a transformative opportunity to enhance the pedagogical development of future science educators. This conceptual paper suggests applying the GenAI tool to evaluate pedagogical content knowledge (PCK) among pre-service science teachers. By holding interactive dialogues with GenAI, pre-service teachers engage in lesson planning in a way that reveals their understanding of content, pedagogy, and PCK while facilitating the practical application of theoretical knowledge. Interpretation of these interactions provides insights into teachers-to-be knowledge and skills, enabling personalized learning experiences and targeted program adjustments. The paper underscores the need to equip pre-service teachers with the necessary competencies to utilize GenAI effectively in their future teaching practices. It contributes to the ongoing discourse on technology’s role in teacher preparation programs, highlighting the potential of addressing existing challenges in evaluating and developing teacher knowledge via GenAI. The suggested future research directions aim to further investigate the GenAI usage implications in educational contexts.
Climate change is a pressing global challenge for humanity, which should be adequately represented in the educational system. However, teachers face a significant challenge due to the vast amount of data and information about climate change available in the media. We aimed to identify aspects that affect teachers’ acceptance of technology in general and how technology may help/hinder their teaching of climate change, in particular. Thirty-five chemistry teachers and chemistry educators were exposed to a novel curriculum about climate change that was developed on a digital platform. This paper described the promoting and inhibiting factors regarding adopting technological tools to teach about electric cars within this curriculum. We applied the lenses of the technology acceptance model (TAM) framework to analyze teachers’ responses. Most of the hindering factors concerned the general disadvantages of integrating technology into teaching (e.g., technical malfunctions); therefore, these aspects should be primary addressed to encourage adopting and applying educational technology. However, factors that are specific to teaching climate change in relation to TAM emerged as well. These factors included the critical consumption of digital data, the need to constantly change one’s teaching practices based on the changing data, as well as the social impact of such a tool on the students’ environment. We wish to stress that the TAM can be applied as a framework to identify teachers’ filters and amplifiers that might promote or inhibit transforming theoretical knowledge into practice.
Successful integration of digital technologies in the education of young children still needs to be solved. Despite a growing body of research focusing on learning through digital technologies in childhood, there are areas of knowledge where the impact of digital technologies has yet to be explored. A prominent example is nanoscience and nanotechnology (NST), a new interdisciplinary field that promises to solve long-standing global challenges. Considering that NST concerns elements that cannot be observed with the naked eye, their understanding by young children requires appropriate teaching methods. These distinctive aspects of NST align well with the capabilities of smart mobile devices, the critical feature of which is their ability to display interactive simulations and playful visualizations. This study investigates and compares the effect of using tablets and alternative experiential teaching on developing the ability to understand nanoscale elements. To implement the research, we conducted a week-long intervention, including experimental and control groups. Children in the experimental group participated in a nanoteaching session during the school curriculum, using educational software on tablets. The children in the control group participated in a precisely similar instruction but without using technology. To assess the children’s performance, the Nanoscale Elementary Knowledge Comprehension Test (TENANO) created for the needs of this study was used. The sample consisted of 101 s-grade primary school children in Greece. The results showed that teaching with tablets compared to alternative experiential teaching contributed significantly to developing young children’s nanoliteracy level. Moreover, gender and non-verbal cognitive ability did not seem to differentiate the development of children’s ability to understand nanoscale entities.
This study explored the impact of authentic out-of-school learning on students' beliefs about their science learning efficacy and career aspirations. The learning activity, designed following an authentic learning framework, was led by research scientists. We examined how students' emotions, induced in an authentic scientific activity, mediated the connection between the perceived authenticity and the self-efficacy/aspiration beliefs. Data were gathered from 177 secondary science students participating in an out-of-school activity using a scanning electron microscope (SEM). Three questionnaires were applied: (1) Perceived authenticity (Post, 7 items, Likert scale); (2) Semantic differential emotion questionnaire (SDEQ) (Pre-Post, 5 items); and (3) Beliefs questionnaire in two parts: Self-efficacy and science aspirations (pre-post, 7 + 5 items, Likert scale). The collected data were integrated into quantitative models of affect with authenticity as an independent variable, the differences in the pre-post belief structures as the dependent variable, and the emotions as mediators. Multiple regression analyses were performed to develop the models by evaluating the size and significance of the relationships between the variables. The results indicated the perceived authenticity significantly predicted both self-efficacy and the career aspiration pre-post differences. However, emotions behaved as a mediating variable only for self-efficacy growth. An additional model evaluated the connection between students' emotions learning science in school and their experience of authenticity and emotions in the out-of-school activity. The study contributes to the literature by revealing underlying affective mechanisms related to out-of-school authentic science activities and suggesting theoretical and empirical justifications.
Artificial intelligence (AI) has made remarkable strides in recent years, finding applications in various fields, including chemistry research and industry. Its integration into chemistry education has gained attention more recently, particularly with the advent of generative AI (GAI) tools. However, there is a need to understand how teachers’ knowledge can impact their ability to integrate these tools into their practice. This position paper emphasizes two central points. First, teachers technological pedagogical content knowledge (TPACK) is essential for more accurate and responsible use of GAI. Second, prompt engineering—the practice of delivering instructions to GAI tools—requires knowledge that falls partially under the technological dimension of TPACK but also includes AI-related competencies that do not fit into any aspect of the framework, for example, the awareness of GAI-related issues such as bias, discrimination, and hallucinations. These points are demonstrated using ChatGPT on three examples drawn from chemistry education. This position paper extends the discussion about the types of knowledge teachers need to apply GAI effectively, highlights the need to further develop theoretical frameworks for teachers’ knowledge in the age of GAI, and, to address that, suggests ways to extend existing frameworks such as TPACK with AI-related dimensions.