This article explores the use of adaptive learning platform (ALP) data to conduct early identification and provide support to students who have a low-performance outcome (C or lower) in a numerical methods engineering course. The data from assigned ALP lessons for two semesters was used to create decision-tree models to identify students who would benefit from advising and tutoring support. In the following two semesters, low-performing students were identified early in the semester and provided with support, and their performance was compared to their peers. The best-performing prediction model achieved an accuracy of 85% in predicting low-performing students in the third week of the course. The support included weekly one-on-one tutoring and advising sessions. Although only 23% of the identified students accepted support, they scored one-third a letter grade better than those who did not. Additionally, students who received support were invited to participate in a focus group at the end of the semester. Positive outcomes reported included improved understanding of course material, higher academic performance, advice on learning strategies, and guidance on non-course-related topics like internships and employment. Although most students valued receiving personalized invitations, a few felt singled out as low-performing. Students acknowledged the significance of individualized support, gave advice on how to word the invitation emails, and made helpful suggestions for improving help sessions, particularly in terms of personalization and recognizing their heavy academic workload.
In 2018, the Department of Bioengineering and the School of Nursing at University of Pittsburgh implemented an interdisciplinary partnership that integrated senior nursing students into the bioengineering capstone Senior Design course as part of a National Institutes of Health education grant. This two-semester course requires senior Bioengineering students to synthesize and extend principles from prior coursework toward the design a medical product meeting an unmet clinical need. Senior Design teams interact with clinicians, patients, and caregivers as part of the overall design process to understand the unique challenges of medical product design, including the requirements for regulatory approval. The teams develop iterative designs, fabricate prototypes, and perform both verification and validation testing to evaluate whether product performance criteria are met. Integrating nursing and bioengineering students was anticipated to provide opportunities for interprofessional learning, earlier and more frequent clinical input to the design process, and exposure to a spectrum of unmet clinical needs. Conversely, nursing students were anticipated to gain an understanding of the medical product design process, including regulatory requirements, to potentially empower future innovativeness. The impact of this interdisciplinary partnership on the anticipated outcomes was assessed over a five-year timeframe using research surveys and student interviews. The design self-efficacy survey was administered in a pre-post manner to assess changes in bioengineering and nursing students’ confidence, motivation, success expectancy, and apprehension for performing design activity. Students’ interprofessional collaborative development was also measured in a retrospective pre-post manner using the interprofessional collaborative competency attainment survey. Finally, a spectrum of student interviews was conducted to obtain perspectives about the interdisciplinary partnership. The data were analyzed using statistical and qualitative data methods. The results were overwhelmingly positive for the partnership. The results make a strong case for such partnerships and suggest benefits for both student groups, including significant effects for design confidence and a multitude of collaborative competencies. For bioengineering students, the nursing students’ clinical knowledge, perspectives, suggestions related to unmet clinical needs, and feedback were mentioned by 84
The metacognitive strategies of planning, monitoring, and evaluating can be promoted through systematic reflection to drive self-directed, lifelong learning. This article reports on a three-year study on systematic written reflection within an undergraduate Fluid Mechanics course to promote planning, monitoring, and evaluation. Students were prompted weekly to reflect on their in-class problem-solving, classroom and exam preparation, performance, behaviors, and learning in a flipped classroom at a large southeastern U.S. university. In addition, they received intentional instruction on how to plan, monitor, and evaluate their problem-solving during class. To enable a comparative assessment, a flipped classroom without these interventions was also implemented as a non-experimental cohort. The cohorts were compared using a final exam, concept inventory, and the Metacognitive Activities Inventory (MCAI). The MCAI indicated a significantly higher positive change (pre- to post-course) in self-regulatory behavior for the experimental cohort ( p = 0.037). The weekly reflections were studied using an inductive content analysis to assess students’ self-regulatory behaviors. They were also used to investigate statistical associations between reflection content and course outcomes. This revealed that academic self-discipline via planning, monitoring one's work, or being careful and diligent may be as aligned with course performance in STEM as is practice with the problem-solving itself. The effects for the final exam in the experimental cohort were positive overall as well as statistically or practically significant for various demographic strata. These results provided evidence for the potential enhancement of course performance with metacognition support. A positive shift in students’ perspectives regarding the value of the reflection questions was observed throughout the study. Therefore, as an implementation guide for other educators, the reflection questions and any changes made in posing them to students are discussed chronologically. Overall, the study points to the desirability of providing metacognition support in a STEM course.
Multiple-chance testing was used to conduct standards -based testing in a blended-format numerical methods course for engineering undergraduates. The process involved giving multiple chances on tests and post-class learning management system quizzes. The effectiveness of standards -based testing was evaluated through various forms of assessment, including an analysis of cognitive and affective outcomes, and compared to a blended classroom that did not use standards-based testing. Based on a two-part final exam, a concept inventory, final course grades, a classroom environment inventory, and focus groups, the results showed that standards -based testing had overall positive effects. Standards-based testing was associated with a more significant percentage of students (15% vs. 3%) earning a high final exam score, a higher proportion of A grades (36% vs. 27%), and a better classroom environment on dimensions of involvement, cohesiveness, and satisfaction. Focus group discussions revealed that students appreciated the benefits of enhanced learning, second chances, and reduced stress with standards-based testing. The study also included an analysis of the impact of standards-based testing on underrepresented minorities, Pell Grant recipients (low socioeconomic groups), and low-GPA students, as well as an examination of test-retaking behaviors. The methodology and comprehensive results of the study are presented in this paper.
During two semesters, a numerical methods course for mechanical engineering students at a large US southeastern university used discussion board questions to promote reflection and metacognition. The course covered eight chapters, each with a related discussion question. The students could choose to answer these questions and receive 2% extra credit for the course. This was intended to help the students who missed some of the 30 online homework assignments that comprised 15% of the final course grade. The questions were also meant to encourage the students to think deeply and creatively. The students could see other students’ responses after they posted their own. The questions ranged from making a meme, writing a nursery rhyme, and explaining a complex or easy concept. Only 64% of the total possible responses were submitted by students, and there was a small-to-medium practical but no statistical significance between the levels of participation among the high- or low-performing students. The submissions were analyzed and determined to be at the low level of Bloom’s taxonomy. They identified complex topics to inform future instruction.
In this study, flipped instruction in an undergraduate engineering course in the 'COVID' online, remote environment was conducted and compared to onsite flipped instruction (i.e. pre-COVID) to explore potential changes in student perceptions. Student perceptions were gathered via survey instruments and investigated further through instructor interviews. This analysis was done at three universities and made possible by extensive research with the flipped classroom at these three schools as part of a previous NSF-funded study between 2014 and 2016. Results gathered in the online remote setting suggest positive changes in student perceptions of flipped instruction compared to the onsite environment, including the decreased perception of the 'load' imposed by the flipped classroom and the 'effort" required. Some desirable outcomes remained unchanged in the remote setting. The recent and emerging literature has suggested the remote, online environment dictated by the pandemic may be beneficial for flipped teaching and learning. These and other findings from conducting flipped classrooms at three engineering schools in the online environment are presented, including perceptions of the classroom environment (via the College and University Environment Inventory), benefits and drawbacks identified, student motivation levels, and perceived learning.
Since the 2014 high-profile meta-analysis of undergraduate STEM courses, active learning has become a standard in higher education pedagogy. One way to provide active learning is through the flipped classroom. However, finding suitable pre-class learning activities to improve student preparation and the subsequent classroom environment, including student engagement, can present a challenge in the flipped modality. To address this challenge, adaptive learning lessons were developed for pre-class learning for a course in Numerical Methods. The lessons would then be used as part of a study to determine their cognitive and affective impacts. Before the study could be started, it involved constructing well-thought-out adaptive lessons. This paper discusses developing, refining, and revising the adaptive learning platform (ALP) lessons for pre-class learning in a Numerical Methods flipped course. In a prior pilot study at a large public southeastern university, the first author had developed ALP lessons for the pre-class learning for four (Nonlinear Equations, Matrix Algebra, Regression, Integration) of the eight topics covered in a Numerical Methods course. In the current follow-on study, the first author and two other instructors who teach Numerical Methods, one from a large southwestern urban university and another from an HBCU, collaborated on developing the adaptive lessons for the whole course. The work began in Fall 2020 by enumerating the various chapters and breaking each one into individual lessons. Each lesson would include five sections (introduction, learning objectives, video lectures, textbook content, assessment). The three instructors met semi-monthly to discuss the content that would form each lesson. The main discussion of the meetings centered on what a student would be expected to learn before coming to class, choosing appropriate content, agreeing on prerequisites, and choosing and making new assessment questions. Lessons were then created by the first author and his student team using a commercially available platform called RealizeIT. The content was tested by learning assistants and instructors. It is important to note that significant, if not all, parts of the content, such as videos and textbook material, were available through previously done work. The new adaptive lessons and the revised existing ones were completed in December 2020. The adaptive lessons were tested for implementation in Spring 2021 at the first author's university and made 15% of the students' grade calculation. Questions asked by students during office hours, on the LMS discussion board, and via emails while doing the lessons were used to update content, clarify questions, and revise hints offered by the platform. For example, all videos in the ALP lessons were updated to HD quality based on student feedback. In addition, comments from the end-of-semester surveys conducted by an independent assessment analyst were collated to revise the adaptive lessons further. Examples include changing the textbook content format from an embedded PDF file to HTML to improve quality and meet web accessibility standards. The paper walks the reader through the content of a typical lesson. It also shows the type of data collected by the adaptive learning platform via three examples of student interactions with a single lesson.
Adaptive learning platforms are increasingly being used as part of varying instructional modalities. Particularly relevant to this paper, adaptive learning is a critical component of personalized, preclass learning in a flipped classroom. Previously inaccessible, data generated by adaptive learning platforms regarding student engagement with the course content provides an invaluable opportunity to gain a deeper understanding of the learning process and improve upon it. We aim to investigate the relationships between adaptive learning platform interactions and overall student success in the course and identify the variables most influential to student success. We present a comprehensive analysis of our adaptive learning platform data collected in a Numerical Methods course, including aggregate statistics, frequency analysis, and Principal Component Analysis, to determine which variables exhibited the most variability and, therefore, the most information in the data. Subsequently, we used the Partitioning Around Medoids clustering approach to investigate naturally occurring clusters of students and how these clusters relate to overall performance in the course. Our results show that overall performance in the course, as measured by the final course grade, is strongly associated with (1) the behavioral interactions of students with the adaptive platform and (2) their performance on the adaptive learning assessments. We also found distinct student clusters (as defined by success in the course) that exhibited distinctly different behaviors. These findings provide qualitative and quantitative information to identify students needing support and to craft an evidence-based support strategy for these students.
Leading scholars have indicated a lack of knowledge on how to propagate and sustain evidence-based instructional practices, such as active learning. However, they have identified social interactions as key for dissemination. Interestingly, the instructional coaching literature has drawn a direct connection between propagation of research-based practices and effective coaching of teachers. The authors have worked to propagate active learning and educational scholarship among instructors in their school of engineering. Their support model for this was informed by the change framework of Henderson and colleagues and consists of learning-community events, instructional coaching, classroom observation, student feedback, and instructor follow-up. Interestingly, the social focus of their model, including one-on-one coaching, was identified as a strength by the participating instructors. Preliminary results from this support program have been promising with respect to instructor participation, propagation of active learning and educational scholarship, and valuation by instructors. In this article however, the authors make a new argument for the infusion of athletic coaching to their support model for potential transformative outcomes. Despite the shared mission of athletic coaches and academic instructors to educate young adults, there is often little-to-no collaboration between them. However, given the origin of coaching in athletics, shouldn't instructional coaching be looking to athletic coaching for transformative insights and support? Along with making an argument for this unique paradigm, the authors suggest an exploratory case study approach for assessing the impact of athletic coaching within an engineering instructional support and propagation program. Our ultimate objective is to inspire and support other educators in adopting this potentially transformative model.
When COVID-19 struck, engineering schools responded to unique issues, including interrupted capstone projects, cooperative education, and study abroad. Students became a focus. This led to a funded study to investigate the pandemic's impact on engineering students' academic motivation, educational valuation, learning, and perceived stress, which were connected through a conceptual model. Approximately seven months after the onset, a large sample of undergraduate engineering students at a public U.S. university (n = 1,140) responded to a survey (41.6% response), followed by focus groups. Jones' MUSIC Model of Academic Motivation and the Perceived Stress Scale (PSS-10) were key components of the conceptual model. Seventy-eight percent (78%) said their motivation was less versus before remote instruction. Two dimensions of the MUSIC Model were only at the middle point of the measurement scale - interest and empowerment. Students scored higher on the PSS-10 (M = 22.2) seven months into the pandemic compared to other groups beforehand. Medium negative correlations were found between the MUSIC dimensions and the PSS-10 score, suggesting decreased motivation accompanied by increased stress. Remote coursework was the most-frequent de -motivator, and the valued college experience cited most was Campus-based instruction. The most-frequent stressor was Academic. In all focus groups, low or decreased motivation was mentioned. This research informs Higher Education about undergraduates' motivation and stress, in particular during COVID-19 and contributes to use of the MUSIC Model and PSS-10 with engineering students. Awareness of motivation and stress experienced during COVID is crucial for responding to future crises.
When students repeatedly reflect, it can enhance their metacognitive abilities, including self-regulatory skills of planning, monitoring, and evaluating. In a fluid mechanics course for undergraduates at a large southeastern U.S. university, in-class problem solving in a flipped classroom was coupled with intentional metacognitive skills instruction and repeated reflection to enhance metacognition. The weekly reflective responses were coded by two analysts to identify the recurring themes and uncover evidence of the development and/or reinforcement of self-regulating behaviors for academic management. To enable a comparison, a flipped classroom without the metacognitive instruction and repeated reflection was also implemented (i.e., non-intervention group). The two cohorts completed identical final exams. Based on our preliminary analysis with year one data, a statistically and practically-significant difference between the two cohorts was found with the free-response scores on the final exam in favor of the intervention cohort that had received the metacognitive support (p < 0.0005; Cohen's d = 0.72). Also, the Metacognitive Activities Inventory (MCAI) indicated a significantly-higher positive change in self-regulatory behavior for the intervention cohort (p = 0.001; d = 0.50). Focus groups were conducted to gather students' perspectives on the reflective activity, with differences found by demographic group. In addition, a significantly higher proportion of females (versus males) viewed the reflections in a positive manner (p = 0.05). Significant associations between themes in the weekly reflections and direct knowledge measures were also uncovered. This included a positive relationship between academic self-management (i.e., diligence and carefulness) and exam performance. Overall, our preliminary results point to a desirable impact of metacognitive instruction and repeated reflection on knowledge outcomes, metacognitive skills, and self-regulatory behaviors.
Contribution: Reflection is a critical skill because it involves thinking about and reviewing one’s work. It contributes to self-regulatory, metacognitive behaviors when practiced regularly. In this research, a unique approach to promote reflection via SPICE simulation after quizzes in a microelectronics course was studied. Background: This study aligns with recent calls from the engineering education community for increased use of systematic reflection. Intended Outcomes: Anticipating the lack of experience with reflection, the authors intended for students to gain experience and realize its benefits. Application Design: Repeated reflection occurred with the review of weekly quiz or exam results after submission via SPICE and question prompts. Reflections were analyzed by the authors for depth. Pre and postsurveys assessed student perspectives on reflection. Findings: When reflective depth levels after an initial quiz were associated with subsequent exam results, students who achieved significantly higher results unexpectedly reflected at the lowest level. This suggested the prompts were not designed to challenge all students to reflect. After a subsequent quiz, an overall lower average depth level and an increased number of reflections at the lowest depth level suggested potential fatigue with the reflective prompts. The presurvey had illustrated the lack of experience with reflection and hence, an opportunity. The postsurvey demonstrated highly positive views on reflection, including the development of metacognitive skills. Although the question prompts were likely not designed for optimal reflection, repeated reflection nonetheless made a positive impression on students in need of such experience. Applying these lessons learned may therefore lead to even greater outcomes with reflection.
Evidence-based testing strategies in the form of multiple cumulative midterm tests preceded by practice tests were recently introduced to a numerical methods course for engineers after the course had been taught for many years in a blended fashion. The instructor introduced these practices in fall 2019, thereby creating his so-called modified blended approach, with the objective of enhancing direct and affective assessment results in his blended classroom implementation. A statistical comparison of results from this modified approach with results from a prior semester of blended instruction was made using final exam and concept inventory scores as well as classroom environment scores based on the CUCEI. This comparison was made for students collectively and for several demographic segments of interest. Based on triangulated results from direct assessments of conceptual understanding and Bloom's taxonomy (lower levels), the modified blended approach with the testing strategies may be the preferred method for this blended classroom for students collectively as well as potentially for Pell grant recipients as a group. The classroom environment and direct assessment results from the higher levels of Bloom's taxonomy did not suggest a preferred instructional method. Support for blended instruction and practice and cumulative testing from the literature is also presented.
Flipped instruction in an undergraduate numerical methods course in the online, remote environment during the COVID-19 pandemic was conducted with and without the use of adaptive-learning lessons for pre-class preparation. This comparison was made to explore potential differences with and without adaptive software relative to exam and concept inventory performance and student perceptions of the classroom environment, learning and motivation, and benefits and drawbacks. Student perceptions were gathered via the College and University Classroom Environment Inventory (CUCEI) and a survey designed to capture feedback specific to flipped instruction. The analysis was made possible by a current NSF grant to study adaptive learning in the flipped classroom at three universities and extensive prior research with the flipped classroom and adaptive learning by the authors. Results gathered in the online flipped classroom with adaptive learning suggested positive changes in the following: classroom environmental perceptions, preference for flipped instruction, perceived responsibility imposed, motivation for independent learning, and perceived learning. Furthermore, based on an open-ended question, there was a significant decrease in the proportion of students who experienced load, burden, or stressors in the online flipped classroom when adaptive learning was available versus not. Multiple-choice exam and concept-inventory results were slightly higher with adaptive lessons (although not significantly so), with the most promising results occurring for Pell grant recipients. The emerging medical education literature has suggested that adaptive learning and flipped instruction will be key to post-pandemic education. The present article begins advocacy for adaptive learning with flipped instruction in engineering education.
Recruitment, retention, and preparation of students in science, technology, engineering, and math (STEM) fields are critical to meeting the global challenges of the 21st century. Gamification-game playing concepts used in another activity-has been postulated as an active teaching strategy that engages students, encourages a sense of community, and develops students' professional skills, which are all core components to recruitment, retention, and preparation. To explore the impact of game design and play on students' higher levels of perceived cognitive/learning engineering-related and course-specific concepts, higher levels of student engagement, a sense of community, professional skills, retention, and easy transferability for university-level educators, board game design and play projects were implemented in three civil, environmental, and construction engineering courses at the University of Pittsburgh, Arizona State University, and Clemson University in 2016. Findings obtained from analyzing students' responses to an open-ended survey and focus group questions indicated that game design and play were effective in learning technical course content, including content related to sustainability and construction terminology, but these results were related to lower levels of Bloom's taxonomy. Both the classroom observations and the focus group/interviews revealed that game design and game play enhanced engagement. The Likert survey results did not show a statistically significant difference in the sense of community but did show a statistically significant positive difference in the professional skill of writing. Lastly, neither the Likert survey results nor the focus group/interview results showed that game design and play impacted retention. Given these results, the findings of this study are cautiously optimistic at best for the applications of game design and play in engineering curricula as a method for recruiting, retaining, and preparing students. Recommendations for improving game design studies as well as the implementation of game design projects in engineering courses are provided. (C) 2021 American Society of Civil Engineers.
In this Innovative Practice Full Paper we present an approach where we coupled several proven pedagogical practices to enhance student engagement during the time of remote/hybrid instruction due to the COVID-19 pandemic. One of these approaches was team-based learning throughout the entire semester which aided in student motivation. A second practice implemented was game-based learning to drive student engagement and excitement. This game-based learning approach used a semester-long scoring system which allowed students to compete for bonus points both on an individual and team basis. This enabled students to practice their teaming skills. Lastly, there was a major focus on diversity & inclusion in addition to teamwork in the course. Students were arranged into teams in an optimized manner by the CATME software. The optimization constraints were chosen using best practices for diversity in race & ethnicity, gender, skill levels, and leadership philosophy, while also considering students with similar schedules for availability purposes. The course also contained instructional modules on effective teamwork as well as contributions in the field of electrical engineering by underrepresented minorities. This paper details the innovative coupling of these practices and how they fit into the course's overall plan. Classroom activity and student perceptions associated with these practices were assessed via structured classroom observation using the COPUS protocol and collection of survey/focus group data, respectively. Assessment results are discussed, along with challenges encountered in this electromagnetics course in the hybrid/remote learning environment.
Through reflection, learning experiences that students most-frequently valued during open-ended, scaffolded problem solving in a bioengineering two-course sequence were identified in this study. Reflective knowledge of this type can inform instructors in orchestrating experiences to scaffold problem solving of this kind and were useful in directly demonstrating students’ development in problem solving. Reflection is necessary for learning and is an important aspect of scaffolding. The literature has made recent calls for the additional use and study of reflection and scaffolding within STEM. This paper is intended to respond to these calls by focusing on students’ reflections about the experiences they most-frequently valued during scaffolded work. Students reflected at multiple points about their scaffolded, problem-solving experiences. These reflections were systematically content-analyzed for the experiences valued by students for their learning and development. These valued experiences can be the focus of instructional and scaffolding efforts for open-ended problem-solving in similar courses.
This research describes a pilot program for propagating active learning within engineering education starting with a group of nine interested instructors from two departments. The first and second authors served as the discipline-based coaches for these instructors, and the propagation program involved community discussions, one-on-one coaching, classroom observation, assessment of student perspectives, and feedback to and follow-up with the instructors. This approach aligned with the professional development and coaching literature as well as emergent change strategies identified by Henderson and colleagues. This work is important because STEM education has not generally taken a research-based approach to dissemination of pedagogical innovations, and research on sustained change is only in its early stages. Using a case study approach involving instructor interviews, documentary data (i.e., discussion notes), and classroom observation, the program was assessed based upon instructor participation and accomplishments (including scholarship of teaching and learning activities), plans for continued active-learning use, and valuation. Of the nine initial instructors, seven participated in the one-year program until the end, including three who also engaged in scholarship of teaching and learning. All seven used active learning, as confirmed by observation or interview. Based on their interviews, instructors identified the program's "people" focus, in particular one-on-one coaching and community discussions, as strengths of the program, as supported by the coaching literature. A finding of this research is that benefits were achieved despite non-ideal levels of instructor participation in all program aspects. The goal is to share an implementation and assessment approach with other educators considering relationship-driven, emergent strategies for adoption or expansion of active learning.
Several theories, including Experiential Learning Theory, describe the importance of reflection for learning, and a host of articles have called for additional research on reflection in engineering education. Ambrose has called for engineering curricula with “opportunities for reflection to connect thinking and doing,” since students learn by doing but only when they reflect on the doing too. Regular reflection plays a critical role in the construction of metacognitive knowledge and self-regulatory skills, which includes monitoring and evaluating one’s own learning, knowledge, and skills. Unfortunately, the development of metacognitive skills is often not formally included within curriculums. However, simple in-class active learning exercises, such as think-pair-share or minute papers, as well as post-exam analysis by students, can promote reflection and metacognition. In a microelectronics course, we recently incorporated post-exam reflective exercises using SPICE simulation tools to guide students’ reflections on errors made and strategies to improve future performance. The instructor was inspired to use this approach after learning of its use in an introductory circuits course. In the circuits course, the instructors had used a reflective approach known as Exam Analysis and Reflection (EAR), which had previously been developed for mechanical engineering courses. In the microelectronics course, we preliminarily incorporated reflective exercises after two exams and applied the EAR with the second. After the first exam, students used the simulator to correct any errors, which introduced them to using simulation for reflection. With the second exam, which was a small quiz, a similar procedure was followed, in which students used the simulator to reconstruct the amplifier circuit on the exam. Thus, students used the simulator to “re-do” the quiz to determine the simulated values, with the goal of having students recognize and question any differences, which could have resulted from calculation errors or natural differences between simulation and hand calculations, among other reasons. Students were then asked to reflect on the following questions from the EAR approach: “How is my exam result different from the simulated result?“, “What went wrong with my solution?”, and “How can I use this to improve my performance in the future?” To assess the impact of using simulation to reflect on their exams, we interviewed students as well as directly assessed their performance. Students were given a final exam problem that was very similar to the quiz problem where they applied the EAR approach. We compared the results from this final exam problem to those from the prior year (without reflection), in which the final exam was the same. We also determined the correlation between the quality of students’ reflections and their performance on this final exam problem. We assessed the quality and depth of the reflections using a four-category rubric from the published literature. The preliminary results have been promising, showing evidence of students’ appreciation of the reflective approach in their interviews and depth in their EAR responses. The interview data also highlighted lessons on improving our initial implementation of simulation for this type of reflection and comparison.
Bryan A. Norman合作论文数Department of Industrial Engineering;University of Pittsburgh2