We use a validated conceptual survey instrument focusing on thermodynamic processes and the first and second laws of thermodynamics as covered in introductory physics to investigate the context dependence of introductory and advanced student responses to introductory thermodynamics problems after instruction. The survey has conceptual problems that incorporate many contexts with the same underlying principles and concepts involving internal energy, work, heat transfer, and entropy. Here we focus exclusively on entropy.  This study used data from over 1000 college students in introductory-level algebra- and calculus-based physics courses as well as upper-level thermodynamics courses. In addition to prior research, think-aloud interviews with a small subset of students in which they were asked to answer the survey problems while thinking-aloud was useful for understanding the context dependence of student responses in some situations. Here we present analysis of data in multiple contexts reflecting students' ideas about the change in entropy of a gas in spontaneous/irreversible processes and in cyclic processes. We find that a persistent belief in the constancy of entropy even for spontaneous/irreversible processes is a common difficulty among introductory students across problems with different contexts, while upper-level students had great difficulty across contexts in which identifying entropy as a state variable is important.

To increase the likelihood of developing a fair, reliable, and valid standardized research-based conceptual learning assessment instrument, researchers must use a truly representative sample of the types of individuals for whom the instrument is designed to evaluate. Researchers developing the Fluids Conceptual Evaluation (FCE) decided to make every attempt to establish a truly representative sample population of post-secondary IPLS students for every stage in development. This involved the collection of data from a variety of different types of institutions and students from around the country. Taking over three years to obtain, the researchers faced many challenges along the way.  This poster presents an overview of these challenges and how the researchers collected the data for this development process as well as suggestions for future conceptual learning assessment developers. Supported by NSF Award #, 2021273.

This poster will present results of a cluster randomized control trial in an introductory electricity and magnetism course at Harvard University. Treatment group students completed self and peer grading of their weekly physics problem sets over the course of four weeks, while those in the control group did not complete any self or peer grading. Use of physics related metacognitive problem solving skills were assessed before and after the treatment using the Physics Metacognitive Inventory. The inventory is multidimensional and assesses six facets of metacognition in physics. Statistical modeling methods combining item response theory and regression analysis allowed for the investigation of treatment effects on each facet of metacognition measured by the inventory. Our findings show that students exposed to the self and peer grading treatment reported significant increases in their use of free body diagrams and their understanding of the utility of free body diagrams during physics problem solving. 

An increasing number of physics faculty members strive to improve their undergraduate and graduate programs to offer high-quality instruction and better serve diverse student populations. However, implementing evidence-based instruction and improving inclusivity are challenging and require systemic organizational change. APS and other disciplinary organizations developed resources to help physics programs increase their capacity to pursue organizational, systemic change. In this study, we design a survey instrument to measure physics departments’ culture around their approach to systemic change. We created and led a series of collaborative design sessions with multiple key stakeholders to inform the survey design and tailor the instrument to the emerging needs of the physics community. In this poster, we will describe the novel approach to the survey design process and outcomes from the co-design sessions, share the survey constructs that are designed to assess systemic change, and discuss how evidence gathered in this project will feed back to APS change initiatives and committees to inform holistic programmatic changes in physics.

The development of the Fluids Conceptual Evaluation (FCE) involves a collaborative effort among multiple institutions to create a reliable, valid, and fair two-tier multiple-choice instrument covering both fluids statics and dynamics featuring multiple representations. We generated 66 Tier 1 conceptual items and gave subsets of these questions to students in think-aloud interviews. We are studying the reasons behind students’ responses to generate Tier 2 responses for field testing.  This poster presents an overview of the FCE and its various items along with our plans for future field testing with diverse student groups. If you are interested in participating in the FCE field test, please reach out to the project lead, Dawn Meredith, at dawn.merdith@unh.edu.

This work is part of a multi-institution collaboration developing the Fluids Conceptual Evaluation (FCE), a fair, valid, and reliable research-based conceptual fluids assessment for Introductory Physics for Life Science (IPLS) courses.  The FCE will utilize two-tier multiple-choice items covering both fluid statics and dynamics.  For each item, a Tier-1 question will ask for a factual answer, while the following Tier-2 question will ask the student to choose an explanation closest to their own reasoning.  In 2021-2023 our group conducted 73 interviews at 10 diverse institutions, asking the students to answer a subset of our Tier-1 questions and to explain their reasoning.  When coding those interviews, we identified student explanations that could serve as the basis for options in Tier-2 questions.  This poster will describe the process of creating our Tier-2 questions and present several examples.  IPLS instructors interested in serving as a field test site for the assessment should contact Project Lead Dawn Meredith.

At Western Washington University, research is underway to investigate student understanding of the terminal speed behavior of falling objects. We have developed a two-part written task involving three spheres, of varying size and mass, each falling at their own terminal speed. In part 1, students rank the spheres according to the drag force exerted on them. In part 2, students rank the spheres according to their terminal speeds. We’ve identified three core difficulties students face when analyzing terminal speed behavior: (1) low salience of the Fnet = 0 condition; (2) high salience of cross-sectional area as a determining factor; (3) difficulty analyzing cases where more than one variable is manipulated. In this poster presentation, we will discuss these core difficulties and others that arise in analyzing terminal speed behavior.

In physics education, a number of studies have developed assessments of teachers’ KSU of physics concepts with existing concept inventories, in which teachers were asked to predict the popular incorrect answers from students. The results provide useful but indirect information to make inference on teachers’ knowledge of the misconceptions that students may be using in answering the questions. A new instrument is developed using a three-tier item design. The items were adapted from the Force Concept Inventory. Each item was designed in three tiers, with tier-1 asking for teachers’ own answers to the question to test their content knowledge, tier-2 asking for teachers’ predictions of popular students’ answers, and tier-3 asking for teachers’ explanations of students’ incorrect answers in an open-ended form. The three-tier design captures teachers’ content knowledge, predictions and explanations in a single item to allow explicit measures of teachers’ own content knowledge and their KSU of students’ misconceptions. The instrument was validated with preservice physics teachers. The assessment results also suggest that the preservice teachers’ KSU on force motion was only moderately developed, and their content knowledge was uncorrelated with their KSU. In addition, a four-level progression scale of KSU was also developed, which categorized the preservice teachers into five proficiency groups. 

The galvanic cell is a typical interdisciplinary topic, which is primarily taught in chemistry but its underpinning is in physics. Research on student learning in galvanic cell has revealed a large number of misconceptions that are difficult to change through traditional instruction in Chemistry. A source of the learning difficulties is that most students lack an explanatory framework to integrate the many complex phenomena and processes into a coherent knowledge system. As a result, students often rely on memorization of terms, laws, and equations in solving problems but without meaningful understanding. To address this issue, this study aims to help students understand the mechanisms in physics that can help explain the chemistry concepts and promote knowledge integration. Specifically, a conceptual framework model about galvanic cell is developed based on the central idea from physics, which is used as the core concept for mechanistic explanations of related chemistry concept. Guided by the conceptual framework, an instrument is developed to assess the knowledge integration in students’ learning of galvanic cell. The results show that the conceptual framework model can effectively represent the knowledge structures of students at different levels of knowledge integration. Implications on developing effective instruction for promoting knowledge integration will also be discussed.

Self-efficacy, crucial for academic achievement in STEM, is typically assessed using pre-post measurements from surveys. However, developing a way to assess in-the-moment impacts to students’ self-efficacy would create many opportunities for further study and intervention. With this goal, we devised a mixed-methods approach combining the Experience Sampling Method (ESM) with individualized daily journal prompts. While rich, this design poses validation challenges. To address these challenges, we analyzed three ESM survey questions, indicative of task-level self-efficacy, for skewness. Deviations from normal distributions suggest  that traditional validation techniques may not apply. We  further examined  the Pearson’s coefficient of skewness for each item and participant that does not assume  the normal distribution of data. We would expect that if all ESM survey items are aligned with self-efficacy then the skewness for each individual should be consistent across those items. In this poster, we will present the shifts in skewness between items and discuss the data from validation interviews that expand on our quantitative findings. 

Flexible assessments are distinct from standard research-based assessments like the Force Concept Inventory in that the items and the item ordering may differ between administrations. Our group has been developing a flexible assessment for upper-level undergraduate quantum mechanics called the Quantum Physics Assessment (QuPA) to address the lack of consensus on what can/should be taught in an undergraduate quantum course. This assessment will exist on a web platform where instructors can select topics they want to assess and other testing parameters. In Fall 2023, we conducted 34 interviews with faculty from institutions across the U.S. asking them about their experience with assessments, what they would improve, and what feedback would be useful. This paper will present the theory and motivation for flexible assessment of quantum mechanics proficiency, discuss our interview protocol, and summarize the general views of the instructors.