This post is part of our [work in] Progress series, an effort to share our preliminary findings on the impact of artificial intelligence in higher education to help the field move at the pace of technology.
Innovating Formative Assessment with AI
In the field of education, innovation often brings a mix of excitement and skepticism. Just as online education and new technologies were met with questions in the past, today’s educators are navigating the possibilities and concerns surrounding generative AI. At the intersection of innovation and efficacy, custom GPTs present an opportunity to revolutionize formative assessment, particularly through the simulation of authentic role-play scenarios.
Custom GPTs can simulate a wide range of realistic scenarios, from classroom and healthcare situations to business environments, exposing preservice teachers, healthcare professionals, and business students to diverse challenges they may encounter in their careers. At WGU Labs, we wanted to see if these scenarios could help learners practice handling different situations and also foster the development of a comprehensive skill set that includes both technical knowledge and essential P21 or durable skills like communication, empathy, ethical decision-making, and critical thinking. To do so, we developed custom GPTs to help learners practice these skills for three different applications: parent/teacher conferences, customer service, and instructional design.
Exploring the Potential of Custom GPTs
Parent/Teacher Conference Preparation
The first custom GPT we designed to explore the potential of role-play in formative assessment was the Parent/Teacher Conference Prep Buddy. Effective communication in parent-teacher interactions is essential for educators to meet students' needs and build partnerships, yet opportunities to practice these skills are limited. The Parent/Teacher Simulation GPT addresses this by offering a safe environment for educators to apply best practices in communication. Using Chat GPT's voice capabilities, the simulation enables more natural, real-time conversations, allowing users to practice not only content but also tone, pacing, and non-verbal cues. This added realism sharpens skills such as active listening, empathy, and clear articulation, fostering positive relationships with parents and supporting student success.
Initial user testing with faculty provided valuable insights on using this tool to build necessary competencies. Based on feedback, we enhanced the design with more detailed student and parent profiles and included background information, such as student data and records of prior conversations, for context. The GPT was programmed to simulate a frustrated parent, requiring learners to quickly apply problem-solving and empathetic listening skills. Faculty using the voice feature found the interactions authentic and relevant, and they noted that repeated use could ease the anxiety many new teachers experience during their first parent-teacher conferences.
Customer Service Simulation
To explore how ChatGPT can help learners practice challenging conversations in other settings, we developed the Customer Service GPT. This tool allows users to practice de-escalation techniques by handling irate customers in various contexts. The simulations replicate real-life pressure and emotional intensity while providing a safe environment for learners to experiment and refine their responses. Faculty and staff who tested the GPT found the interactions realistic and noted that using voice made the experience feel much more authentic.
Instructional Design Simulation
Lastly, we leveraged the same principles with which we created the other GPTs to develop a tool that would help pre-service teachers develop personalized instructional methods. The Universal Design for Learning (UDL) in Action GPT is designed to provide pre-service teachers with practical experience in applying Universal Design for Learning (UDL) principles to differentiate instruction in diverse classroom scenarios. Differentiating instruction and applying Universal Design for Learning (UDL) principles are complex skills that require significant time, practice, and experience to master. In traditional pre-service teacher education programs, opportunities for hands-on practice in these areas are often limited, making it challenging for future educators to develop the necessary competencies. The "UDL in Action GPT" addresses this gap by providing a simulated environment where pre-service teachers can repeatedly practice and refine their skills in applying UDL principles, ultimately leading to more effective and inclusive teaching practices.
Key Takeaway: Designing GPTs to be easily personalized enhances learning
Each custom GPT integrates an element of choice to enhance the learning experience. In the Parent/Teacher Conference GPT, learners first select the grade level and subject area they would like to teach, then choose from three unique student profiles to focus on during the conference. In the Customer Service GPT, learners select the type of environment they want to practice in, such as retail, hospitality, food service, call center, or healthcare. In the UDL in Action GPT, learners pick a grade level and subject area, then work to modify a provided lesson plan to meet the needs of a specific learner profile. Initial user feedback highlighted the value of this choice element, as it allows learners to tailor their practice to scenarios most relevant to their future career goals.
The Role of AI in Building Essential Skills
Developing P21 and durable skills is critical in education and healthcare where effective communication, conflict resolution, and empathy are paramount. Through AI-driven role-play, preservice teachers can engage in scenarios that require them to navigate complex interpersonal dynamics. These simulations provide a risk-free environment where learners can practice and refine their responses. They can make mistakes and learn from them without the fear of real-world repercussions. This iterative practice builds their confidence and prepares them for the nuanced situations they will face in their careers.
Aligning Role-Play Activities with Learning Objectives
To ensure that these AI-driven role-play activities are purposeful and aligned with educational goals, educators can develop rubrics that tie the simulations to specific learning objectives. This approach ensures that each role-play scenario not only tests the learner’s ability to respond to a situation but also reinforces key competencies required in their field.
For instance, a rubric for a classroom management scenario might assess the preservice teacher’s ability to de-escalate the situation, maintain student engagement, and apply appropriate disciplinary measures. Similarly, a healthcare simulation could evaluate the learner’s ability to communicate effectively with patients, maintain confidentiality, and make ethical decisions.
Immediate Feedback: The AI Advantage
One of the key benefits of using custom GPTs in formative assessment is their ability to provide immediate, constructive feedback. After a role-play exercise, a well-designed GPT can offer detailed insights into the learner’s performance, highlighting strengths and areas for improvement. This real-time feedback accelerates learning by encouraging reflection and reinforcing positive behaviors while helping to correct mistakes promptly. When the GPT is programmed with rubrics aligned to learning objectives, it offers specific, targeted feedback, clearly identifying where the learner can grow and improve.
Challenges and Limitations in Current GPT Implementations
As promising as AI-driven role-play activities are, there are still challenges to overcome. For instance, while voice interaction enhances the realism of these scenarios, current limitations in Chat GPT's voice capabilities pose significant barriers. The desktop app, which is optimal for extended role-play sessions, is currently only available on macOS 14+ with Apple Silicon (M1 or better), excluding a substantial number of potential users on Windows. The mobile app does offer voice features, but processing detailed text-based feedback on a smaller screen can be cumbersome.
Creating and engaging with custom GPTs for extended periods requires an enterprise account. During user testing, we had to share our screens to allow participants to interact with the role-play activities if they didn’t have their own paid accounts. This limitation quickly revealed that Chat GPT is more suited as a prototyping tool for these interactions. To integrate these activities into our courses in the near future, we’ll need to transition to a more accessible platform.
Additionally, we’ve noticed that despite programming the GPT to engage as a challenging persona, it often becomes more agreeable after initial resistance. If a user responds very defensively, the GPT may shift from a difficult persona to more of an instructional coach. This behavior suggests that the underlying programming of Chat GPT is designed to avoid sustained conflict, which can diminish the effectiveness of these role-play activities. Addressing these issues is crucial as we continue to refine the tool to better meet educational needs.
The Future of AI in Formative Assessment
As we continue to develop and refine these custom GPTs, we are committed to sharing our findings and insights. Our goal is to empower educators and learners with tools that enhance learning experiences and prepare them for the complexities of their future careers. The integration of AI into formative assessment is not just about innovation for innovation's sake; it’s about making learning more accessible, effective, and aligned with real-world demands.
