Abstract
FILLER TEXT
Introduction
Many adolescents have experienced cyberbullying, such as offensive name-calling, purposeful embarrassment, physical threats, and sexual harassment. Instances of cyberbullying are associated with youth depression, self-harm, and even suicide attempts. Large language models (LLMs) pose the risk of increasing the level of toxic online interactions even more, further jeopardizing youth’s online safety and digital well-being. The intervention of bystanders, so-called upstanding, is an effective approach to support the victims, but adolescents struggle in taking this role. It is, therefore, an important skill to learn and practice for digital interactions. Faced with a wide teacher shortage, especially in subjects that teach upstanding to cyberbullying like technology or health class, it is doubtful that students can receive enough personal attention to learn how to be upstanders.
Teacher-built chatbots could scale up personalized instruction about how to upstand to cyberbullying. While promising, previous research findings were limited to primarily Wizard-of-Oz studies. Translating them into actual chatbots that have an impact in the classroom requires solving technical issues around lack of data and necessitates that the chatbot fits into the wider curriculum. Giving teachers control of LLM-based chatbots could solve both.
LLM-Chains give non-AI-experts the ability to build LLM applications with fine-grained control, but it is unknown if and how they can address the teachers’ needs. LLMs drastically reduce training data requirements and with LLM-Chains, non-AI-experts can design a flow of individually configured LLMs to solve a larger task. It is thus a promising approach for teacher-built chatbots. For chatbots, LLM-Chains have, however, only been evaluated on simple toy tasks so far and it is unclear if they can enable teachers to build complex chatbots that teach teens upstanding skills.
In this work, we investigate to what extend LLM-Chains are a suitable approach to empower teachers to build chatbots that fit into their upstanding-to-cyberbullying education and what other kinds of support (or "levers") they need. We have developed a prototyping platform to evaluate conversational AI interventions that cultivate teen upstanding behaviors. Leveraging this platform, we built a system as a probe and invited 13 middle school teachers to explore building a chatbot, collecting their experiences through think-aloud and interviews, which allowed us to gain their in-depth perspectives. With our probe, the teachers could gain hands-on experience building and interacting with the chatbot, thus providing deeper insights into their needs than discussing purely hypothetical situations.
Our findings show that teachers’ needs for levers reflect their larger chatbot design goal:
To construct a piece of educational theatre, where teens learn by rehearsing different upstanding behaviors in the social situation surrounding concrete instances of cyberbullying.
Teachers perceive their role as "playwrights" wanting to write a script for role-play social situations, ensuring that the chatbot guides students to specific behaviors while allowing students to explore different perspectives. This mindset shapes their needs for levers to further personalized instruction. To give just one example, LLM-Chains enable teachers to customize the chatbot to their class. However, new levers are necessary to allow for more controlled improvisations so students can practice upstanding more concretely, applying their knowledge to commonly encountered situations. We discuss the implications of these findings for designing levers that enhance the instructional value of chatbots for cyberbullying interventions and identify new research questions that still need to be answered in the context of chatbot use for classroom instruction.
This paper makes two contributions. First, it presents a rare description of how teachers envision using chatbots in their classrooms for K-12 prosocial online behavior education and furthers our understanding of what design and technical components can help them reach their goals. Second, it identifies new research and design opportunities about how LLMs and chatbot design tools can deliver on teachers’ needs and ensure that chatbots can have an actual impact in the classroom. While LLMs are often seen as disruptive to teachers’ educational and evaluative work, our work offers a complimentary perspective on how LLMs can augment it by delivering teacher-orchestrated and student-improvised personalized instruction.
Conclusion
In this work, we explore what technical and design components teachers need to build chatbots that assist in bystander education through Co-Pilot, an LLM-Chain based, no-code chatbot design tool. To create chatbot tools that fulfill teachers’ needs, tool designers will want to consider the teachers’ goal of constructing role-play scenarios and their perception of being playwrights of these social interactions. Teachers want to control and adapt the chatbot while at the same time allowing the chatbot enough improvisation so that students can explore different bystander actions and scenarios and practice socio-emotional skills. This view helps to understand how far current language model technology can be utilized for chatbot building and what new solutions still need to be found. We hope that researchers and designers of future tools will consider these factors to ensure that chatbots for adolescent cyberbullying education have a successful impact in the classroom.
