Does Using Prompting Strategies Affect the Production of Higher-Quality Socio-Scientific Issue Activity? Analysis of Nuclear Energy and Pesticide Use Cases for High School Chemistry Teaching

Abstract

This study analyzed the socio-scientific issue (SSI) activity plans for nuclear energy and pesticide use produced by ChatGPT 3.5 and ChatGPT 5 and took five chemistry teachers' ideas about the plans. Seven SSI activities were formed using varying prompt strategies for each SSI topic, and 14 chemistry SSI activities were analyzed using an SSI-based framework. Results showed that the quality of activity produced by ChatGPT 3.5 and 5.0 is heterogeneous and depends on the prompt type. SSI activities obtained with simple prompts received a minimum score in both SSI topics. Chain-of-thought (CoT) prompt type scored almost at a maximum in both SSI types. While the scores of the activities on the two different SSI topics according to the prompt types generally show a homogeneous trend, this is not the case for the output formatting 5E strategy. Pesticide use activity received 5 points, whereas nuclear energy activity generated through the 5E prompt received 17 points. Additionally, ChatGPT 5 can produce more consistent, comprehensive, and pedagogically stronger content, especially with guided and structured prompt types (e.g., CoT). However, most SSI indicators are still not adequately represented in the content generated with the simple prompt. Chemistry teachers found most of the SSI activities to be interesting and applicable. To conclude, the quality of the activities generated through prompting types varies. Furthermore, the mastery of the SSI framework by chemistry teachers or preservice teachers who utilize ChatGPT to generate SSI-based activity plans also impacts the quality of the output. To conclude, users familiar with the prompting techniques and the SSI framework will produce pedagogically sound outputs.