Browsing by Author "Celik, B."

Now showing 1 - 6 of 6

Automating Thematic Analysis With Multi-Agent LLM Systems
(CEUR-WS, 2025) Sankaranarayanan, S.; Borchers, C.; Simon, S.; Tajik, E.; Atas, A.H.; Celik, B.; Shahrokhian, B.
Thematic analysis (TA) is a method used to identify, examine, and present themes within data. TA is often a manual, multistep, and time-intensive process requiring collaboration among multiple researchers. TA’s iterative subtasks, including coding data, identifying themes, and resolving inter-coder disagreements, are especially laborious for large data sets. Given recent advances in natural language processing, Large Language Models (LLMs) offer the potential for automation at scale. Recent literature has explored the automation of isolated steps of the TA process, tightly coupled with researcher involvement at each step. Research using such hybrid approaches has reported issues in LLM generations, such as hallucination, inconsistent output, and technical limitations (e.g., token limits). This paper proposes a multi-agent system, differing from previous systems using an orchestrator LLM agent that spins off multiple LLM sub-agents for each step of the TA process, mirroring all the steps previously done manually. In addition to more accurate analysis results, this iterative coding process based on agents is also expected to result in increased transparency of the process, as analytical stages are documented step-by-step. We study the extent to which such a system can perform a full TA without human supervision. Preliminary results indicate human-quality codes and themes based on alignment with human-derived codes. Nevertheless, we still observe differences in coding complexity and thematic depth. Despite these differences, the system provides critical insights on the path to TA automation while maintaining consistency, efficiency, and transparency in future qualitative data analysis, which our open-source datasets, coding results, and analysis enable. © 2025 for this paper by its authors.
Design, Development and Implementation of a Question-Answer Based Instructional Design Theory
(Iated-int Assoc Technology Education & development, 2017) Atas, A.; Celik, B.; Yildirim, Z.
The purpose of this study is to document design, development, and implementation of a question-answer based instructional design theory. Sigel and Saunders (1977) state that many studies report the benefits of asking questions and being asked questions as enhancing students' thinking and reflects students' thinking process. The impact of asking question or being asked questions are inevitably influential on learning. As students experience quality questions and questioning, their awareness of learning arise (Hunkins, 1995). Walsh and Sattes (2005) also state that when a student asks questions, it indicates that student is not absent-minded; in fact, motivated and attentive. Considering the possible impact of questioning on learning, it is envisioned that preparing course related questions and asking them during the instruction (before, during, after) can contribute students' learning. The main focus of this theory is to create a disequilibrium/cognitive conflict in students' minds in order to probe them to think about their learning process. The source of disequilibrium might be the questions asked by the instructor or students' own internal processes as well. Under this epistemological approach, QAS is an eclectic approach combines principles of various learning theories. Basically, Piaget's Cognitive Development Theory and Bandura's (2009) Triadic Reciprocal Determinism Model framed this approach. The main aim of creating interaction among student and instructor through questions and answers is to create cognitive conflict mentioned by Piaget (1977) as well. The reciprocal inquiry circle during in-class instruction also is a reflection of Bandura's view. This circle represent the continuous questionanswer process between instructor and students like the reciprocal effect of behavior, personal factors and environment in Triadic Reciprocal Determinism Model. Reigeluth and Frick's (1999) Formative Research was employed throughout the research by conducting two main iterations. Data collection strategies were in-class observations focusing on working and not working parts of theory, questionnaires aiming to gather students' evaluation on possible impact of the theory implementation on their learning, motivation and as the last semi-structured interviews to get deep interpretations and evaluations. The convenient sampling strategies was followed by enrolling 12 sophomore students who took instructional design course in a state university in Ankara, Turkey. The implementation lasted 6 weeks. The data were analyzed and reported by considering effectiveness (impact of questions, provided feedback on learning), efficiency (questions types, time, number of questions, technology use) and appeal (fun, motivation, satisfaction, anxiety) of the question-based instructional theory.
Enhancing Educational Assessment Through Genai: Leveraging Revised Bloom's Taxonomy for Question Generation With Chatgpt
(Peter Lang AG, 2025) Atas, A.H.; Celik, B.; Aldemir, T.
With advancing technology, effective evaluation methods are increasingly crucial. This chapter examines how well ChatGPT can generate educational assessment questions on the cognitive process dimension of the Revised Bloom's Taxonomy (RBT), consisting of six levels: remember, understand, apply, analyze, evaluate, and create. The chapter begins with emphasizing the significance of AI-driven technology in educational assessment and discusses related studies. Then Bloom Taxonomy's original and revised versions were explained. To illustrate the practice of generating different types of questions, multiple-choice and open-ended questions were generated with ChatGPT. We offered a preliminary prompt structure with determined variables: the role, number of questions, type of questions, and cognitive process level of RBT. To check the efficiency of the prompt, questions were generated across six cognitive process levels of RBT. Additionally, the pitfalls of generating assessment questions by ChatGPT and suggestions for further research were addressed. Consequently, by providing a structured framework for various cognitive levels, RBT significantly facilitates the use of prompt engineering in generating questions with ChatGPT. Although it may not yield successful results to the desired extent regarding learning outcomes, ChatGPT appears to have high potential as a valuable resource for educators in the initial stages of question generation. Lastly, it is worth emphasizing that repeating the process of generating questions with ChatGPT in different domains and using ChatGPT-generated questions without being filtered through the cognitive verification of the teacher for validity and meaningfulness should be avoided at this stage. Further research is recommended to explore the effectiveness and comprehensibility of the ChatGPT-generated questions. © 2024 Peter Lang Group AG, Lausanne.
Expert Heuristic Collaborative Approach for Usability Testing in a Mooc Portal
(Jomard Publishing, 2023) Celik, B.; Esfer, S.; Cagiltay, K.
Despite the widespread use of Massive Open Online Courses (MOOCs) in online learning, the usability of MOOC portals remains a major challenge. While expert heuristic evaluation principles have been proposed as a potential solution to usability problems, there is a lack of research on the effectiveness of these principles in a real-life and big-size project context. This case study examines the usability test process derived from expert heuristic evaluation principles within a national MOOC portal context and identifies how a collaborative approach can be used to enhance the effectiveness of usability testing. Since MOOCs meet the needs of the learners by keeping the investment to a minimum and creating close links to user interaction, we, as instructional designers, experienced a continuous and sequentially usability test process from the beginning of the design stage tests of 100 MOOCs to the end-user test phases. That’s why the mainly expert heuristic and collaborative approach (EHCA) is used for finding the usability problems in an iterative design process as Nielsen emphasized. Between 2015-2017, a distinctive, ongoing procedure based on immediate feedback and reliability was performed by both instructional designers (n=8), online learning experts (n=4), quality assurance testers (n=4) and end users (n=10) through Nielsen's major test components (learnability, efficiency, memorability, errors, and satisfaction). The test documents were collected, semi-structured interviews were done, testing processes were observed throughout the MOOC portal creation process, and the “collaborative approach” focusing on more expert collaboration and gradual evaluation, immediate feedback, and revised test procedure, and more prototypes and fewer problems were embraced. This approach which answers the question of “how to make a more effective evaluation of the usability of the MOOC portal” can be considered a prototype of a structure that can give new design suggestions over the years. One of the contributions of the study can be summarized as providing the relationships between the expert heuristic and collaborative approach with real-life and big-size project evidence. So, the implications made for theory and practice have the potential to make critical contributions to the usability test process. Hence, in this study, we tried to offer a flexible and collaborative usability testing process with expert team members and several methods to minimize the effort and time for testing by always dealing with the usability problems of MOOC portals during the process. Therefore, the outputs of this study can be beneficial to various MOOC portals or any kind of online learning platform in terms of sustainability and efficiency. © 2023, Jomard Publishing. All rights reserved.
Prognostic Factors in Oligodendrogliomas: a Clinical Study of Twenty-Five Consecutive Patients
(Asian Pacific Organization for Cancer Prevention, 2015) Atalay, T.; Ak, H.; Celik, B.; Gulsen, I.; Seckin, H.; Tanik, N.; Bavbek, M.
Background: The purpose of this study was to evaluate the prognostic significance of Ki-67 and subjective microvascular density (SMVD) indexes together with other factors in patients with oligodendroglioma. Materials and Methods: In this retrospective study, oligodendroglioma specimens obtained from twenty-five consecutive patients were evaluated for Ki-67 and SMVD indices to help determine histological grading and investigate the fidelity of these markers in clinical prognosis. Other potentially prognostic factors were Karnofsky performance scale, tumor histological grade, and adjuvant radiotherapy. Results: The Ki-67 proliferation index appeared to have a strong correlation with the grade of the tumor and the survival. Age, gender, adjuvant radiotherapy, surgical resection type (complete versus incomplete) did not have any influence on recurrence. The SMVD index correlated significantly with the 3 to 5-year survival. Conclusions: Ki-67 and MVD indexes are important and useful markers in estimating the prognosis of oligodendrogliomas.
A Systematic Design Model for Gamified Learning Environments: Geld Model
(IGI Global, 2021) Aldemir, T.; Atas, A.H.; Celik, B.
This formative research study is an attempt to develop a design model for gamified learning experiences situated in real-life educational contexts. This chapter reports on the overall gamification model with the emphasis on the contexts and their interactions. With this focus, this chapter aims to posit an alternative perspective to existing gamification design praxis in education which mainly focuses on separate game elements, by arguing that designing a gamified learning experience needs a systematic approach with considerations of the interrelated dimensions and their interplays. The study was conducted throughout the 2014-15 academic year, and the data were collected from two separate groups of pre-service teachers through observations and document collections (n=118) and four sets of interviews (n=42). The results showed that gamification design has intertwined components that form a fuzzy design model: GELD. The findings also support the complex and the dynamic nature of gamified learning design, and the need for a more systematic approach to design and development of such experiences. © 2022, IGI Global.