Browsing by Author "Tat, Osman"

Now showing 1 - 3 of 3

Discovering Hidden Patterns: Applying Topic Modeling in Qualitative Research
(Assoc Measurement & Evaluation Education & Psychology, 2024) Tat, Osman; Aydogan, Izzettin
In qualitative studies, researchers must devote a significant amount of time and effort to extracting meaningful themes from large sets of texts and examining the links between themes, which are frequently done manually. The availability of natural language models has enabled the application of a wide range of techniques to automatically detecting hierarchy, linkages, and latent themes in texts. This paper aims to investigate the coherence of the topics acquired from the analysis with the predefined themes, as well as the hierarchy between topics, the similarity, and the proximity-distance between topics by means of the topic model based on BERTopic using unstructured qualitative data. This paper aims to investigate the coherence of the topics acquired from the analysis with the predefined themes, as well as the hierarchy between topics, the similarity, and the proximity-distance between topics by means of the topic model based on BERTopic using unstructured qualitative data. The qualitative data for this study was gathered from 106 students engaged in a university-run pedagogical formation certificate program. In BERTopic procedure, the paraphrase-multilingual-MiniLM-L12-v2 model was used as the sentence transformer model, UMAP was used as the dimension reduction method, and HDBSCAN algorithm as the clustering method. It was found that BERTopic successfully identified six topics corresponding to the six predicted themes in unstructured texts. Moreover, 74% of the texts containing some certain themes could be classified accurately. The algorithm effectively discerned which themes were analogous and which had significant distinctions from others. It was concluded that BERTopic is a procedure which is capable of identifying themes that researchers may not notice, depending on the data density in qualitative data analysis, and has the potential to enable qualitative research to reach more detailed findings.
Investigating the Performance of Artificial Neural Networks in Predicting Affective Responses
(Assoc Measurement & Evaluation Education & Psychology, 2025) Tat, Osman; Aydoğan, İzzettin
In this study it is aimed to examine the performance of an artificial neural network trained using items reflecting a latent trait in predicting responses to an item reflecting the same trait. This latent trait is the awareness of being able to communicate with people from different cultures, which is included in the PISA 2018 assessment. Relevant scale items were used as research variables. In addition to determining the extent to which the predicted responses overlap with the actual responses by analyzing the artificial neural network models, it was examined how the predicted responses affect the assumed latent construct and the reliability of the responses. Thus, the performance of artificial neural networks in predicting responses to affective items was evaluated. The responses expected from individuals for the items examined overlap with the responses given by individuals at a relatively moderate. However, it is observed that although the prediction values partially weaken the model fit indices, they still manage to keep them strong. In addition, the predicted values improved the factor loadings and the variance explained for the latent trait. Similarly, it is noticed that the predicted values also positively affect the reliability.
How Justified Are the Criticisms of Bias Against the Oecd's Global Competence Framework
(Pergamon-elsevier Science Ltd, 2025) Aydogan, Izzettin; Tat, Osman
In psychometric terms, bias is defined as the failure to ensure the psychological equivalence of the latent traits and related items intended to be measured by measurement procedures for different groups and therefore the measurements produce results in favor of or against at least one group. There has been both public and scientific criticism that it contains bias in terms of some features such as the OECD's global competence framework is designed according to the principles of the western liberal tradition, that students interpret words and expressions in some items differently, that their understanding of poverty and privilege is limited, and that students' lack of access to communication tools due to socio-economic conditions. In this research, we examined whether the OECD's global competence framework presented to students in the PISA 2018 assessment is statistically biased in terms of sociological features that encompass and even go beyond criticisms of the framework. In this context, we used six international classification indices and analyzed data on around 143 thousand students from 27 PISA participating countries. We believe that results will clarify criticisms of bias in the OECD's global competence framework.