Students’ Critical Thinking Disposition in Artificial Intelligence-Assisted Mathematics Learning

Dilla Santria Ulfa; Fiki Alghadari; Bella Arisha; Janina C. Sercenia

doi:10.58421/misro.v5i4.2015

DOI:

https://doi.org/10.58421/misro.v5i4.2015

Authors

Dilla Santria Ulfa Universitas Jambi
Fiki Alghadari Universitas Jambi
Bella Arisha Universitas Jambi
Janina C. Sercenia De La Salle University

Keywords:

Cognitive passivity, Critical evaluation, Critical thinking disposition, Artificial intelligence, Reflective monitoring

Abstract

Artificial Intelligence (AI) can be used by students to learn and solve math problems in seconds. However, the learning habits of Indonesian students tend to rarely involve the ability to think at a higher level. This study aims to analyze students’ critical thinking dispositions in AI-assisted learning based on factor structure and experience category mapping. This study involved students (N=99) at a junior high school in the city of Jambi, Indonesia, so the findings are exploratory and limited to the context of the school. Data were collected using a perception-scale questionnaire on AI-assisted math learning behavior, analyzed with Exploratory Factor Analysis (EFA) and Rasch analysis. The findings identified five factors as the basis of student response patterns associated with critical thinking dispositions in AI-assisted math learning—reflective monitoring, internalization of understanding, passive dependence on AI, critical evaluation, and cognitive load. The mapping results showed that most students rarely experienced cognitive overload or internal understanding. At the same time, only a small percentage routinely performed reflective monitoring and critical evaluation, accompanied by early indications of a tendency toward passive dependence on AI. However, these findings need to be interpreted conservatively given the relatively low reliability of these factors. Based on the difficulty level of item approval in Rasch’s analysis, student responses showed inconsistencies in several factors, which could indicate a risk of cognitive passivity; However, these findings reflect the overall difficulty level in item approval, not a direct classification of each student’s level of disposition. This is especially true of the passive dependency factor on AI, which has the lowest reliability among the five factors. Therefore, strengthening working memory in AI-assisted mathematics learning requires an emphasis on reflection, evaluation, and justification activities to support knowledge development through internalization.

Downloads

Download data is not yet available.

References

M. Turmuzi, S. Azmi, and N. M. I. Kertiyani, “ChatGPT in school mathematics education: A systematic review of opportunities, challenges, and pedagogical implications,” Teach. Teach. Educ., vol. 170, p. 105286, Feb. 2026, doi: 10.1016/j.tate.2025.105286.

S. Schorcht, F. A. Müller, and N. Buchholtz, “No one-size-fits-all: a study of prompt techniques and large language models to enhance AI’s mathematics educational quality,” ZDM – Math. Educ., pp. 1–15, Apr. 2026, doi: 10.1007/s11858-026-01784-6.

W. McGalliard and S. Otten, “AI Responses to Challenging Problems and Educator Responses to AI Availability,” Digit. Exp. Math. Educ., vol. 11, no. 2, pp. 319–332, Aug. 2025, doi: 10.1007/s40751-024-00167-4.

B. Pepin, N. Buchholtz, and U. Salinas-Hernández, “A Scoping Survey of ChatGPT in Mathematics Education,” Digit. Exp. Math. Educ., vol. 11, no. 1, pp. 9–41, Apr. 2025, doi: 10.1007/s40751-025-00172-1.

G.-J. Hwang, H. Xie, B. W. Wah, and D. Gašević, “Vision, challenges, roles and research issues of Artificial Intelligence in Education,” Comput. Educ. Artif. Intell., vol. 1, p. 100001, 2020, doi: 10.1016/j.caeai.2020.100001.

H. Bastani, O. Bastani, A. Sungu, H. Ge, Ö. Kabakcı, and R. Mariman, “Generative AI without guardrails can harm learning : Evidence from high school mathematics,” PNAS (Proceedings Natl. Acad. Sci., vol. 122, no. 26, pp. 1–8, 2025, doi: 10.1073/pnas.2422633122/-/DCSupplemental.Published.

C. Walkington, “The implications of generative artificial intelligence for mathematics education,” Sch. Sci. Math., no. February, pp. 1–10, 2025, doi: 10.1111/ssm.18356.

B. Liu, W. Zhang, and F. Wang, “Can Generative Artificial Intelligence Effectively Enhance Students’ Mathematics Learning Outcomes?—A Meta-Analysis of Empirical Studies from 2023 to 2025,” Educ. Sci., vol. 16, no. 1, p. 140, Jan. 2026, doi: 10.3390/educsci16010140.

N. C. Facione, P. A. Facione, and C. A. Giancarlo, “The Disposition Toward Critical Thinking as a Measure of Competent Clinical Judgment : The Development of the California Critical Thinking Disposition Inventory 1,” J. Nurs. Educ., vol. 33, no. 1994, pp. 345–350, 1994.

S. Wang and H. Zhang, “Pedagogical partnerships with generative AI in higher education: how dual cognitive pathways paradoxically enable transformative learning,” Int. J. Educ. Technol. High. Educ., vol. 23, no. 1, p. 11, Mar. 2026, doi: 10.1186/s41239-026-00585-x.

OECD, “PISA 2022 Results,” 2023.

I. R. Khaesarani and R. Ananda, “Students ’ mathematical literacy skills in solving higher- order thinking skills problems,” Al-JabarJurnal Pendidik. Mat., vol. 13, no. 1, pp. 81–99, 2022.

B. E. Susilo, D. Darhim, and S. Prabawanto, “Critical thinking skills based on mathematical dispositions in problem-based learning,” J. Phys. Conf. Ser., vol. 1567, no. 2, p. 022101, Jun. 2020, doi: 10.1088/1742-6596/1567/2/022101.

E. D. Minarti, F. Alghadari, and M. Hutajulu, “Mathematical disposition ability and critical thinking: Evaluation of middle school students,” J. Phys. Conf. Ser., vol. 1657, no. 1, p. 012017, Oct. 2020, doi: 10.1088/1742-6596/1657/1/012017.

S. Gökçe and P. Güner, “Pathways from cognitive flexibility to academic achievement: mediating roles of critical thinking disposition and mathematics anxiety,” Curr. Psychol., vol. 43, no. 20, pp. 18192–18206, May 2024, doi: 10.1007/s12144-024-05642-0.

P. Güner and S. Gökçe, “Linking critical thinking disposition, cognitive flexibility and achievement: Math anxiety’s mediating role,” J. Educ. Res., vol. 114, no. 5, pp. 458–473, Sep. 2021, doi: 10.1080/00220671.2021.1975618.

L. L. Li, G. Mao, L. Zhang, and J. Ying, “The role of peer learning in enhancing prompt literacy: effects on speech writing quality and critical thinking disposition in AI-assisted contexts,” Educ. Inf. Technol., vol. 31, no. 4, pp. 1233–1255, Feb. 2026, doi: 10.1007/s10639-025-13849-2.

N. Lapawi, K. Osman, and H. Husnin, “Impact of Generative Artificial Intelligence (AI) as a Tool on Disposition and Computational Thinking Skills in STEM Education,” in GenAI in Novel Educational Applications: Practices of Integrating GenAI in the K-12 Classroom, M. E. Auer and S. A. Nikou, Eds., Springer Nature Switzerland, 2026, pp. 93–116. doi: 10.1007/978-3-032-16157-4_5.

Z. Wang, W. Li, F. Liu, J. Yao, and Y. Wang, “Enhancing or eroding? Exploring how generative artificial intelligence usage influences college students’ critical thinking disposition,” Curr. Psychol., vol. 45, no. 11, p. 1033, Jun. 2026, doi: 10.1007/s12144-026-09589-2.

C. Zhai, S. Wibowo, and L. D. Li, “The effects of over-reliance on AI dialogue systems on students’ cognitive abilities: a systematic review,” Smart Learn. Environ., vol. 11, no. 1, p. 28, Jun. 2024, doi: 10.1186/s40561-024-00316-7.

M. Gerlich, “AI Tools in Society : Impacts on Cognitive Offloading and the Future of Critical Thinking,” Societies, vol. 15, no. 6, pp. 1–28, 2025, doi: https://doi.org/10.3390/soc15010006.

W. Liu, H. Ma, S. Ma, Y. Chen, and G. Li, “The development and validation of a critical thinking disposition scale for high school students,” Acta Psychol. (Amst)., vol. 262, no. October 2025, 2026, doi: 10.1016/j.actpsy.2025.106055.

S. Ivanov, “The dark side of artificial intelligence in higher education,” Serv. Ind. J., vol. 43, no. 15–16, pp. 1055–1082, 2023.

M. A. Tashtoush, A. B. Qasimi, N. H. Shirawia, and L. A. Hussein, “The Efficacy of Utilizing Artificial Intelligence Techniques in Developing Critical Thinking in Mathematics among Secondary School Students and their Attitudes Toward it,” Iraqi J. Comput. Sci. Math., vol. 6, no. 1, pp. 37–50, Feb. 2025, doi: 10.52866/2788-7421.1231.

B. Wu, Y.-N. He, Y. Song, and H.-H. Li, “Fostering critical thinking in higher education: an intelligent dialogue-based approach empowered by conversational AI,” Interact. Learn. Environ., vol. 34, no. 4, pp. 2061–2078, Apr. 2026, doi: 10.1080/10494820.2025.2538750.

H. Lijie, S. Mat Yusoff, and A. F. Mohamad Marzaini, “Influence of AI-driven educational tools on critical thinking dispositions among university students in Malaysia: a study of key factors and correlations,” Educ. Inf. Technol., vol. 30, no. 6, pp. 8029–8053, Apr. 2025, doi: 10.1007/s10639-024-13150-8.

N. R. Nasr, C.-H. Tu, L. Sujo-Montes, S. Haniya, C.-J. Yen, and S. Hossain, “Generative artificial intelligence impact on critical thinking in higher education: a comprehensive bibliometric analysis and systematic review,” J. Comput. Educ., pp. 1–42, Jun. 2026, doi: 10.1007/s40692-026-00396-z.

W. A. Edmonds and T. D. Kennedy, An applied guide to research designs: Quantitative, qualitative, and mixed methods. Sage Publications, 2016.

P. A. Facione, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill,” Informal Log., vol. 20, no. 1, pp. 61–84, Jan. 2000, doi: 10.22329/il.v20i1.2254.

A. G. Yong and S. Pearce, “A Beginner’s Guide to Factor Analysis: Focusing on Exploratory Factor Analysis,” Tutor. Quant. Methods Psychol., vol. 9, no. 2, pp. 79–94, Oct. 2013, doi: 10.20982/tqmp.09.2.p079.

A. Field, Discovering statistics using IBM SPSS statistics. Sage Publications Limited, 2024.

T. G. Bond and C. M. Fox, Applying the Rasch model: Fundamental measurement in the human sciences. Psychology Press, 2015.

D. Kember et al., “Development of a Questionnaire to Measure the Level of Reflective Thinking,” Assess. Eval. High. Educ., vol. 25, no. 4, pp. 381–395, Dec. 2000, doi: 10.1080/713611442.

J. Říčan, V. Chytrý, and J. Medová, “Aspects of self-regulated learning and their influence on the mathematics achievement of fifth graders in the context of four different proclaimed curricula,” Front. Psychol., vol. 13, p. 963151, Oct. 2022, doi: 10.3389/fpsyg.2022.963151.

B. Rittle-Johnson, A. M. Loehr, and K. Durkin, “Promoting self-explanation to improve mathematics learning: A meta-analysis and instructional design principles,” ZDM, vol. 49, no. 4, pp. 599–611, Aug. 2017, doi: 10.1007/s11858-017-0834-z.

A. Ghanizadeh, “The interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education,” High. Educ., vol. 74, no. 1, pp. 101–114, Jul. 2017, doi: 10.1007/s10734-016-0031-y.

N. Nizaruddin and I. Kusmaryono, “How are Critical Thinking Skills Related to Students’ Self-Regulation and Independent Learning?,” Pegem J. Educ. Instr., vol. 13, no. 4, pp. 85–92, Aug. 2023, doi: 10.47750/pegegog.13.04.10.

R. R. Mohammadi, M. Saeidi, and A. Abdollahi, “Modelling the interrelationships among self-regulated learning components, critical thinking and reading comprehension by PLS-SEM: A mixed methods study,” System, vol. 117, p. 103120, Oct. 2023, doi: 10.1016/j.system.2023.103120.

W. Zeng et al., “Promotion of self-regulated learning through internalization of critical thinking, assessment and reflection to empower learning (iCARE): A quasi-experimental study,” Nurse Educ. Today, vol. 142, p. 106339, Nov. 2024, doi: 10.1016/j.nedt.2024.106339.

D. S. Pambudi, “Exploration of Prospective Mathematics Teachers’ Mathematical Connections When Solving the Integral Calculus Problems Based on Prior Knowledge,” Eur. J. Math. Sci. Educ., vol. 3, no. 2, pp. 105–116, Dec. 2022, doi: 10.12973/ejmse.3.2.105.

B. A. Simonsmeier, M. Flaig, A. Deiglmayr, L. Schalk, and M. Schneider, “Domain-specific prior knowledge and learning: A meta-analysis,” Educ. Psychol., vol. 57, no. 1, pp. 31–54, Jan. 2022, doi: 10.1080/00461520.2021.1939700.

J. Schoenherr and S. Schukajlow, “Characterizing external visualization in mathematics education research: a scoping review,” ZDM – Math. Educ., vol. 56, no. 1, pp. 73–85, Feb. 2024, doi: 10.1007/s11858-023-01494-3.

J. Schoenherr, A. R. Strohmaier, and S. Schukajlow, “Learning with visualizations helps: A meta-analysis of visualization interventions in mathematics education,” Educ. Res. Rev., vol. 45, p. 100639, Nov. 2024, doi: 10.1016/j.edurev.2024.100639.

P. Ninghardjanti, M. C. Umam, A. Subarno, W. Winarno, N. R. Langgi, and J. Widodo, “Evaluating the impact of AI on the critical thinking skills among the higher education students by combining the TAM model and critical thinking theory,” Front. Educ., vol. 10, p. 1719625, Nov. 2025, doi: 10.3389/feduc.2025.1719625.

F. M. Singer and C. Voica, “Playing on patterns: is it a case of analogical transfer?,” ZDM – Math. Educ., vol. 54, no. 1, pp. 211–229, Apr. 2022, doi: 10.1007/s11858-022-01334-w.

Q. Xia, T. K. F. Chiu, C. S. Chai, and K. Xie, “The mediating effects of needs satisfaction on the relationships between prior knowledge and self‐regulated learning through artificial intelligence chatbot,” Br. J. Educ. Technol., vol. 54, no. 4, pp. 967–986, Jul. 2023, doi: 10.1111/bjet.13305.

A. Álvarez-espinoza, “El concepto dialéctico de internalización en Vygotski: aproximaciones a un debate,” Psicol. Conoc. y Scoiedad, vol. 8, no. 1, pp. 5–35, 2018.

T. T. Wijaya, Q. Yu, Y. Cao, Y. He, and F. K. S. Leung, “Latent Profile Analysis of AI Literacy and Trust in Mathematics Teachers and Their Relations with AI Dependency and 21st-Century Skills,” Behav. Sci. (Basel)., vol. 14, no. 11, p. 1008, Oct. 2024, doi: 10.3390/bs14111008.

B. Hazan-Liran and P. Miller, “The Influence of Manipulating and Accentuating Task-Irrelevant Information on Learning Efficiency: Insights for Cognitive Load Theory,” J. Cogn., vol. 7, no. 1, p. 36, Apr. 2024, doi: 10.5334/joc.361.

B. Jose, J. Cherian, A. M. Verghis, S. M. Varghise, M. S., and S. Joseph, “The cognitive paradox of AI in education: between enhancement and erosion,” Front. Psychol., vol. 16, p. 1550621, Apr. 2025, doi: 10.3389/fpsyg.2025.1550621.

J. Tian and R. Zhang, “Outsourcing thinking to AI? Focused immersion, AI dependency, and the double-edged impact on critical thinking,” Humanit. Soc. Sci. Commun., vol. 13, no. 1, p. 1015, Apr. 2026, doi: 10.1057/s41599-026-07153-8.

Y. Fan et al., “Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance,” Br. J. Educ. Technol., vol. 56, no. 2, pp. 489–530, Mar. 2025, doi: 10.1111/bjet.13544.

C. Hou, G. Zhu, and V. Sudarshan, “The role of critical thinking on undergraduates’ reliance behaviours on generative AI in problem‐solving,” Br. J. Educ. Technol., vol. 56, no. 5, pp. 1919–1941, Sep. 2025, doi: 10.1111/bjet.13613.

G. R. Lau, W. Y. Low, L. Tay, Y. A. Guevarra, D. Gašević, and A. Hartanto, “Understanding critical thinking in generative artificial intelligence use: Development, validation, and correlates of the critical thinking in AI use scale,” Comput. Hum. Behav. Reports, vol. 22, p. 101103, May 2026, doi: 10.1016/j.chbr.2026.101103.

J. P. Leighton, Y. Cui, and M. Cutumisu, “Key Information Processes for Thinking Critically in Data-Rich Environments,” Front. Educ., vol. 6, p. 561847, Feb. 2021, doi: 10.3389/feduc.2021.561847.

O. Chen, F. Paas, and J. Sweller, “A Cognitive Load Theory Approach to Defining and Measuring Task Complexity Through Element Interactivity,” Educ. Psychol. Rev., vol. 35, no. 2, p. 63, Jun. 2023, doi: 10.1007/s10648-023-09782-w.

F. Paas and J. J. G. van Merriënboer, “Cognitive-Load Theory: Methods to Manage Working Memory Load in the Learning of Complex Tasks,” Curr. Dir. Psychol. Sci., vol. 29, no. 4, pp. 394–398, Aug. 2020, doi: 10.1177/0963721420922183.

J. Sweller, J. J. G. van Merriënboer, and F. Paas, “Cognitive Architecture and Instructional Design: 20 Years Later,” Educ. Psychol. Rev., vol. 31, no. 2, pp. 261–292, Jun. 2019, doi: 10.1007/s10648-019-09465-5.

J. Sweller, “The Development of Cognitive Load Theory: Replication Crises and Incorporation of Other Theories Can Lead to Theory Expansion,” Educ. Psychol. Rev., vol. 35, no. 4, p. 95, Dec. 2023, doi: 10.1007/s10648-023-09817-2.

J. Sweller, “Cognitive Load During Problem Solving : Effects on Learning,” Cogn. Sci., vol. 285, pp. 257–285, 1988, doi: 10.1207/s15516709cog1202.

X. Liu, Y. Xiao, and D. Li, “Assessing strategic use of artificial intelligence in self-regulated learning: Instrument development and evidence from Chinese university students,” Int. J. Educ. Technol. High. Educ., vol. 22, no. 1, p. 69, Nov. 2025, doi: 10.1186/s41239-025-00567-5.

X. Du, M. Du, Z. Zhou, and Y. Bai, “Facilitator or hindrance? The impact of AI on university students’ higher-order thinking skills in complex problem solving,” Int. J. Educ. Technol. High. Educ., vol. 22, no. 1, p. 39, Jul. 2025, doi: 10.1186/s41239-025-00534-0.

A. J. Martin et al., “Integrating generative AI and load reduction instruction to individualize and optimize students’ learning,” Learn. Individ. Differ., vol. 121, p. 102723, Jul. 2025, doi: 10.1016/j.lindif.2025.102723.

W. Pyke, J. Lunau, and A.-H. Javadi, “Does difficulty moderate learning? A comparative analysis of the desirable difficulties framework and cognitive load theory,” Q. J. Exp. Psychol., vol. 78, no. 10, pp. 2181–2195, Oct. 2025, doi: 10.1177/17470218241308143.

M. Lan and X. Zhou, “A qualitative systematic review on AI-empowered self-regulated learning in higher education,” npj Sci. Learn., vol. 10, no. 1, p. 21, May 2025, doi: 10.1038/s41539-025-00319-0.

S. Joo, I. Han, and I. Park, “The effect of metacognitive prompts using generative AI on cognitive load, task performance, and self-efficacy in online self-regulated learning,” Educ. Psychol., pp. 1–24, Jan. 2026, doi: 10.1080/01443410.2026.2618717.

X. Xu, L. Qiao, N. Cheng, H. Liu, and W. Zhao, “Enhancing self‐regulated learning and learning experience in generative AI environments: The critical role of metacognitive support,” Br. J. Educ. Technol., vol. 56, no. 5, pp. 1842–1863, Sep. 2025, doi: 10.1111/bjet.13599.

H. M. Nguyen and D. Goto, “Unmasking academic cheating behavior in the artificial intelligence era: Evidence from Vietnamese undergraduates,” Educ. Inf. Technol., vol. 29, pp. 15999–16025, Feb. 2024, doi: 10.1007/s10639-024-12495-4.

M. Jou and Y. Hao, “Enhancing visible learning in higher education through transparent and responsible AI: an empirical model based on cognitive load and creative self-beliefs,” J. Comput. High. Educ., pp. 1–24, Jun. 2026, doi: 10.1007/s12528-026-09496-2.