The Effectiveness of the Five-Tier Multiple Choice Diagnostic Test Instrument in Measuring Misconceptions and Science Literacy in The Acids Bases Topic in Senior High Schools

DOI:

https://doi.org/10.58421/misro.v5i2.1659

Authors

Keywords:

4D Model, Acid-Base Misconception, Diagnostic Assessment, Five-Tier Multiple Choice, Nearpod, Scientific Literacy

Abstract

Misconceptions in acid-base chemistry are prevalent and persistent, yet conventional assessments cannot distinguish them from a lack of knowledge or guessing, leaving teachers without actionable diagnostic data. This study aimed to develop, validate, and evaluate a five-tier multiple-choice digital diagnostic e-instrument (eFTMCDI) for acid-base chemistry in Indonesian secondary schools. Using the 4D development model, six experts validated a 30-item draft across two rounds; preliminary psychometric testing involved 30 students, and dissemination was conducted with 61 students across two schools with contrasting academic profiles. The final 23-item instrument demonstrated very high reliability (Cronbach's α = 0.850), 93.3% of items at moderate difficulty, and Very Valid expert ratings across all six dimensions, with practicality confirmed as Very Good by students (96.25%) and teachers (97.75%). Field testing revealed distinct conceptual profiles between schools: the regular-track school was dominated by No Understanding (56.5%). In comparison, the elite-track school exhibited a notably higher Misconception rate (24.8% vs. 13.6%) — a pattern detectable only through the dual confidence-rating architecture of Tiers 2 and 4. Scientific literacy scores differed significantly between schools (84.12 vs. 62.45; p < 0.001). The eFTMCDI may provide a potential tool to support evidence-based formative assessment by simultaneously profiling misconception typology and scientific literacy within a single digital session.

Downloads

Download data is not yet available.

References

R. W. Bybee, Achieving Scientific Literacy: From Purposes to Practices. Portsmouth, NH: Heinemann, 1997.

N. Amelia and R. Illah, "Penerapan asesmen formatif berbasis digital untuk meningkatkan literasi sains peserta didik SMA," J. Sci. Educ. Innov., vol. 9, no. 1, pp. 50–65, 2025. https://doi.org/10.24114/jsei.v9i1.31580

I. R. Suwarma, R. Nurani, and N. Syafitri, "Profil asesmen sains berbasis kurikulum merdeka di SMA Indonesia," J. Pendidikan Indones., vol. 14, no. 1, pp. 45–55, 2025. https://doi.org/10.23887/jpi-undiksha.v14i1.42106

OECD, PISA 2022 Assessment and Analytical Framework. Paris: OECD Publishing, 2023. https://doi.org/10.1787/dfe0bf9c-en

S. Anastasopoulou, I. Papadopoulos, and G. Stamelos, "Types of assessments in science education and their impact on students' critical thinking," Int. J. STEM Educ., vol. 12, no. 1, pp. 1–18, 2025. https://doi.org/10.1186/s40594-025-00468-2

M. Jacomuzzi, G. Brandao de Souza, and E. Maloney, "The role of assessment in advancing science education literacy," Int. J. STEM Educ., vol. 12, no. 2, pp. 30–45, 2025. https://doi.org/10.1186/s40594-025-00476-2

R. Duit, "Students' and teachers' conceptions and science education," Int. J. Sci. Educ., vol. 36, no. 7, pp. 1059–1078, 2014.

Kemendikbudristek, Panduan Implementasi Kurikulum Merdeka. Jakarta: Direktorat SMA, 2023.

H. D. Barke, A. Hazari, and S. Yitbarek, Misconceptions in Chemistry: Addressing Perceptions in Chemical Education. Berlin: Springer, 2008.

M. Üce and İ. Ceyhan, "Misconception in chemistry education and practices to eliminate them: Literature analysis," J. Educ. Train. Stud., vol. 7, no. 3, pp. 202–208, 2019. https://doi.org/10.11114/jets.v7i3.3990

S. Soeharto et al., "Students' misconception profiles in acid-base concepts: A diagnostic study," J. Pendidikan IPA Indones., vol. 10, no. 3, pp. 384–395, 2021.

F. D. Mubarokah, S. Mulyani, and N. Y. Indriyanti, "Identifying students' misconceptions of acid-base concepts using a three-tier diagnostic test," J. Turkish Sci. Educ., vol. 15, pp. 51–58, 2018.

F. T. M. Panggabean et al., "Analysis of 7th-grade students' misconceptions of acid-base," J-PEK (Jurnal Pembelajaran Kimia), vol. 8, no. 1, pp. 1–7, 2023.

I. S. Caleon and R. Subramaniam, "Do students know what they know and what they don't know? Using a four-tier diagnostic test," Res. Sci. Educ., vol. 40, no. 3, pp. 313–337, 2010.

D. F. Treagust, "Development and use of diagnostic tests to evaluate students' misconceptions in science," Int. J. Sci. Educ., vol. 10, no. 2, pp. 159–169, 1988. https://doi.org/10.1080/0950069880100204

L. Brandao de Souza and V. Jacomuzzi, "Multi-tier diagnostic assessment as a solution for conceptual misconception detection in chemistry," Chem. Educ. Res. Pract., vol. 26, no. 2, pp. 88–107, 2025.

M. H. A. Shiddiqi et al., "Systematic literature review: Analysis of misconception problems and diagnostic instruments for learning chemistry," J. Penelitian Pendidikan IPA, vol. 10, no. 4, pp. 168–179, 2024.

D. A. Rokhim, H. R. Widarti, and S. Sutrisno, "Five-tier instrument to identify students' misconceptions and representation: A systematic literature review," Orbital: Electron. J. Chem., pp. 202–207, 2023. http://dx.doi.org/10.17807/orbital.v15i4.17709

H. M. Dirman and F. Mufit, "Design and validity of five tier-multiple choice test E-instruments using I-Spring Quiz Maker," J. Penelitian Pendidikan IPA, vol. 8, no. 6, pp. 3170–3179, 2022. https://doi.org/10.29303/jppipa.v8i6.2300

T. Duran and M. Dikmenli, "Use of multi-tier diagnostic tests in science education: A systematic review," J. Educ. Sci. Environ. Health, vol. 10, no. 1, pp. 224–244, 2024. https://doi.org/10.55549/jeseh.755

A. A. Rupp and J. L. Templin, "Unique characteristics of diagnostic classification models," Measurement, vol. 6, no. 4, pp. 219–262, 2008. https://doi.org/10.1080/15366360802490866

M. Duran and M. Dikmenli, "The effectiveness of five-tier digital diagnostic test in identifying students' misconceptions," Int. J. Sci. Educ., vol. 46, no. 2, pp. 225–241, 2024.

K. K. Islamiyah, S. Rahayu, and I. W. Dasna, "The effectiveness of remediation learning strategy in reducing misconceptions on chemistry: A systematic review," Tadris: J. Keguruan Ilmu Tarbiyah, vol. 7, no. 1, pp. 63–77, 2022.

A. Saputra and H. Parbuntari, "Development of a five tier diagnostic test of misconceptions on chemical equilibrium material," BIOCHEPHY: J. Sci. Educ., vol. 5, no. 1, pp. 163–174, 2025. DOI: 10.52562/biochephy.v5i1.1466

M. Imaduddin et al., "Exploring the pre-service basic science teachers' misconceptions using the six-tier diagnostic test," Int. J. Eval. Res. Educ., vol. 12, p. 1627, 2023.

S. Zheng, M. Li, and X. Wang, "Integrating digital multi-tier assessments to promote science literacy," Int. J. Educ. Technol. Higher Educ., vol. 22, no. 1, pp. 1–19, 2025. https://doi.org/10.1186/s41239-025-00466-5

A. S. Adi, D. K. Sari, and M. Winarsih, "The effectiveness of digital multi-tier diagnostic test integrated with scientific literacy," J. Educ. Learn., vol. 13, no. 1, pp. 21–34, 2024.

UNESCO, The State of Online and Distance Learning in Southeast Asia: COVID-19 Impact and Response. Bangkok: UNESCO Bangkok, 2021.

T. Windani, "Real-time assessment as a tool to diagnose student misconceptions in chemistry education," Int. J. Educ. Technol., vol. 6, no. 2, pp. 199–210, 2024.

M. Abdullah, D. Suryadi, and M. Imaduddin, "Analisis miskonsepsi siswa dalam pembelajaran kimia menggunakan digital diagnostic test," J. Pendidikan IPA Indones., vol. 14, no. 1, pp. 110–120, 2025. https://doi.org/10.15294/jpii.v14i1.31205

D. Novita, S. Suyono, and S. Suyatno, "Analysis of student conceptions and conceptional changes about chemical equilibrium materials," IJORER, vol. 4, no. 6, pp. 782–794, 2023.

M. Yuberti et al., "Misconception and scientific literacy: How significant is the correlation?" Bull. Sci. Educ., vol. 3, no. 1, pp. 16–30, 2022. https://doi.org/10.29303/bse.v3i1.98

K. Schuessler et al., "Developing and evaluating an e-learning and e-assessment tool for organic chemistry in higher education," Front. Educ., vol. 9, p. 1355078, 2024.

Kemendikbud, Capaian Pembelajaran Kimia SMA Kurikulum Merdeka. Jakarta, 2022.

R. Johar and A. Rizeky, "Evaluasi efektivitas instrumen diagnostik berbasis digital dalam pembelajaran sains," J. Pendidikan Evaluasi Pendidikan, vol. 28, no. 1, pp. 55–72, 2024.

A. K. Nashoih and A. Laila, "Model pengembangan instrumen evaluasi berbasis R&D: Kajian metodologis dan implementasi," J. Penelitian Evaluasi Pendidikan, vol. 28, no. 2, pp. 88–104, 2024.

S. Thiagarajan, D. S. Semmel, and M. I. Semmel, Instructional Development for Training Teachers of Exceptional Children: A Sourcebook. Bloomington, IN: Indiana University, 1974.

Sugiyono, Metode Penelitian Pendidikan: Pendekatan Kuantitatif, Kualitatif, dan R&D. Bandung: Alfabeta, 2019.

E. P. Widoyoko, Evaluasi Program Pembelajaran. Yogyakarta: Pustaka Belajar, 2012.

Arikunto, Dasar-dasar Evaluasi Pendidikan, 8th ed. Jakarta: Rineka Cipta, 2016.

Downloads

Additional Files

Published

2026-06-23

How to Cite

[1]
R. Adisman, M. Erna, D. Futra, and S. Dwirianto, “The Effectiveness of the Five-Tier Multiple Choice Diagnostic Test Instrument in Measuring Misconceptions and Science Literacy in The Acids Bases Topic in Senior High Schools”, J.Math.Instr.Soc.Res.Opin., vol. 5, no. 2, pp. 1797–1820, Jun. 2026.

Issue

Vol. 5 No. 2 (2026): June

Section

Articles

License

Copyright (c) 2026 Journal of Mathematics Instruction, Social Research and Opinion

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.