The Analysis Needs Mapping for the Development of an Artificial Intelligence-Item Response Theory (AI-IRT)-Based CAT Web System for Adaptive Science Assessment

Muhammad Gibran Alif Prasetya; Arif Widiyatmoko; Sudarmin Sudarmin; Novi Ratna Dewi

doi:10.58421/misro.v5i1.1256

DOI:

https://doi.org/10.58421/misro.v5i1.1256

Authors

Muhammad Gibran Alif Prasetya Universitas Negeri Semarang
Arif Widiyatmoko Universitas Negeri Semarang
Sudarmin Sudarmin Universitas Negeri Semarang
Novi Ratna Dewi Universitas Negeri Semarang https://orcid.org/0000-0002-0210-6496

Keywords:

Artificial Intelligence, Computerized Adaptive Test, Instructional Design, Item Response Theory, Science Assessment

Abstract

Assessment of science learning outcomes in junior high school science education is still largely dominated by conventional tests that do not adequately accommodate differences in students’ abilities and provide limited diagnostic feedback. This limitation highlights the need for a more adaptive and data-driven assessment system. This study aims to map the needs for developing a web-based Computerized Adaptive Test (CAT) system integrating Artificial Intelligence and Item Response Theory (AI-IRT) for science assessment at the junior high school level. The study employed a descriptive qualitative approach at the Analysis stage of the ADDIE development model. Data were collected through semi-structured interviews with one junior high school science teacher and five students, and analyzed using NVivo through open coding, axial coding, and selective coding. The findings reveal six key needs: adaptive assessment systems, measurement fairness and precision, diagnostic feedback, efficiency and automation, infrastructure readiness, and user-friendly interface design. These findings demonstrate the potential of integrating AI and IRT to support more accurate, personalized, and efficient science assessment. The results provide a conceptual and operational foundation for developing an AI-IRT-based CAT web system tailored to junior high school science learning.

Downloads

Download data is not yet available.

References

S. Rifky, “Dampak Penggunaan Artificial Intelligence Bagi Pendidikan Tinggi,” ijmst, vol. 2, no. 1, pp. 37–42, Feb. 2024, doi: 10.31004/ijmst.v2i1.287.

I. Arifah, “Pendidikan yang Didukung AI untuk Masa Depan Berkelanjutan: Mengintegrasikan Teknologi untuk Mencapai Sustainable Development Goals 2030,” Seminalu, vol. 1, no. 1, pp. 47–55, 2023, [Online]. Available: https://prosiding.unipar.ac.id/index.php/seminalu/article/view/14

A. Ihichr, O. Oustous, Y. E. B. E. Idrissi, and A. A. Lahcen, “A Systematic Review on Assessment in Adaptive Learning: Theories, Algorithms and Techniques,” ijacsa, vol. 15, no. 7, 2024, doi: 10.14569/IJACSA.2024.0150785.

M. A. Adeoye, E. F. Prastikawati, and R. Riwayatiningsih, “Beyond Traditional Testing: Exploring the Efficacy of Adaptive Assessments in Shaping Future Learners,” jone, vol. 11, no. 1, pp. 153–169, Feb. 2025, doi: 10.15294/jone.v11i1.19934.

Z. Muarifin, “Lunturnya Moralitas Pendidikan Di Era Artificial Intelligence,” creativity, vol. 2, no. 2, pp. 221–234, Nov. 2024, doi: 10.62288/creativity.v2i2.25.

L. Chen, P. Chen, and Z. Lin, “Artificial Intelligence in Education: A Review,” IEEE Access, vol. 8, pp. 75264–75278, 2020, doi: 10.1109/ACCESS.2020.2988510.

Q. Xia, X. Weng, F. Ouyang, T. J. Lin, and T. K. F. Chiu, “A scoping review on how generative artificial intelligence transforms assessment in higher education,” Int J Educ Technol High Educ, vol. 21, no. 1, p. 40, May 2024, doi: 10.1186/s41239-024-00468-z.

A. Huda, F. Firdaus, D. Irfan, Y. Hendriyani, A. Almasri, and M. Sukmawati, “Optimizing Educational Assessment: The Practicality of Computer Adaptive Testing (CAT) with an Item Response Theory (IRT) Approach,” JOIV : Int. J. Inform. Visualization, vol. 8, no. 1, p. 473, Mar. 2024, doi: 10.62527/joiv.8.1.2217.

S. A. Khosibah, A. Rahmaningrum, and C. T. Kusumawardani, “Potensi dan Praktik Literasi Artificial Intelligence (AI) dalam Pendidikan Anak Usia Dini di Indonesia: Systematic Literature Review,” JEA, vol. 11, no. 1, pp. 55–69, Jul. 2025, doi: 10.18592/jea.v11i1.16329.

F. R. Sanggulu, A. H. Panai, and I. N. A. Pascasarjana, “Transforming Elementary Science Education: Developing Ai-Based Interactive Learning Media (Aripas) For Enhanced Engagement And Comprehension,” Int. J. Res. Publ. Rev., vol. 6, no. 1, pp. 2474–2477, Jan. 2025, doi: 10.55248/gengpi.6.0125.0439.

F. Azfar and S. Sutiah, “Pengembangan Media Pembelajaran Video Avatar Berbasis Artificial Intellegence (AI) bagi Peserta Didik Tingkat Pendidikan Dasar,” JI, vol. 6, no. 4, pp. 1497–1509, Oct. 2024, doi: 10.36088/islamika.v6i4.5193.

F. Naseer, M. N. Khan, M. Tahir, A. Addas, and S. M. H. Aejaz, “Integrating deep learning techniques for personalized learning pathways in higher education,” Heliyon, vol. 10, no. 11, p. e32628, Jun. 2024, doi: 10.1016/j.heliyon.2024.e32628.

M. Abrar, W. Aboraya, R. A. Khaliq, K. P. Subramanian, Y. A. Husaini, and M. A. Husaini, “AI-Powered Learning Pathways: Personalized Learning and Dynamic Assessments,” ijacsa, vol. 16, no. 1, 2025, doi: 10.14569/IJACSA.2025.0160145.

R. Nurhayati, T. Nur, S. P, N. Adillah, Agustina, and M. Urva, “Dinamika Pembelajaran Pendidikan Agama Islam Berbasis Artificial Intelligence (AI),” PROSIDING SENTIKJAR, vol. 3, pp. 1–7, Oct. 2024, doi: 10.47435/sentikjar.v3i0.3131.

J. Afrita, “Peran Artificial Intelligence dalam Meningkatkan Efisiensi dan Efektifitas Sistem Pendidikan,” comserva, vol. 2, no. 12, pp. 3181–3187, Apr. 2023, doi: 10.59141/comserva.v2i12.731.

N. W. Asbara et al., “Penerapan Ai Sebagai Alat Bantu Proses Pembelajaran Di Tingkat Pendidikan Sekolah Dasar,” JMM, vol. 8, no. 1, p. 831, Feb. 2024, doi: 10.31764/jmm.v8i1.20083.

Y. C. Cahyaningrum, “Penerapan Artificial Intelligence Menggunakan Fuzzy Logic dalam Dunia Pendidikan,” j. amp. : j. ilm. bid. tek. elect. and comp., vol. 13, no. 2, pp. 62–68, Nov. 2023, doi: 10.33369/jamplifier.v13i2.30757.

A. Kurniawati, P. R. Ningsih, and A. D. Rahmawati, “Innovative AI-Based Digital Learning through E-Books and Videos: A Strategy to Support the Merdeka Curriculum in Primary Schools,” J.Act.R.Edu, vol. 3, no. 1, pp. 1–11, Jul. 2025, doi: 10.52620/jare.v3i1.179.

V. J. Owan, K. B. Abang, D. O. Idika, E. O. Etta, and B. A. Bassey, “Exploring the potential of artificial intelligence tools in educational measurement and assessment,” EURASIA J Math Sci Tech Ed, vol. 19, no. 8, p. em2307, Aug. 2023, doi: 10.29333/ejmste/13428.

R. D. A. Budiman, “Effectiveness of AI-Driven Assessments in Enhancing Learning Evaluation through Predictive Technology in Vocational Secondary School,” IJIET, vol. 15, no. 7, pp. 1410–1417, 2025, doi: 10.18178/ijiet.2025.15.7.2342.

M. Yahya, Wahyudi, and Akmal Hidayat, “Implementasi Artificial Intelligence (AI) di Bidang Pendidikan Kejuruan Pada Era Revolusi Industri 4.0,” semnasdies, vol. 1, pp. 190–199, Jul. 2023, doi: 10.59562/semnasdies.v1i1.794.

Ameliyana and D. F. Wardhani, “Pengembangan Sistem AI untuk Pembelajaran yang Dipersonalisasi: Inovasi dalam Pendidikan Modern,” Inovasi Pembangunan : Jurnal Kelitbangan, vol. 13, no. 1, 2025, doi: https://doi.org/10.35450/jip.v13i1.931.

M. Yanto, Mad Sa’I, and Nailatur Rizqiyah, “Personalisasi Pendidikan Berbasis AI dalam Meningkatkan Kualitas Belajar Siswa,” ent jpips, pp. 507–522, May 2025, doi: 10.19105/ejpis.v1i.19116.

Y. A. Nugroho, A. Widodo, E. T. Pebrina, J. Iskandar, and M. Nadeak, “Digitalization in Higher Education: How Information Systems Improve Operational and Strategic Performance,” J.IJOMER, vol. 2, no. 01, pp. 90–98, Jun. 2025, doi: 10.70508/dyrdm592.

S. U. Khoiriah, Lia Karunia Lam Uli Lubis, and Diva Kayla Nazwa Anas, “Analisis Perkembangan Sistem Manajemen Pendidikan di Era Society 5.0,” jispendiora, vol. 2, no. 2, pp. 117–132, Jun. 2023, doi: 10.56910/jispendiora.v2i2.650.

H. Alzghaibi, A. H. Alharbi, Y. H. Mughal, M. H. Alwheeb, and A. S. Alhlayl, “Assessing primary health care readiness for large-scale electronic health record system implementation: Project team perspective,” Health Informatics J, vol. 29, no. 1, p. 14604582231152790, Jan. 2023, doi: 10.1177/14604582231152790.

C. Syawaludin, “Pemanfaatan Artificial Intelligence dalam Pengembangan Strategi Pembelajaran di Lingkungan Pendidikan Dasar,” RIGGS, vol. 4, no. 4, pp. 451–457, Nov. 2025, doi: 10.31004/riggs.v4i4.3411.