Analisis Kualitas Instrumen Asesmen Sains Siswa Kelas V Sekolah Dasar dalam Konteks Deep Learning Menggunakan Model Rasch

Zikra; Dewi Juita; Putri Zuhra; Yusmaridi M; Festiyed; Skunda Diliarosta; Fatni Mufit

doi:10.31539/kskmqp27

Authors

Zikra Universitas PGRI Sumatera Barat danUniversitas Negeri Padang
Dewi Juita Universitas Islam Negeri Imam Bonjol dan Universitas Negeri Padang
Putri Zuhra Universitas Negeri Padang
Yusmaridi M Sekolah Dasar Negeri Percobaan Padang
Festiyed Universitas Negeri Padang
Skunda Diliarosta Universitas Negeri Padang
Fatni Mufit Universitas Negeri Padang

DOI:

https://doi.org/10.31539/kskmqp27

Abstract

This study aimed to evaluate the psychometric quality of a science assessment instrument designed for fifth-grade elementary school students in a deep learning instructional context using the Rasch model. This study used a descriptive quantitative approach with a one-parameter logistic Rasch model (1-PL) analyzed using Winsteps software version 5.10.4.0 applied to 20 dichotomous multiple-choice items administered to 29 fifth-grade elementary school students; the analysis covered item difficulty, construct validity, unidimensionality based on PCA of standardized residuals, person ability, person fit, and instrument reliability. The results showed strong reliability with a KR-20 coefficient of 0.89, person reliability of 0.84, and item reliability of 0.81; the raw variance explained by the measures reached 42.9% exceeding the 40% Rasch standard and all unexplained variance contrasts remained below 10% confirming that the instrument is unidimensional; 19 of the 20 items were valid while item A10 showed misfit (Outfit MNSQ = 2.45; ZSTD = 2.15); and the distribution of student ability indicated 20.69% in the high category, 62.07% moderate, and 17.24% low with six students exhibiting inconsistent response patterns. This study concludes that the science assessment instrument developed within the deep learning framework demonstrates adequate psychometric quality for measuring fifth-grade elementary school students' science competence.

Keywords: Deep Learning, Item Response Theory, Rasch Model, Science Assessment, Unidimensionality, Construct Validity.

References

Asikainen, H., & Gijbels, D. (2017). Do students develop towards more deep approaches to learning during studies? A systematic review on the development of students' deep and surface approaches to learning in higher education. Educational Psychology Review, 29(2), 205–234. https://doi.org/10.1007/s10648-017-9406-6

Bond, T. G., & Fox, C. M. (2015). Applying the Rasch model: Fundamental measurement in the human sciences (3rd ed.). Routledge.

Boone, W. J. (2016). Rasch analysis for instrument development: Why, when, and how? CBE—Life Sciences Education, 15(4), rm4. https://doi.org/10.1187/cbe.16-04-0148

Cook, D. A., & Beckman, T. J. (2006). Current concepts in validity and reliability for psychometric instruments: Theory and application. The American Journal of Medicine, 119(2), 166.e7–166.e16. https://doi.org/10.1016/j.amjmed.2005.10.036

Darmana, A., Pulungan, A. N., Nasution, H. A., Nst, M. A., & Faradilla, P. (2024). Test instrument validation with Rasch model. In Proceedings of the 5th International Conference on Innovation in Education, Science, and Culture (ICIESC 2023). European Alliance for Innovation. https://doi.org/10.4108/eai.24-10-2023.2342115

Fanani, A., & Kusmaharti, D. (2018). Pengembangan pembelajaran berbasis HOTS (higher order thinking skill) di sekolah dasar kelas V. Jurnal Pendidikan Dasar, 9(1), 1–11. https://doi.org/10.21009/JPD.081

Fullan, M., Quinn, J., & McEachen, J. (2018). Deep learning: Engage the world change the world. Corwin.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (2016). Fundamentals of item response theory. SAGE.

Hamzah, F. M., Rashid, M. N. A., Rahman, M. N. A., & Rasul, M. S. (2022). Evaluating the validity and reliability of authentic learning instruments using Rasch model. International Journal of Global Optimization and Its Application, 1(3), 182–189. https://doi.org/10.56225/ijgoia.v1i3.69

Isrokatun, I., Sunaengsih, C., Maulana, M., Syahid, A. A., Karlina, D. A., & Rohaeti, P. (2024). HOTS-based learning versus HOTS-based test instruments: A study analysis of elementary school teachers' and students' readiness. Mimbar Sekolah Dasar, 11(3), 595–612. https://doi.org/10.53400/mimbar-sd.v11i3.75404

Lau, C. K. H., & Liem, G. A. D. (2024). Assessment approaches and deep learning. Educational Psychology Review, 36(1), Article 10. https://doi.org/10.1007/s10648-024-09856-3

Linacre, J. M. (2023). Winsteps Rasch measurement computer program user's guide. Winsteps.com. https://www.winsteps.com

Misbach, I. H., & Sumintono, B. (2014). Pengembangan dan validasi instrumen "Persepsi siswa terhadap karakter moral guru" di Indonesia dengan model Rasch. Proceedings of the National Psychometrics Seminar.

Naira, H., Abdul Aziz, S. N., & Othman, M. R. (2024). Rasch analysis of chemistry assessment. Malaysian Journal of Innovative Education and Assessment Science (MYJIEAS), 4(1), 1–14.

Oliva, J. M., & Blanco-Lopez, A. (2023). Rasch analysis and validity of the construct understanding of the nature of models in Spanish-speaking students. European Journal of Science and Mathematics Education, 11(2), 344–359. https://doi.org/10.30935/scimath/12712

Postareff, L., Mattsson, M., Lindblom-Ylänne, S., & Hailikari, T. (2019). The complex relationship between emotions, approaches to learning, study success and study progression during the transition to university. Higher Education, 77(3), 441–457. https://doi.org/10.1007/s10734-018-0290-x

Purwaningsih, N., Utaminingsih, S., & Surachmi, S. (2022). Development of assessment instruments based on higher order thinking skills (HOTS) in thematic learning of grade IV elementary school students. Jurnal Prakarsa Paedagogia, 5(1), 165–175. https://doi.org/10.24176/jpp.v5i1.8674

Qi, H., Kim, M., Li, Y., Sandoval, C. L., Schurgers, C., Lubarda, M. V., Wagstaff, J. M., Nader, G., & Phan, A. (2023). From rote learning to deep learning: Filling the gap by enhancing engineering students' reasoning skills through explanatory learning activities. Journal of Engineering Education, 112(4), 1027–1048. https://doi.org/10.1002/jee.20535

Quinn, J., McEachen, J., Fullan, M., Gardner, M., & Drummy, M. (2019). Dive into deep learning: Tools for engagement. Corwin.

Reeve, B. B., Hays, R. D., Bjorner, J. B., Cook, K. F., Crane, P. K., Teresi, J. A., Thissen, D., Revicki, D. A., Weiss, D. J., Hambleton, R. K., Liu, H., Yu, L., Cella, D., & Waldron, T. K. (2007). Psychometric evaluation and calibration of health-related quality of life item banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5), S22–S31. https://doi.org/10.1097/01.mlr.0000250483.85507.04

Scoulas, J. M., Aksu Dunya, B., & De Groote, S. L. (2021). Validating students' library experience survey using Rasch model. Library and Information Science Research, 43(1), Article 101071. https://doi.org/10.1016/j.lisr.2021.101071

Smith, E. V. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3(2), 205–231.

Sumintono, B., & Widhiarso, W. (2015). Aplikasi pemodelan Rasch pada assessment pendidikan. Trim Komunikata.

Suryana, D., Putri, M. A., Supriatna, M., & Yudha, E. S. (2022). Analisis Rasch model: Validitas dan reliabilitas instrumen korban bullying. Hisbah: Jurnal Bimbingan Konseling dan Dakwah Islam, 19(2), 113–125.

Tesio, L., Simone, A., & Martinuzzi, G. (2023). Interpreting results from Rasch analysis 2: Advanced model applications and the data-model fit assessment. Disability and Rehabilitation, 45(3), 604–617. https://doi.org/10.1080/09638288.2023.2169772

Wibisono, S. (2014). Aplikasi model Rasch untuk validasi instrumen pengukuran fundamentalisme agama bagi responden Muslim. Jurnal Pengukuran Psikologi dan Pendidikan Indonesia, 3(3), 729–750.

Winje, Ø., & Londal, K. (2023). The why, what and how of deep learning: Critical analysis and additional concerns. Nordic Journal of Studies in Educational Policy, 9(2), 74–87. https://doi.org/10.1080/20004508.2023.2194502

Wolfe, E. W., & Smith, E. V. (2007). Instrument development tools and activities for measure validation using Rasch models: Part II—Validation activities. Journal of Applied Measurement, 8(2), 204–234.

Yudha, R. P. (2023). Higher order thinking skills (HOTS) test instrument: Validity and reliability analysis with the Rasch model. Eduma: Mathematics Education Learning and Teaching, 12(1), 21–38. https://doi.org/10.24235/eduma.v12i1.9468