Document Type : Original Research Paper


1 Advanced Technology Laboratory in Electronic Education, Amirkabir University of Technology,Tehran,Iran

2 Faculty of Computer Engineering and Information Technology, Amirkabir University of Technology,Tehran,Iran


One of the important issues in a learning management system is the assessment of the learner’s knowledge level. In basic methods of assessment, questions are simply ticked by the learners, and then automatically scored by machine. In this sort of examinations, two problems may be raised. Firstly, the learner may answer and mark the questions accidentally by guessing the answer without having enough knowledge about the related subject. Secondly, he/she may slip in answering the questions due to his/her inattention, although he/she has enough knowledge about the related subject. In this work, an improved method for knowledge level assessment is presented. In the proposed method learning concepts are modeled based on the hierarchical construct of learning objectives, and the learner’s knowledge model is used to estimate his/her knowledge level. This model is established on the basis of Bayesian networks, and considering the hierarchical construct of learning objectives. To evaluate the proposed method, a set of questions by considering the learning objectives was designed and ordered into different levels of complications. Then these questions were used for assessments. In this work also a new algorithm for updating the nodes in a Bayesian network is introduced. By means of this algorithm the effect of guessing and slipping answers in an assessment is undoubtedly reduced. Finally, by presenting a model for knowledge and use of the hierarchical construct of learning objectives, an effective solution for three above mentioned problems of guessing answers, slipping answers, and the origin of wrong answers have been established.


Main Subjects

[1] Brusilovsky P. and Sosnovsky S., Individualized exercises for self-assessment of programming knowledge: An evaluation of quizpack, ACM Journal of Educational Resources in Computing, Vol. 5, No. 6, 2005. [2] VanLehn K. and Martin J., Evaluation of an assessment system based on Bayesian student modeling, International Journal of Artificial Intelligence and Education, Vol. 8,No 2, 1997, pp. 179-221. [3] Martin J. and Vanlehn K., Student assessment using Bayesian nets, International Journal of Human-Computer Studies, Vol. 42,No. 6, 1995, pp. 575–591. [4] Conati C. and Vanlehn K., POLA: A student modeling framework for probabilistic on-line assessment of problem solving performance, In Proceedings of UM-96, 5th International Conference on User Modeling, User Modeling, Inc., 1996, pp. 75-82. [5] VanLehn K. and Martin J., Evaluation of an assessment system based on Bayesian student modeling, International Journal of Artificial Intelligence and Education,Vol. 8, No. 2, 1997, 179-221. [6] Mayo M. and Mitrovic A., Optimizing ITS behavior with Bayesian networks and decision theory, International Journal of Artificial Intelligence in Education, Vol. 12, 2001, pp. 124–153. [7] Conati C., Gertner A. and Vanlehn K., Using bayesian networks to manage uncertainty in student modeling, User Modeling and UserAdapted Interaction, Vol. 12,No. 4, 2002, pp.371-417. [8] Desmarais M.C. and Pu X., A bayesian inference adaptive testing framework and its comparison with item response theory, International Journal of Artificial Intelligence in Education, Vol. 15, 2005, pp. 291-323. [9] Vanlehn K., Lynch C., Schulze K., Shapiro J. A., Shelby R., Taylor L., Treacy D., Weinstein A., and Wintersgill M., The andes physics tutoring system: Lessons learned, International Journal of Artificial Intelligence in Education, Vol. 15,No. 3, 2005. [10] Bekele R. and Menzel W., A Bayesian Approach to Predict Performance of a Student (BAPPS): A case with ethiopian students, In Proceedings of International Conference of Artificial Intelligence and Applications, AIA2005, Vienna, Austria, 2005. [11] Pardos Z.A., Heffernan N.T., Anderson B. and Heffernan C., Using fine-grained skill models to fit student performance with Bayesian networks, On-line Proceedings of the Workshop on Educational Data Mining at the Eighth International Conference on Intelligent Tutoring Systems, Taiwan, 2006, pp.5-12. [12] Arroyo I, Beal C., Murray T., Walles R. and Woolf B. Wayang outpost: Intelligent tutoring for high stakes achievement tests, In Proceedings of the 7th International Conference on Intelligent Tutoring Systems (ITS2004), Springer Berlin / Heidelberg, 2004, pp. 468-477. ارزشیابی تطبیقپذیر و تخمین سطح دانش با استفاده از مدل دانش و به کمک شبکه بیزی احمد کاردان و همکار 959 نشریه علمی پژوهشی فناوری آموزش، سال سوم، جلد 3 ،شماره 4 ،تابستان 8311 [13] Razzaq L. and Heffernan N.T., Towards designing a user-adaptive web-based elearning system, In CHI '08 Extended Abstracts on Human Factors in Computing Systems, Florence, Italy, April, 2008,pp. 3525-3530. [14] Kardan A. and Kardan S., Learning object tendency: A new Concept for adaptive learning improvement, In Proceedings of 3rd International Conference of Virtual Learning (ICVL 2008), Constanta, Romania, 2008, pp. 237-248. [15] Collins J., Greer J., Huang S., Adaptive assessment using granularity hierarchies and Bayesian nets, In: Frasson C., Gauthier G., Lesgold A.(eds) Intelligent tutoring systems, Third International Conference, ITS’96 ,June, Montreal, Canada, 1996, pp. 569-577. [16] Molenda M., Historical and philosophical foundations of instructional design: A north american view, In: R. D. Tennyson, F. Schott, N. Seel and S. Dijkstra, (Eds.), Instructional design: International Perspective: Theory, research, and models. Vol. 1, pp. 41-54. Mahwah, NJ: Erlbaum. [17] Salaberry M.R., Pedagogical design of computer mediated communication tasks: Learning objectives and technological capabilities, Modern Language Journal, Vol. 84, part 1, 2000, pp. 28-37. [18] Boyle T., Design principles for authoring dynamic, reusable learning objects, Australian Journal of Educational Technology, Vol. 19, part 1, 2003, pp. 46-58. [19] Gagne R.M., Wager W.W., Golas K.C., Keller J.M., Principles of instructional design, 5th edition, Wadsworth Publishing, 2004. [20] Mitchell T.M., Machine learning, McGrawHill, 1997, pp. 154-184. [21] Birenbaum M., Toward adaptive assessment - the student's angle, Studies in Educational Evaluation, Vol. 20, No.2, 1994, pp. 239-255. [22] Dowling C.E., Hockemeyer C., Ludwig A.H., Adaptive assessment and training using the neighborhood of knowledge states, Lecture Notes in Computer Science, Issue 1086, 1996, pp. 578-586. [23] Challis D., Committing to quality learning through adaptive online assessment, Assessment and Evaluation in Higher Education, Vol. 30,No. 5, 2005, pp. 519-528 [24] Hatzilygeroudis I., Koutsojannis C., Papavlasopoulos C., Prentzas J., Knowledgebased adaptive assessment in a web-based intelligent educational system, In Advanced Learning Technologies 2006, Sixth International Conference on, IEEE, 2006, pp. 651 – 655. [25] Sitthisak O., Gilbert L. and Davis H.C., Towards a competency model for adaptive assessment to support lifelong learning, In: TENCompetence Workshop on Service Oriented Approaches and Lifelong Competence Development Infrastructures, January, Manchester, UK, 2007.