Objectives: The ExaMe system for the evaluation of targeted knowledge has been in development since 1998. The new features of the ExaMe system are introduced in this paper. Especially, the new three-layer architecture is described. Besides the system itself, the properties of fixed tests in the ExaMe system are studied. In special detail, the reliability of the fixed tests is discussed. The theory background is explained and some limitations of the reliability are pointed out.
Methods for estimation of the reliability: Three characteristics used for estimation of reliability of educational tests are discussed: Cronbach's alpha, standardized item alpha and split half coefficient. The relation between these characteristics and reliability and between characteristics themselves is investigated. In more detail, the properties of Cronbach's alpha, the characteristics mostly used for the estimation of reliability, are discussed. A confidence interval is introduced for the characteristics.
Results and conclusions: Since 2000, the serviceability of the ExaMe evaluation system as the supporting evaluation tool has been repeatedly shown at the courses of Ph.D. studies in biomedical informatics at Charles University in Prague. The ExaMe system also opens new possibilities for self-evaluation and distance learning, especially when connected with electronic books on the Internet. The estimation of reliability of tests contains some limitations. Keeping them in mind, we can still get some information about the quality of certain educational tests. Therefore, the estimation of reliability of the fixed tests is implemented in the ExaMe system.