I am a massive fan of Moodle Quiz. It really is one of the most powerful and comprehensive pedagogical tools in Moodle. From its multiple question types (http://docs.moodle.org/20/en/Question_types) to its many configuration and grading options, its various methods of giving students targeted feedback and its comprehensive reporting, it leaves most other quiz engines in the shade.
Reporting is one of Moodle Quizzes biggest strengths and an aspect of the quiz that I think people overlook. Today I want to take some timeout just to do some justice to Moodle quiz and share with the world just how fantastic it is.
Very generally there are two types of reports that can be generated for Moodle quizzes, student report and statistics report. Lets look at each of these reports in a little detail.
Student Report
The student report (grades) allows the teacher to see how students are performing in the quiz. The student report allows the teacher to see at a glance averages for questions and students but also allows the teacher to drill down to individual students to individual question and view a log for a student’s behaviour in a given question.
The screen shot below demonstrates a simple example of how summative information for a quiz is displayed.
Grade report for Moodle Quiz
For a teacher to review an individual learners attempt he or she just needs to click on one of the marks. This will bring up the question and outline the students response to the question including a log of what the student did while answering the question.
Question level logs
Teachers need to not only know how their students are performing but they also need to know how well their quizzes are performing. Moodle provides a “statistics” report providing basic psychometric analysis of quizzes to do this.
Statistics Report
The statistics report is broken into two parts; quiz information, which provides summative stats on the quiz, and quiz structure analysis, which provides detailed information about the quiz’s questions.
Quiz information contains the following information about a given quiz:
- Quiz name
- Course name
- Open and close dates for the quiz
- Total number of first/graded attempts
- Average grade for first/all attempts
- Median grade
- Standard deviation of grades
- Score distribution skewness (for first attempts) – indicating whether there is a long tail on the distribution curve to the left (negative skew) or right (positive skew)
- Coefficient of internal consistency (sometimes called Cronbach Alpha) – This is a measure of whether all the items in the quiz are testing basically the same thing. Thus it measures the consistency of the text, which is a lower bound for the validity. Higher numbers here are better [1].
- Error ratio – the variation in the grades comes from two sources. First some students are better than others at what is being tested, and second there is some random variation. We hope that the quiz grades will largely be determined by the student’s ability, and that random variation will be minimised. The error ratio estimates how much of the variation is random, and so lower is better [1].
- Standard error – this is derived from the error ratio, and is a measure of how much random variation there is in each test grade. So, if the Standard error is 10%, and a student scored 60%, then their real ability probably lies somewhere between 50% and 70% [1].
- Q# – shows the question number (position), question type icon, and preview and edit icons
- Question name – the name is also a link to the detailed analysis of this question (See Quiz Question Statistics below).
- Attempts – how many students attempted this question.
- Facility Index – the percentage of students that answered the question correctly.
- Standard Deviation – how much variation there was in the scores for this question.
- Random guess score – the score the student would get by guessing randomly
- Intended/Effective weight – Intended weight is simply what you set up when editing the quiz. If question 1 is worth 3 marks out of a total of 10 for the quiz, the the intended weight is 30%. The effective weight is an attempt to estimate, from the results, how much of the actual variation was due to this question. So, ideally the effective weights should be close to the intended weights.
- Discrimination index – this is the correlation between the score for this question and the score for the whole quiz. That is, for a good question, you hope that the students who score highly on this question are the same students who score highly on the whole quiz. Higher numbers are better.
- Discriminative efficiency – another measure that is similar to Discrimination index.
- Where random questions are used, there is one row in the table for the random question, followed by further rows, one for each real question that was selected in place of this random question.
- When quiz questions are randomized for each quiz, the quiz module determines a default position.
- Quiz statistics calculations gives further details on all these quantities.
References:
