Documentation

User Tools

Site Tools

Trace:
emis_exams_module

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
emis_exams_module [2022/08/18 04:02] – [Exam Items] ghacheyemis_exams_module [2022/08/22 04:04] (current) – [By benchmarks] ghachey
Line 37: Line 37:
 ==== Subject Areas, Standards and Benchmarks ==== ==== Subject Areas, Standards and Benchmarks ====
  
-Exams (aka. assessments) are typically given at various stages of a student's education life cycle (e.g. Grade 3, Grade 6, Grade 8, Grade 12, etc.) and on various subject areas (e.g. English, Local Language, Mathematics, Sciences). The subjects are then broken down further into standards which are broken down into benchmarks and finally broken down into indicators all of which students are assessed on. A simplified version of this hierarchical structure is depicted below in the Grade 3 Mathematics subject area.+Exams (aka. assessments) are typically given at various stages of a student's education life cycle (e.g. Grade 3, Grade 6, Grade 8, Grade 12, etc.) and on various subject areas (e.g. English, Local Language, Mathematics, Sciences). The subjects are then organized down further into standards which are organized down into benchmarks and finally organized down into indicators all of which students are assessed on. A simplified version of this hierarchical structure is depicted below in the Grade 3 Mathematics subject area.
  
   * Standard 3.1: Number Sense   * Standard 3.1: Number Sense
Line 54: Line 54:
 ==== Exam Items ==== ==== Exam Items ====
  
-To assess students a number of items (questions) make up an exam. This can be anywhere from 40 items to 100 items. There is no hard rule on the number of items but typically it will be a multiple of 4 since the results are usually compiled into 4 levels of achievement discussed in the next section. And since the analysis is possible all the way down to indicators it is usually required to have 4 items minimum to assess a particular indicator. So for the simplified example above the whole test could have 20 items (a small exam).+To assess students a number of items (questions) make up an exam. This can be anywhere from 40 items to 72 items, but it could be more or less. There is no hard rule on the number of items but typically it will be a multiple of 4 since the results are usually compiled into 4 levels of achievement discussed in the next section. And since the analysis is possible all the way down to indicators it is usually required to have 4 items minimum to assess a particular indicator. So for the simplified example above the whole test could have 20 items (a small exam).
  
   * Standard 3.1: Number Sense   * Standard 3.1: Number Sense
Line 88: Line 88:
  
 For simplicity, the levels of achievement will be referred to as simply Level 1, Level 2, Level 3 and Level 4 for the remaining of this document. For simplicity, the levels of achievement will be referred to as simply Level 1, Level 2, Level 3 and Level 4 for the remaining of this document.
-===== Candidate Count =====+===== Analysis Method - Candidate Count ===== 
 + 
 +The candidate count analysis can be done at various levels including: 
 + 
 +  * By individual candidate (student) 
 +  * By exam (whole test) 
 +  * By standards 
 +  * By benchmarks 
 +  * By indicators 
 + 
 +Regardless of the level at which the analysis is performed, the methodology is the same. You take the total number of items and you divide by 4 to get your "cut-offs". For example, if there are 4 items (e.g. analysis for an indicator) you have the following: 
 + 
 +  * 0 or 1 items correct out of 4 => Level 1 
 +  * 2 items correct out of 4 => Level 2 
 +  * 3 items correct out of 4 => Level 3 
 +  * 4 items correct out of 4 => Level 4 
 + 
 +If you have 40 items (e.g. analysis for a whole test) you have the following: 
 + 
 +  * 0 to 10 items correct out of 40 => Level 1 
 +  * 11 to 20 items correct out of 40 => Level 2 
 +  * 21 to 30 items correct out of 40 => Level 3 
 +  * 31 to 40 items correct out of 40 => Level 4 
 + 
 +<note important>The consistency with the cut-off is actually very important. You should not have different cut-offs when producing results for individual students then with national results as they do not compare or mean the same thing at all.</note> 
 + 
 +Let's review the analysis at different levels with examples. 
 + 
 +==== By individual candidate (student) ==== 
 + 
 +To get the level of achievement of an individual you look at all the correct items for the whole test and see where the student falls within the cut-offs. This test has 20 items and thus the cut-offs are as follows: 
 + 
 +  * 0 to 5 items correct out of 20 => Level 1 
 +  * 6 to 10 items correct out of 20 => Level 2 
 +  * 11 to 15 items correct out of 20 => Level 3 
 +  * 16 to 20 items correct out of 20 => Level 4 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-candidates.png?nolink |}} 
 + 
 +==== By exam ==== 
 + 
 +From our original example we have the exam for Math Grade 3. This exam as 20 items. Therefore the cut-offs are as below: 
 + 
 +    * 0 to 5 items correct out of 20 => Level 1 
 +    * 6 to 10 items correct out of 20 => Level 2 
 +    * 11 to 15 items correct out of 20 => Level 3 
 +    * 16 to 20 items correct out of 20 => Level 4 
 + 
 +What follows is a illustration of how this analysis would be produced. The items for the whole exams are highlighted in blue (all exam items). 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-exam.png?nolink |}} 
 + 
 +And the resulting chart analysis. 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-exam-chart.png?nolink |}} 
 + 
 +==== By standards ==== 
 + 
 +From our original example we have two standards. And based on the standard's indicators and their items on the exams we know how many items for each of our standards. 
 + 
 +  * Standard 3.1: Number Sense [12 ITEMS] with the cut-offs as: 
 +    * 0 to 3 items correct out of 12 => Level 1 
 +    * 4 to 6 items correct out of 12 => Level 2 
 +    * 7 to 9 items correct out of 12 => Level 3 
 +    * 10 to 12 items correct out of 12 => Level 4 
 +  * Standard 3.2: Geometry and measurement concepts. [8 ITEMS] with the cut-offs as: 
 +    * 0 to 2 items correct out of 8 => Level 1 
 +    * 3 to 4 items correct out of 8 => Level 2 
 +    * 5 to 6 items correct out of 8 => Level 3 
 +    * 7 to 8 items correct out of 8 => Level 4 
 + 
 +What follows is a illustration of how this analysis would be produced. The items for the two different standards are highlighted in their own respective colors. 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-standards.png?nolink |}} 
 + 
 +And the resulting chart analysis. 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-standards-chart.png?nolink |}} 
 + 
 + 
 + 
 + 
 +==== By benchmarks ==== 
 + 
 +From our original example we have three benchmarks. And based on the benchmarks' indicators and their items on the exam we know how many items for each of our benchmark. 
 + 
 +  * Benchmark 3.1.1: Use place value understanding and properties of operations to perform … [12 ITEMS] with the cut-offs as: 
 +    * 0 to 3 items correct out of 12 ⇒ Level 1 
 +    * 4 to 6 items correct out of 12 ⇒ Level 2 
 +    * 7 to 9 items correct out of 12 ⇒ Level 3 
 +    * 10 to 12 items correct out of 12 ⇒ Level 4 
 +  * Benchmark 3.2.1 Solve problems of time and temperature. apply knowledge to real world problems. [4 ITEMS] with the cut-offs as: 
 +    * 0 to 1 items correct out of 4 ⇒ Level 1 
 +    * 2 items correct out of 4 ⇒ Level 2 
 +    * 3 items correct out of 4 ⇒ Level 3 
 +    * 4 items correct out of 4 ⇒ Level 4 
 +  * Benchmark 3.2.2 Find the area and perimeter of figures. [4 ITEMS] with the cut-offs as: 
 +    * 0 to 1 items correct out of 4 ⇒ Level 1 
 +    * 2 items correct out of 4 ⇒ Level 2 
 +    * 3 items correct out of 4 ⇒ Level 3 
 +    * 4 items correct out of 4 ⇒ Level 4 
 + 
 +What follows is a illustration of how this analysis would be produced. The items for the three different benchmarks are highlighted in their own respective colors.  
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-benchmarks.png?nolink |}} 
 + 
 +And the resulting chart. 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-benchmarks-chart.png?nolink |}} 
 + 
 + 
 +==== By indicators ==== 
 + 
 +From our original example we have 5 indicators. And we know that each indicators have 4 items each on the exams. 
 + 
 +  * Indicator 3.1.1.1: Use base-ten blocks to count, read and write numbers to ...[4 ITEMS] with the cut-offs as: 
 +    * 0 to 1 items correct out of 4 ⇒ Level 1 
 +    * 2 items correct out of 4 ⇒ Level 2 
 +    * 3 items correct out of 4 ⇒ Level 3 
 +    * 4 items correct out of 4 ⇒ Level 4 
 +  * Indicator 3.1.1.2: Understand and use properties of multiplication (e.g. commutative property ...[4 ITEMS] with same cut-offs as above 
 +  * Indicator 3.1.1.3: Divide with tables of 6, 7, 8, and 9 using models ...[4 ITEMS] with same cut-offs as above 
 +  * Indicator 3.2.1.1: Tell time to the minute. Read time on a digital clock...[4 ITEMS] with same cut-offs as above 
 +  * Indicator 3.2.2.1: Understand the meaning of area. Use square units to find[4 ITEMS] with same cut-offs as above 
 + 
 +What follows is an illustration of how this analysis would be produced. The items for the five different indicators are highlighted in their own respective colors.  
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-indicators.png?nolink |}} 
 + 
 +And the resulting chart. 
 + 
 +{{ :exams-module:exams-module-items-candidatecount-by-indicators-chart.png?nolink |}} 
 + 
 + 
 + 
 + 
 +===== Analysis Method - Indicator Count ===== 
 + 
 +Candidate count is the simpler more natural way to analyze the results. However, it is not the only way. The indicator count traditionally used in SOE Assessment does it differently. Instead of counting students performing at each levels of achievement for benchmarks, standards and whole exams this methods count the indicators performing at the levels of achievement for benchmarks, standards and whole exams. 
 + 
 +<note important>While both the candidate count and indicator count end up with similar results they are not the same analysis!</note> 
 + 
 +How exactly does this work? First, the analysis on indicators must be done and it is done just like in the candidate count method shown above. That is the starting point and everything else is based on the indicators results. 
 + 
 +==== By exam (whole test) ==== 
 + 
 +Since every single indicator is part of the exam. For the analysis of the whole we count all the Level 1, Level 2, Level 3 and Level 4 for all indicators. Totalling them up should equal the number of student (i.e. i our example  10) multiplied by the number of indicators (i.e. in our example 5), therefore 50. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-exam.png?nolink |}} 
 + 
 +This then translates into the following analysis. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-exam-chart.png?nolink |}} 
 + 
 +==== By standards ==== 
 + 
 +For standards, we need to group the indicators into their respective standards as shown in the following figure. And we do a count on those. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-standards.png?nolink |}} 
 + 
 +This will then translate into the following. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-standards-chart.png?nolink |}} 
 + 
 +==== By benchmarks ==== 
 + 
 +Similarly to standards, for benchmarks we also group the indicators into their respective benchmarks and count from there. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-benchmarks.png?nolink |}} 
 + 
 +Resulting in the following analysis. 
 + 
 +{{ :exams-module:exams-module-items-indicatorcount-by-benchmarks-charts.png?nolink |}} 
  
  
  
-===== Indicator Count ===== 
  
emis_exams_module.1660795324.txt.gz · Last modified: 2022/08/18 04:02 by ghachey