Differences

This shows you the differences between two versions of the page.

--- user_manual [2017/10/19 04:38] – [Excel Workbook Submission] andrewerbs
+++ user_manual [2020/03/23 08:27] (current) – removed ghachey
@@ Line 1: / Line 1: @@
-====== Pacific EMIS User Guide ======
-FIXME Develop a generic simple Pacific EMIS User Guide for the online
-application that can be used by any country adopting the Pacific
-EMIS. This user guide should be simple; only including description of
-how to use the system and essentially nothing else. James wrotes some
-bits of it embedded in some early reports though the application's UI
-changed and will continue to change as the product improves over
-time. What can be taken from James work will and much will have to be
-rewritten.
-===== Pacific EMIS Overview =====
-**Introduction**
-The Pacific EMIS is a software platform developed to collect school data and produce national education indicators.
-**Census and Digest.** To report against international and national indicators, you must understand some key definitions, and relate these to the structure of the education system in your own country. Then, you build these definitions into your Pineapples EMIS by setting up system parameters and lookup lists.
-FIXME
-==== Brief History ====
-The Pacific EMIS software platform has been built upon work undertaken by UNIQUEST under NZAID funding to create a tool for managing data for the education system and report on the 'Education for All' Goals. Since then, the system has grown to be used throughout the Pacific region, being adapted to the unique requirements of each Pacific island country.
-FIXME
-==== Accessing and Login ====
-FIXME
-==== Registering New Users ====
-FIXME
-The Pacific EMIS supports two types of authentication:
-  * Basic User Authentication
-  * Active Directory User Authentication
-Both method work but the user interfaces to easily manage this has yet to be developed. For now, the technical team handles the user creation and assignment of appropriate access control (i.e. permissions).
-=== Basic User Authentication ===
-For example, creating a new basic user can be done currently by visiting the URL /register as shown in the illustration below. This would create a user with all permissions granted.
-{{ :user-manual:register-new-user.png }}
-=== Active Directory User Authentication ===
-==== Overview of Features (Modules) ====
-FIXME
-===== About Indicators =====
-FIXME
-This section explains what an indicator is and how to create a good one.\\
-**Indicators**\\
-FedEMIS holds a great deal of data about for example, schools, teachers and students. We can use all this data to develop indicators that can be compared over time. The most common and meaningful indicators are ratios of two independent pieces of data. This removes any effect of scale, and gives a measure for which targets can be set.
-For example, the question “How many teachers does the education system need?” has no general answer suitable for all time – as the population grows, more teachers will be needed. However, a meaningful target can be specified by using student/teacher ratio – the number of students in the system for each teacher. This ratio can be compared over time to track whether the supply of teachers is improving or deteriorating in relation to the target. In the case of student/teacher ratios, the two independent values being used are enrolment (students) and teacher numbers.
-Through FedEMIS all this data is used to develop the following indicators that can be compared over time to answer the following questions:
-\\
-//**What Makes a Good Indicator?**//
-An indicator is a single number, calculated from data in the database, that gives us meaningful information about some aspect of the education system.
-A good indicator is one that:
-  *	is conceptually easy to understand;
-  *	is easy to calculate from available data;
-  *	is clearly related to the area we wish to monitor;
-  *	can be meaningfully compared across time;
-  *	can be meaningfully compared between nations.\\
-//**Easy To Understand**//
-Many good indicators are easy to understand or visualise. Some, such as Pupil/Classroom Ratio and Pupil/Teacher Ratio, you may be able to visualise in your “mind’s eye”.
-That said, some important indicators of efficiency are more abstract in their definitions. Indicators such as the Coefficient of Efficiency do not have any obvious visual interpretation. However, the power of these indicators is to “distill” the effect of many pieces of data into a single value. As a consequence these indicators are more abstract.\\
-//**Available Data**//\\
-We must be able to calculate the indicator from available, verifiable data. When selecting between potential indicators, we should prefer those that
-          * use data that is simpler to collect;
-  * 	require less input data items\\
-over those that require more data or less accessible data.\\
-//**Clear Relation**//\\
-The relationship between the indicator and the aspect of the education system we are monitoring must be clear. The indicator must respond in
-	direction – the indicator value gets better when the situation improves, and get worse when the situation deteriorates.
-	magnitude – bigger changes in the indicator represent bigger gains or losses in the situation.\\
-//**Proxies**//\\
-Sometimes, it may be difficult to collect data to explicitly measure the aspect of the system we are monitoring. In such cases, we may prefer to use another indicator that is easier to calculate, even though that indicator is not apparently related to the item we wish to measure. Such a “replacement” indicator is known as a proxy.
-For example, the Millennium Development Goals for Education wish to measure the rate of Primary Completion. This rate is not possible to calculate using only enrolment numbers. Pupils enrolled in the last year of primary education will not necessarily successfully complete that year, but, conversely, not all those who do complete will go on to start Secondary schooling the following year.
-To provide an indicator of Primary Completion that is possible to calculate from Enrolment and Population data only, the Millennium Development Goals use the Gross Enrolment Ratio for the last year of primary as a proxy for Primary Completion Rate. (GER is discussed in detail later in this chapter.)\\
-//**Meaningful Comparison**//\\
-To allow meaningful comparison over time and between countries, an indicator must be independent of scale. That is, values should be comparable between large and small education systems.
-To illustrate, the value Number of primary enrolments, while important data, is not a useful indicator. Natural growth in population may ensure that your school population rises, but this tells you nothing about the quality or performance of the education system.  Similarly, two countries with very different population sizes will most likely show very different numbers of enrolments. Again, this provides no useful insight into how well the education system is working, or how the performance of those countries compares.
-By contrast, the value Number of primary enrolments as a percentage of primary age population is a good candidate for an indicator. Nations large and small aspire to the same target value – 100% - and can sensibly compare results. Changes in this value over time do reflect the progress of your nation towards universal primary education. \\
-//**Ratios are Best**//\\
-This example demonstrates an important principle – useful indicators are often a ratio of two values. This removes any effect of scale, and gives a measure for which we can usefully set targets.
-For example, the question “How many teachers does the education system need?” has no general answer suitable for all time – as the population grows, we will obviously need more teachers. But we can meaningfully specify a target pupil/teacher ratio – the number of pupils in the system for each teacher – and compare this ratio over time to track whether the supply of teachers is improving or deteriorating.
-Enrolment Ratios, Pupil/Teacher Ratio, Pupil/Toilet Ratio, Repetition Rate, Survival Rates, Transition Rates, Percentage of qualified teachers – the most common and meaningful indicators are all ratios of two independent pieces of data. \\
-//**Gender Parity Index**//\\
-In this context, we should mention a family of indicators that divide a value calculated for females by the same value calculated for males.
-Such an indicator is called a gender parity index. GPI indicators highlight the progress towards gender equality in the education system.
-To illustrate, consider the Enrolment Ratio (Primary enrolments divided by primary age population) we discussed before. We can calculate this value for the whole population, and separately for boys and girls. Dividing the ratio for girls by the ratio for boys gives us a GPI for Enrolment Ratio. If the same percentage of girls is at school as boys, the GPI is equal to 1. Values less than 1 indicate that girls are enrolled in school less than boys. Values greater than 1 indicate girls are enrolled in school more.
-Notice that any GPI is not symmetrical: 1 represents equality, and while the value of a GPI can never be lower than 0, it can become arbitrarily large if the value for girls is much bigger than the value for boys.\\
-**Monitoring and Evaluation**\\
-Indicators are essential to the twin process of monitoring and evaluation.
-Your education department or ministry will initiate projects or programs of action aimed to improve the education system in your country. Such programs may range from the small and very specific to an overarching sector-wide five-year Strategic Plan.
-Each program will aim to achieve certain outcomes, and will plan various activities designed to achieve those outcomes.
-When you monitor, you check that the activities for an on-going project or program are being carried out as planned.
-When you evaluate, you judge whether the outcomes for a completed project or program were achieved.
-Indicators provide you with the numerical measures you need to perform this monitoring and evaluation. Note that the indicators you use to monitor a project may be different from those you use to evaluate it.
-However, calculating the value of an indicator at various times is not enough to allow us to measure our progress, or evaluate success or otherwise of our efforts. We need to compare those calculated values to determined standards, such as
-	target;
-	baseline;
-	benchmark.
-//**Target**//\\
-When you establish a project or program, you can specify target values for each indicator to declare explicitly the results you want to achieve.
-Targets for monitoring indicators may declare the amount of activity that must be done in each year of the program.
-Targets for evaluation indicators reflect the outcomes we hope to achieve from the program.
-//**Baseline**//\\
-To understand the changes made by any program, you must know the situation before the program started. This means you must calculate the values for all program indicators prior to the program commencement. This set of indicators is known as the baseline data.
-To establish useful baseline data, you need a full year of data collection prior to the commencement of your program. For this reason, it is important to establish the monitoring and evaluation indicators as early in the design of the program as possible. This allows time to plan and implement any required data collection as part of your FedEMIS survey process.
-//**Benchmark**//\\
-Benchmarks are values for indicators calculated in other countries, or across the region or the world. Using benchmarks, we can learn from the experience of others as to what may be best practice. Benchmarks help us to understand what it may be possible to achieve.
-//**M&E Framework**//\\
-The set of all indicators used to monitor or evaluate a program, together with the targets, baselines and benchmarks for those indicators, is called the Monitoring and Evaluation Framework for the program. A Monitoring and Evaluation Framework can be associated with the Education Sector Strategic Plan. Such a framework is the key to understanding progress towards the objectives of the strategic plan.
-===== Education Indicators (At a Glance) =====
-FIXME
-Use this section to understand the core indicators of Pacific EMIS, how they can be used and how they are calculated.\\
-**How Many Children are At School?**\\
-Using databases of enrolments in schools give us basic demographic capabilities such as number of students. Such numbers don't tell us much though. We can also use these data to give us other more telling indicators such as:\\
-//**Enrolment Ratios:**//\\
-Enrolment ratios (below) measure the number of actual enrolments against the number of potential enrolments to help identify children not attending school.
-//**Gross Enrolment Ratio (GER):**//\\
-This well-established indicator tells us the number of pupils enrolled in a given level of education, regardless of age, expressed as a percentage of the population in the theoretical age group for the same level of education.
-To Calculate: Divide the number of pupils (or students) enrolled in a given level of education regardless of age by the population of the age group which officially corresponds to the given level of education, and multiply the result by 100. \\
-//**Net Enrolment Ratio (NER):**//\\
-NER is a refinement of GER. It is defined as the number of enrolments of the official age group for a given level of education, expressed as a percentage of the corresponding population.
-To create a meaningful enrolment ratio, we need to identify that part of the population which is the “official” age range in question.  For this reason, we need to identify the “official” age group for any education level by considering the minimum and maximum year of education for that education level, and the official start age for primary school.
-To Calculate: Divide the number of pupils (or students) enrolled who are of the official age group for a given level of education by the population for the same age group and multiply the result by 100.
-//**Age Specific Enrolment Ratio (ASER):**//\\
-ASER shows the percentage of children of that age who are enrolled in school at any level. We can calculate age participation rates for single years of age, and for ranges of ages. Commonly, we may want to calculate age participation rates for age brackets corresponding to the official enrolment age for each Education Level. The APR differs from the NER of primary in that it includes children of primary age enrolled in other Education Levels. (These may include a small number of early entries into Secondary, or primary age children enrolled in ECE programs.)\\
-These well-established indicators are based on population. Therefore, they can only be as specific as the population data they use. For example, if population data is only available by state, GER and NER can only be calculated by state, or nationally. It is unusual to calculate enrolment ratios by school. This is because it is not practical to try to identify a source population who should be enrolled at a single school.\\
-//**GER can exceed 100%:**//\\
-It is important to note that there will be children attending primary school who are outside the official age range. (Most often, these children are older than the official age range.) Therefore, it is possible – indeed common - for the GER to exceed 100%.
-There are two principal explanations for the presence of children older than the official age range:
-§	children may be older than the official start age when they first enrol in school;
-§	some children may repeat one or more class levels, and so they do not complete the education level in the official number of years.
-GER values greater than 100% should prompt you to examine repetition rates and intake age to explore the reasons for this overage enrolment.\\
-//**NER can exceed 100%:**//\\
-NER should not exceed 100%. However, it is not uncommon for calculated NER to take values greater than 100% - implying there are more children of official age in school than there are children of official age!
-When NER is greater than 100%, one or both of these conditions must have occurred:
-§	the enrolment values are too high;
-§	the population data is too low.
-Enrolment values may be too high if schools systematically overstate their level of enrolment. This can occur when there is an incentive to inflate the enrolment, such as a school grant calculation tied to reported enrolment numbers.
-You may accidentally inflate enrolments if FedEMIS supplies estimated enrolments for schools that have closed or are not operating. For this reason, it is important to verify that all schools which do not return a survey are in fact still operating.
-The population numbers that you supply to FedEMIS are often estimates, rather than actual values taken from a population census. A demographer can generate population estimates for future years using mortality and fertility rates calculated from the census data.
- NER values greater than 100% may flag that the calculated population projection is too low. The National Statistics Office may need to reconsider the assumptions behind the population modelling, then regenerate the population estimates.\\
-//**Interpreting Enrolment Ratios:**//\\
-NER measures the percentage of children who are in school at the appropriate level. For NER to reach 100% for a particular education level:
-§	all children of the official age range for that education level must be attending school; and
-§	those children must be attending school at the appropriate education level.
-GER must always be greater than NER. The difference between them is a measure of the overage and underage enrolment. \\
-//**The Path to Universal Education:**//\\
-Taken together, changes in GER and NER over time mark the progress of a nation towards universal primary education. As rising quality reduces repetition, and most children commence school at the official age (ie Net Intake Rate approaches 100%), NER and GER stabilise near 100%.\\
-//**Student Attendance Rate (SAR):**//\\   SAR is ratio of the student’s attendance. If a student’s absences are measured in days they can be compared against the actual school days. For example if student X missed 15 days (A) from a potential 150 school days (B) then the Student’s Attendance ratio is 85%. This ratio can then be aggregated by school, island, nationally, gender, school type etc. The challenge then is to classify absence (sickness, family reasons), qualify absence (approved, non-approved) and quantify the absence (late, absent all day, absent part-day).\\
-**How Far Do They Get?**\\
-It is important to measure how far children progress through the education system before leaving. GER, NER and participation rates do not in themselves gives us this information. We must use other indicators for this purpose. The most important indicators of progress through the system are:\\
-//**Transition Rate:**//\\  Rates measure the percentage of children who reach the last year of one level of education (for example, primary, junior secondary) who proceed (or transition) to the next level of education (junior secondary, senior secondary, Tertiary). A high Transition rate also reflects the intake capacity of the next level of education.
- To Calculate: Divide the number of new entrants in the first grade of the specified higher cycle or level of education by the number of pupils who were enrolled in the final grade of the preceding cycle or level of education in the previous school year, and multiply by 100.
-//**Survival Rate:**//\\ Survival Rate measures the percentage of students enrolled in a given level of education, who reach a second given level without dropping out. Survival Rate shows how well the education systems can retain students (with or without repetition) and indicates the dropout rate. It is also used to measure the impact of repetition and dropout on internal efficiency. The distinction between survival rate with and without repetition is necessary to compare the extent of wastage due to dropout and repetition. \\
-Survival rate to the last class of Primary school is an official indicator to track progress toward Millennium Development Goal (MDG) 2 - Indicator 2.2 Proportion of pupils starting grade 1 who reach the last grade of primary. In this regard, survival is defined as the percentage of a cohort of students who enter the first class of Primary Education and who are expected to reach the last class of Primary Education, regardless of repetition. Survival rates approaching 100% indicate a high level of retention and low incidence of drop out.\\
-Survival rate only relates to children who have commenced school – hence the reference to students. For a more complete picture of the proportion of children in the overall population who complete schooling, survival rates should be considered along with net enrolment rate in the first year of schooling.   \\
-Some caution needs to be exercised in the interpretation of the survival rates with the presence of Drop Ins. The Drop Ins do not invalidate the survival rates calculated however, it can not be implied that the students who started in Class 1 are the same students who are in Class 6 because some of those surviving to Class 6 may be Drop Ins. This is why the statement to be made for survival rates is “for every 100 students starting Class 1, 106 students will reach Class 6” rather than “106% of students starting Class 1 will reach Class 6”.   \\
-//**Repetition Rate:**// This measures the number of pupils enrolled in a given level of education, who repeat that same level the following year. High or rising Repetition Rates can indicate problems in the internal efficiency of the education system and possible poor quality instruction. The aim is to have a low and reducing repetition rate. \\
-//**Completion Rate:**//\\
-Completion/graduation rate for 8th and 12th grades (private and public). Primary completion (completed Grade 8) differs from Survival in that those that survive to Grade 8 may not complete Grade 8. \\
-**How Long Do They Stay?**\\
-As you measure the levels of enrolment reached by children in the education system, you’ll also want to consider how long it takes them to reach those levels. In particular, you need to know how many years a child remains in the system.\\
-//**School Life Expectancy:**//\\
-School Life Expectancy is an important and simple measure of how long a child of a given age may expect to remain in school.\\
-//**Pupils Years in Grade:**//\\
-We may wish to know how many years a pupil who reaches a particular class level spends at that level. This indicator, the Pupil Years in Grade, is another calculated from the reconstructed cohort method. \\
-//**Measuring Efficiency:**//\\
-Survival, transition and completion rates do not indicate the path taken by the child to reach a particular level, only that they got there eventually. Relating the levels of education reached to the number of years taken to reach them introduces the concept of the efficiency of the education system.\\
-**What Do They Learn?**\\
-//**Standardised Testing:**//\\
-In the Pacific region, most nations periodically conduct a Standard Test of Achievement in the fourth and sixth years of primary education. These tests measure basic competencies in literacy and numeracy, and basic general knowledge.
-In FSM this test is known as **NMCT - National Minimum Competency Test** For this indicator, it is proposed to measure the Percent of students achieving “Proficiency” level & above, at selected grade levels for standardized tests (e.g., SAT ) or criterion-referenced tests (e.g. Locally developed tests)\\
-**Do We Have Enough Teachers?**\\
-Teachers are the biggest investment that any country makes in its education system. Indicators to help planners manage the supply and training of teachers are critical to any education ministry.\\
-//**Pupil/Teacher Ratio:**//\\
-Pupil Teacher Ratio (PTR) is the most important indicator of teacher numbers. PTR is simply the number of pupils divided by the number of teachers. An EMIS should calculate PTR at a school, and at any aggregate level derived from school – island, District, Authority and so on. It is not meaningful to calculate PTR by gender.
-UNESCO recommends that for EFA 11, PTR is calculated using gross teacher numbers. However, the PTR we calculate in this way may not truly reflect the ratio of pupil to teachers in classrooms, due to part-time teachers, and teacher employed as full-time administrators.
-If your education system includes many part-time teachers, you may wish to report on the indicator Pupil/Teacher FTE Ratio. This indicator divides the number of pupils by the sum of the teachers’ FTE (full-time-equivalent) value. Full time teachers have an FTE of 1, teachers teaching only one day out of 5 have FTE .2. Teacher FTE needs to be collected as part of the school census.
-Other derivatives of this ratio can be based on teacher’s highest qualifications and certification status.\\
-//**Teaching Staff highest qualification:**//\\
-Teaching staff grouped by highest qualifications. \\
-//**Teacher Attendance Rate:**//\\
-Teacher (and student) attendance is directly related to student outcomes: the more teachers are absent, the more their students’ achievement suffers. \\
-**How Much Do We Spend?**\\
-Within the limits of the national budget, the demands of education must compete with other government priorities.
-It is the challenge of every education ministry to use those funds to deliver the education outcomes that will most benefit the individual students and the nation as a whole. Often Gross Domestic Product (GDP) or Gross National Income (GNI) are used.\\
-//**Education Budget per student:**//\\
-This simple indicator divides the total education budget by the number of students. Like most indicators, this indicator can be disaggregated. For example, if individual State budgets are known, the indicator can be disaggregated to show state ratios.
-Taken on its own, this indicator is limited. For example, should the unit cost be increasing or decreasing? Is higher expenditure (increasing) or increased efficiency (decreasing) the goal?\\
-**Other Indicators**\\
-===== Managing Schools =====
-FIXME
-===== Managing Teachers =====
-FIXME
-===== Managing Performance Assessment =====
-FIXME
-===== Managing Surveys =====
-==== Online Web Application Survey Submission ====
-FIXME
-==== Excel Workbook Submission ====
-FIXME
-===Main features of the workbook===
-The workbook main objective is to improve the quality of data collected at the state level and transport that data to NDOE (students & enrolments, staff, schools and WASH data). It is updated by each SDOE and sent to NDOE twice a year (a) in September/October as an enrolment update and (b) in June/July as both for the outcomes of the year (graduations, drop outs, transfers etc).
-States can use this workbook in a number of ways:
-States can use the workbook as their own EMIS (Chuuk and Kosrae). They can keep the student and teacher data up-to-date in response to events, and send copies of the data to NDOE at the required times. States can also share the workbook data with common reporting tools (Chuuk are using MS Access) in order to generate their own analysis. States will also have access to FedEMIS which will hold a good deal of reporting at national, state and school level.
-States that run their own EMIS (i.e Pohnpei state use PEDMS and Yap are implementing YAP WEB) need to coordinate data between their own systems and the workbook twice a year. Each case will be different depending on how the state gathers and stores its data.  This requires some level of skill in data manipulation, import/export and cut/paste. NDOE can assist/advise if requested.
-The workbook will also provide two other functions to improve admin (as at 31 August 2017 these functions are planned but are not ready or tested):
-a facility to Rollover into a new year, setting up enrolments based on each individual
-an ability for a master workbook to import one or more secondary workbooks. This allows states with large student numbers to share the work among more than 1 staff member, creating copies of the workbook, and then importing those secondary copies into the master. Schools that are trained in use of a secondary workbook (by SDOE) can also complete a workbook and forward it to SDOE, where SDOE can import that school workbook into the master.
-==== PDF eSurvey Submission ====
-The PDF eSurvey is simply a PDF document that teachers and principals
-can use to enter the data of their surveys instead of completing this
-process on paper. This new technology as a number of advantages:
-  * Easier work for data entry officers as the need to re-enter all data in the database is now no longer necessary. Data entry officers can now better use their time for verification and more creative analysis and use of the Pacific EMIS.
-  * PDF eSurveys can be completed by anyone with a tablet or laptop and there is no need for a reliable Internet connection