1         Background

Rwanda is among the top performing countries in sub-Saharan Africa in primary education with a net enrolment rate of 98.3 per cent (98 per cent for boys and 98.5 per cent for girls) (MINEDUC 2018). Progress towards Sustainable Development Goal (SDG) 4 is encouraging, with gender parity achieved at the primary-level, primary, and secondary levels of education.

However, these gains at the primary level need to be matched with improvements in quality of education throughout the system. Results from the Learning Achievement in Rwandan Schools, 2018 indicate that there are significant issues in numeracy and literacy for most students in Rwanda.

The curriculum has undergone a major reform and has been introduced across all grades from primary-level to upper secondary. The curriculum is learner-centred, competence-based, inclusive, flexible, transparent and accountable. The revision aims to develop children to become lifelong learners with the necessary fundamental skills to thrive in the 21st century. The curriculum recognizes the importance of play and integrates play-based learning throughout to achieve the goals laid out within the document.

UNICEF has, therefore, been exploring strategies to bridge the skill gap in teacher abilities to effectively deliver the competency-based curriculum. This is because it is acknowledged that the implementation of such a curriculum demands a quantum shift from traditional methods of instruction to participatory and interactive methods that allow for active involvement by learners in groups and as individuals. This, in turn, requires a dramatic change in the curriculum and delivery methodology for in-service teacher training.

To date, UNICEF Rwanda has been supporting the National School-based Mentorship Programme (SBMP), the Ministry of Education’s (MINEDUC) in-service programme, utilising school-based mentors, to effectively implement the revised curriculum.

Given that the initial capacity of teachers has been improved to understand the basics of child-centred teaching and learning and the application of the revised curriculum, UNICEF and the Government of Rwanda (GoR) intend to focus on building the capacities of teachers in LtP. Partnership with the LEGO Foundation to strengthen LtP is an opportunity to support the commitment of the MINEDUC for effective implementation of the competency-based curriculum.  

In the primary years, active, play-based learning approaches can transform the educational experiences of children in primary grades and strengthen learning motivation and outcomes.  Play creates powerful learning opportunities across all areas of development, including motor, cognitive and social and emotional skills.=

1.1  Learning through Play

UNICEF Rwanda, in partnership with the LEGO Foundation, will be working to embed Learning Through Play (LtP) in the primary in-service teacher training system. 

Building upon the primary school curriculum, which already supports play-based teaching and learning, this partnership will work to build the capacity of teachers to integrate LtP within the classrooms. 

This will be achieved through the integration of LtP into the existing in-service teacher training programme, primarily involving school-based mentors (SBM). The SBM are teachers who support their colleagues in developing the knowledge and skills to improve quality education in Rwanda.  Within the scope of this project, school-based mentors will be equipped with the skills to work with teachers and build their respective knowledge, attitude, and skills to apply LtP pedagogy in primary schools. 

Ensuring that the results achieved are sustained, the work will include building a supportive environment at the school and community-levels, through the transformation of attitudes and behaviours towards learning through play.

The results of this partnership will be realized beyond the lifecycle of the project, as it is expected that long-term systematic change will be achieved through the development of an evidence-base and entrenching the work in existing government systems.  

Working in each of the 2,900 Government primary schools in Rwanda, approximately 29,000 teachers (10 teachers per school) will be reached and supported to integrate LtP in their classrooms, resulting in approximately 1,450,000 primary school children benefiting from the transformative power of learning through play to develop the intellectual, emotional, social and creative skills to serve them, their communities, and society.

1.2  Timeframe

Learning through Play programme will start January 2020 and run through December 2023. The baseline evaluation is expected to be undertaken in March 2020 and the endline evaluation in September 2023.

1.3 Purpose

The purpose of this research is twofold:     

i)    to establish a baseline by which to inform programme design and implementation;

ii)  to set a basis of comparison for the end-line research, to be conducted September 2023. 

The end-line research is intended to include the element of impact assessment and capture the extent to which the programme has improved LtP in primary-level classrooms, improved quality of learning of primary-level students, and improved the learning through play knowledge, attitudes and practices of teachers, head teachers, and parents of primary-level students.

The end-line research will build on the baseline data to analyse programme impact, programme successes and failures, and to develop new guidance to inform and improve UNICEF’s and IEE LtP and in-service teacher training.

Given the expected workload of documenting and evaluating the initiative at national and local level, the volume of technical deliveries, and the required technical expertise, the tasks are beyond the current capacity of UNICEF Rwanda’s in-house personnel. Therefore, support from an experienced institution is required to implement the research.

2. Object and Objectives of the Research

The primary objective of the research is to establish a baseline for the Learning Through Play programme which UNICEF and Inspire, Educate, Empower (IEE), with the support of LEGO FOUNDATION, will be implementing from January 2020 – December 2023.  

The stakeholders involved in the research are UNICEF, MINEDUC, REB, District officials, NISR, and non-governmental organisations.

The Theory of Change / Logic Model is elaborated upon in section 3.

In order to inform learning through play programming, baseline and end-line research capturing the impact of the Learning Through Play interventions is required to assess successes, shortcomings, the replicability, and its contribution to evidence-based policy change.

The baseline research will be used to assist in designing the programme and informing programme implementation.  In addition, the baseline research will provide a basis for comparison to the end-line (intended to be conducted in 2023).  

Primary objectives of the baseline study:

1.  to establish a baseline by which to inform programme design and implementation;

2.  to set a basis of comparison for the end-line research, to be conducted September 2023.  

 Specific aims of the baseline study are as follows:

1.    Establish the baseline information on selected aspects of behaviours required to achieve the outcomes that are the target of the intervention;

2. Establish the baseline information as to the capacity of the existing systems (government/school structures and capacities), parents (behaviours and attitudes) and teachers (skills, knowledge and attitudes) to effectively support delivery of the programme;    

3 .Scope, Focus and Research Criteria

The baseline and end-line will ensure adequate data collection of primary-level students, their schools and teachers, and of their parents and community members. The sample size for the research will be largely driven by the selected research design and should be proposed by the research institution. But, the sample should be representative for the project taking into consideration the need to disaggregate data by gender and geography (urban/rural, different Districts).

The development hypothesis of Learning Through Play is that IF: 1) LtP pedagogies are integrated into the in-service SBM  programme; 2) school systems are supportive environments for LtP; 3) that teachers develop the knowledge, attitudes and behaviours to incorporate LtP pedagogies into their classrooms; and 4) that parents/caregivers are supportive for the implementation of LtP pedagogies, THEN girls and boys, between the ages of 6 – 12 years, are afforded the opportunity for quality learning through play to develop the necessary 21st century skills to be engaged, creative, and productive lifelong learners. 

The input of the Learning Through Play is, therefore, multi-dimensional, and includes provision of play-based materials to Government primary schools; training and capacity development of teachers, head teachers, and other education stakeholders; and engagement of parents and community members.

The research should be designed to capture data on key project inputs.

The methodology should be in accordance with the methodology outlined in the methodology chapter below. The institution is expected to follow questionnaires appropriate for quantitative and qualitative impact research and implement the survey in the field.   All survey instruments should be based on a thorough review of relevant literature and existing questionnaires, including Multiple Indicator Cluster Surveys (MICS). 

The research should focus on and include a wide array of beneficiaries and stakeholders. These include children, teachers, head teachers, parents and community members.    

The baseline study is to support the end-line programme evaluation aiming to assess the impact of the programme on learning through play. For this reason, the baseline study should establish an objective measure of capacities of parents and teachers in ensuring quality, play-based, learning of primary-level children.

Thus, the study should establish the robust baseline information on impact and outcome indicators following the programme results framework impact and outcome indicators.

The baseline research will be formed around the following areas of intervention:

i)    the quality of learning through play;

ii)   education stakeholder support of and involvement in learning through play;

iii)   community and parental involvement in supporting children’s primary-level learning through play;

The main indicators will include, but not be limited to:

• Direct child-level indicators such as: learning outcomes, Umudehe category, level of education of main caretakers, aligned with the national curriculum;
• School-level indicators such as:  use of learning through play materials, school leadership involvement;
• Knowledge, attitudes and awareness such as parents/caregiver and community knowledge, beliefs and expectations of learning through play; Parent/caregiver (family) and community engagement of learning through play outside of the classroom;  

The OECD DAC Criteria for Evaluating Development Assistance will be the guiding reference in establishing the questions for end-line research; the baseline research should consider evaluation questions around relevance and questions that provide a base to measure the effectiveness, impact and sustainability at the end-line.

4. Methodological Approach & Process

The research needs to adhere to the Government of Rwanda’s research standards. Further key policies and performance standards to be referenced in evaluating the programme are described in the United Nations Research Group (UNEG) “Standards for Research in the UN System” and in UNICEF Research “Policies and Principles”. The basics of human rights-based approach and results-based approach to programming are described in the UNICEF Programme Policy and Procedure Manual.

The research methodology will be guided by the norms and standards of the UNEG, and the UNEG guidelines on integrating Human Rights (HR) and Gender Equity (GE) in Research. In order to be responsive to HR and GE aspects, special consideration will be given to gender, sex, distance from service locations and wealth when stakeholders and beneficiaries’ view are sought in data collection. In the design phase of the research framework, careful considerations will be given to such inclusion aspects. In the analysis phase, appropriate disaggregation will be attempted to shed light on HR and GE elements.

The research institution will work with UNICEF and key stakeholders involved in in-service teacher training and primary-level education to develop and finalize the design and will conduct the research. The research team will be responsible for designing tools, field visits, data collection, data analysis and drafting of the report. The research team will work with stakeholders to coordinate the work, conduct interviews, conduct the data collection and analysis, and disseminate the findings of the research.

4.1 Methodological Approach

The suggested methodology is as follows:

Type of methodology: The research institution is expected to propose the most appropriate methodology in order to assess impact and attribute results to the intervention. The methodology should include quantitative and qualitative data. The qualitative component will draw on the understanding and perception of the main stakeholders involved in the Learning Through Play programme. It is expected that the research institution specifically elaborates on (innovative and participatory) tools in the methodology to capture a baseline in measuring a change in knowledge, attitudes and awareness towards LtP amongst teachers,  parents/caregivers, and representatives of the administrative structures, as well as a baseline in measuring an increase in capacity at teachers’ level.

Data Source: Primary data will be collected from intervention primary-level classrooms involved with LtP. Moreover, data will be collected from teachers and parents/caregivers of children at the primary-level age. Primary data will be collected at institutional level from primary service providers and representatives of the administrative structures responsible for service delivery. 

Suggested qualitative data analysis

The qualitative analysis will focus on obtaining key information from a wide array of primary-level stakeholders in the selected sites, including primary-level district focal points, primary-level head teachers and teachers and parents/caregivers of children in the baseline intervention. Specifically, the qualitative data analysis will focus on knowledge, attitudes, and practices of parents and community members, as well as key education stakeholders, in regard to their engagement and understanding of LtP amongst their primary-level aged children. 

The qualitative data collection will be conducted simultaneously to quantitative data collection.

As defined under the scope and objectives of the study, the methodology should provide a robust baseline following the programme objectives and critical programme components including assessing knowledge, attitudes, and behavior of teachers and parents. The methodology should take into consideration is that the baseline should be a solid base for assessment of the impacts of the programme at its end.

Suggested quantitative data analysis

The quantitative analysis will include both descriptive analysis and advanced multivariate regression analysis, if relevant. It will provide information on all pre-intervention indicators for the intervention.

Appropriate questionnaires developed using the literature review and the lessons learned from already implemented studies will be used to develop questionnaires, adjusted to Rwandan context. The quantitative analysis will include both descriptive analysis and advanced multivariate regression analysis. It will provide information on all pre-intervention indicators for the intervention/treatment and control groups (if the methodology includes control groups). In addition, the multivariate regression analysis may be used to test the soundness of the theory of change by identifying main determinants of primary-level reading, at baseline level.

It is expected that the research institution incorporates the qualitative and quantitative data analysis, applies triangulation where possible, and develops one comprehensive analysis that will be presented in the baseline report.

4.2.Sampling

The consultancy institution will develop the research sampling strategy and design the sample, in line with the proposed and accepted methodology. 

The sample should be representative for the project taking into consideration the need to disaggregate data by gender and geography (urban/rural, different Districts).

The overall methodology and sampling strategy need to take into account how to differentiate between the impact of the various primary-level interventions being rolled out (including those besides Learning Through Play) and include maturation effects on the children in the study. In addition, the methodology and sampling strategy needs to consider a potential attrition rate. 

The Learning Through Play sampling strategy will based on random sampling of primary caregivers within selected sites. The eligibility criteria will be that participants will have to be the primary caregiver and legal guardians of a child between the ages of 6 - 12 years of age who is enrolled in primary-level school and live in the same household as the child.

4.3 .Process

The research process and methodology will include three phases:

Phase 1. Inception: 

·  Research Plan development – draft work schedule to be submitted to the UNICEF for review and approval.

·  In-depth desk review of available information on LtP programme documents, primary-level data from MINEDUC, Education Sector Strategic Plan, and any other relevant information – to guide the development of the research questions. Review the current status of children in Rwanda with focus on primary-level education and in-service teacher training components.

·  Preliminary discussions with UNICEF Rwanda, IEE and the Basic Education Technical Working Group, to facilitate a common in-depth understanding of the conceptual framework, refining the research questions and adjusting data collection methods, tools and sources. The consultancy team will have the opportunity to discuss and propose amendments to the methodology, as long as the purpose of the study is maintained and expected deliverables are produced at the required level of quality. Any proposed changes will be discussed at planning meetings with key stakeholders at the beginning of the process.

 .  Drafting of Inception report (deliverable 1), including the details of the methodology, sampling approach, revised list of key indicators for qualitative and quantitative measurement, list of proposed research participants, as well as a Research Matrix for each finally agreed research question and a detailed analysis plan, to be presented to and approved by the members of the Steering Committee (UNICEF, Rwanda Education Board). The proposed methodology needs to be sufficient to capture all the indicators agreed for this research purpose.

Phase 2. Baseline Data collection:

·  Develop data gathering tools. The battery of tools is to be developed based on internationally recognized tools, such as the Early Grade Reading Assessment (EGRA). Measures are to be selected that determine key areas of the Learning Through Play intervention and are intended for use as part of the end-line research to assess the impact of the Learning Through Play programme.

·  The development of data gathering tools, interview and observation guides using a consultative participatory process, will include the following steps:

o   Facilitate a meeting to share the tools for comments with the Task Force overseeing the consultancy.

o   Pre‐test the tools and facilitate a meeting with the Task Force, to validate the pre‐ testing data obtained and fine tune the tools based on the experiences from the team of interviewers/enumerators.

o   Train research assistants /enumerators for data collection in all the selected sites to ensure that standardised methodology and application of the tools are fully understood.

o   Conduct collaborative planning for field work with the Task force, including development of agenda and activities.

·  Work with team of interviewers/observers to collect data using quantitative, qualitative observations and other appropriate methods (measurable indicators) from children, parents, teachers, head teachers, and community members:

o   In alignment with the agreed methodology, the consultant firm will collect quantitative data at intervention/treatment and control sites.

o   The consultant firm will conduct focus group discussions and in-depth interview with parents and community members in the baseline intervention and control groups

·  Convene a briefing meeting with the supervising task force on the on‐going survey for corrective measures to be undertaken if need be.

·  Ensure the quality of information collected from fields, cross check with the validity and reliability of information collected and verify

· The research institution will submit regular progress report and field reports throughout data collection period and provide a final report on the baseline data collection. (deliverable 2)

Phase 3. Analysis and reporting phase: 

· Process data (data entry, cleaning, and analysis) using the SPSS/STATA/ EpiInfo/ NVIVO or any acceptable statistical data analysis package for quantitative as well as qualitative data gathering and analysis.

·  Following the data collection and analysis phase, the evaluation team will make a presentation of the preliminary key findings (First and preliminary findings report - deliverable 3) to the Task Force.

·    A detailed draft technical report will be shared with key partners for review, and once all feedback of Task force members have been addressed, the consultant firm will share a final report (deliverable 4) for review and validation by the Primary-level Evaluation Task force.

· A concise (approximately 40 pages) summary report (deliverable 5) of the key findings of the validated technical report drafted and shared with the Task force.

·  All outputs from the research are subjection to approval by task force.

Please refer to the attached request for proposal for detailed TORs and instructions.