TERMS OF REFERENCE

Baseline Research for Itegure Gusoma Programme

Position Title: Institutional consultancy for the baseline research of the Pre-primary / Itegure Gusoma programme

Level:  High-level professional institution

Location: Kigali, Rwanda

Duration: 1 year

Start Date: June 2017

Reporting to: Chief of Education

Budget PBA No: SC170216

 

Contents

1       Background. 2

1.1        Itegure Gusoma. 2

1.2        Timeframe. 3

1.3        Purpose. 3

2       Object and Objectives of the Research. 4

3       Scope, Focus and Research Criteria. 4

4       Methodological Approach & Process. 5

4.1        Methodological Approach. 6

4.2        Sampling. 7

4.3        Process. 7

5       Major Tasks, Deliverables & Timeframe. 8

6       Stakeholder Participation. 10

7       Existing information sources. 10

8       Ethical Consideration/confidentiality. 11

9       Research team composition / qualifications and requirements. 11

10    Supervision. 13

11    Terms and conditions. 13

12    How to apply. 14

1 Background

While Rwanda has one of the highest primary school enrolment rates in Africa (97.6 percent),[1] ensuring that all students learn to read in the early grades remains a challenge. A reading assessment conducted in 2015 by the USAID-funded Literacy, Language, and Learning (L3) activity found that half of Primary Grade 1 (P1) students were unable to read a single word of grade-level text.[2] One reason for this alarming finding is that students do not enter P1 with the necessary school readiness and emergent reading skills.

Global evidence suggests that investment in quality pre-primary education is an effective way to spur improvements in early-grade reading.[3] Save the Children’s Early Literacy and Maths Initiative End-line Assessment found that children who attended high quality pre-primary centers in Rwanda were not only better prepared for P1, but also retained their advantage into P1 and had doubled the gains of their peers who attended low quality pre-primary centers.[4]

The Government of Rwanda has recognized the importance of pre-primary education; progressive expansion of pre-primary and school readiness programs are among the key priorities in the Ministry of Education’s (MINEDUC) Education Sector Strategic Plan (ESSP 2013-2018). In this plan MINEDUC has set targets to increase the net enrolment rate in pre-primary from the low 2015 level of 14.2 percent (14 percent boys, 14.5 percent girls)[5] to 30 percent by 2017/2018.[6] This ambition is further defined in the agreement between MINEDUC and the Global Partnership for Education which reflects an aim to increase the pre-primary gross enrollment rate (GER) in the 22 poorest performing districts (i.e., the districts that had GER of less than 17 percent in 2014) from 10 percent in 2014 to at least 17 percent in 2017.

1.1 Itegure Gusoma

To that end, UNICEF and USAID are partnering with the Ministry of Education (MINEDUC) and Rwanda Education Board (REB) to model pre-primary education, with a particular emphasis on early literacy and emergent reading.  The programme is called Itegure Gusoma (Get Ready to Read) and will target children in 80 pre-primary schools, in sixteen districts: Bugesera, Burera, Gatsibo, Gicumbi, Gisagara, Karongi, Muhanga, Ngoma, Ngororero, Nyagatare, Nyamagabe, Nyamasheke, Nyaruguru, Rubavu, Rusizi and Rwamagana.

Within the programme, the pre-primary schools will be equipped with reading books and appropriate teaching and learning materials; teachers, head teachers, and education stakeholders will be trained in early grade reading and play-based education; and parents and community members will be equipped with the skills to support pre-primary reading. 

The goal of Itegure Gusoma is to increase the percent of learners in targeted schools who demonstrate increased learning outcomes in Kinyarwanda in Primary 1 through early grade reading support at the pre-primary level. This will be achieved by strengthening the quality of pre-reading instruction in targeted pre-primary schools.

The specific dimensions of quality instruction that will be addressed by Itegure Gusoma include: 1) availability and use of evidence-based, gender-sensitive pre-primary reading materials; 2) teachers’ implementation of evidence-based, gender-sensitive instructional and assessment practices; 3) supervision of pre-primary reading instruction; 4) community and parental involvement, and 5) gender and social inclusion.

Schools will be receiving support to improve early-grade reading in addition to Itegure Gusoma. By January 2018 all schools will be supported by the USAID-funded Mureke Dusome activity, which targets P1-3 students, aims to:

1. Strengthen the capacity of school leadership to improve student literacy through school-community partnerships;
2. Increase effective community and parental involvement to improve literacy skills; and
3. Foster a culture of reading.

In addition, by January 2019 all schools will be supported by the USAID-funded Soma Umenye activity, which also targets P1-3 students and aims to improve early-grade reading skills by:

1. Improving classroom insruction in early-grade reading (P1-3); and
2. Improving systemic capacity for early-grade reading instruction.

Itegure Gusoma is working toward similar objectives at the pre-primary level. The influence of multiple programs needs to be taken into consideration when designing the research i.e. the roll-out of these various early reading interventions may be attributable to activities outside of Itegure Gusoma. 

 

1.2 Timeframe

Itegure Gusoma has started in March 2017 and will run through till March 2020.   From March – December 2017, the programme design will be finalized, materials developed, and trainings and capacity development undertaken. 

Implementation at the school-level will run from January 2018 to December 2019, in 80 randomly assigned schools in sixteen districts:  Bugesera, Burera, Gatsibo, Gicumbi, Gisagara, Karongi, Muhanga, Ngoma, Ngororero, Nyagatare, Nyamagabe, Nyamasheke, Nyaruguru, Rubavu, Rusizi and Rwamagana.

To determine the impact of Itegure Gusoma baseline and end-line research is planned with the baseline to be conducted from September 2017 and the end-line to be conducted September 2019.   

1.3 Purpose

To support refinement of the programme design and its subsequent end-line evaluation, the programme design envisages implementation of the Baseline Study that will provide comprehensive and critical information:

1. to establish a baseline by which to inform programme design and implementation;
2. to set a basis of comparison for the end-line research, to be conducted in September 2019. 

The end-line research is intended to capture the extent to which Itegure Gusoma has improved instruction in pre-primary classrooms, improved the pre-reading skills of pre-primary students, and improved the literacy-related knowledge, attitudes and practices of teachers and parents of pre-primary students. The research will also determine the impact of Itegure Gusoma on reading outcomes for students in Primary 1.

The end-line research will build on the baseline data to analyse programme impact, programme successes and failures, and to develop new guidance to inform and improve UNICEF’s and VSO early reading and pre-primary programming.

Given the expected workload of documenting and evaluating the initiative at national and local level, the volume of technical deliveries, and the required technical expertise, the tasks are beyond the current capacity of UNICEF Rwanda’s in-house personnel. Therefore, support from an experienced institutions is required to implement the research.

2 Object and Objectives of the Research

The primary objective of the research is to establish a baseline for the Itegure Gusoma programme which UNICEF and VSO, with the support of USAID, will be implementing from May 2017 – March 2020.  

The stakeholders involved in the research are UNICEF, MINEDUC, REB, District officials, NISR, and non-governmental organisations.

The Theory of Change / Logic Model is elaborated upon in section 3.

In order to inform pre-primary and early literacy programming, baseline and end-line research capturing the impact of the Itegure Gusoma interventions is required to assess successes, shortcomings and the replicability to nationwide scale-up and its contribution to evidence-based policy change. The overall objective is to understand whether the intended objectives of the project have been achieved, in line with the plan, as compared with the results of the baseline research.

The baseline research will be used to assist in designing the programme and informing programme implementation.  In addition, the baseline research will provide a basis for comparison to the end-line (intended to be conducted in 2019). Moreover, it will provide a basis on which an end-line will be undertaken to assess the impact that the Itegure Gusoma programme on a range of pre-primary indicators. 

Primary objectives of the baseline study:

1. Inform the delivery and design of the integrated programme in selected programme sites; and
2. Allow for the end-line (anticipated in 2019) impact evaluation of the programme that will determine the extent to which the integrated programme intervention has been able to meet its objective.

 Specific aims of the baseline study are as follows:

1. Establish the pre-intervention indicators along the selected key impact and outcome indicators defined in the programme results framework;
2. Establish the baseline information on selected aspects of behaviours required to achieve the outcomes that are the target of the intervention;
3. Establish the baseline information as to the capacity of the existing systems (government/school structures and capacities), parents (behaviours and attitudes) and teachers (skills, knowledge and attitudes) to effectively support delivery of the programme;    

3 Scope, Focus and Research Criteria

The baseline and end-line will ensure adequate data collection in control and intervention groups of pre-primary students, their schools and teachers, and of their parents and community members. The sample size for the research will be largely driven by the selected research design.  But, the sample should be representative for the project taking into consideration the need to disaggregate data by gender and geography (urban/rural).

The Itegure Gusoma programme (which the baseline and end-line are to be based upon), propositions that an improvement in pre-primary literacy resulting in early and emergent reading skills for students in Nursery 3 will yield gains in reading and learning Kinyarwanda in Primary 1.   

The development hypothesis of Itegure Gusoma is that if the quality of pre-reading instruction in pre-primary classroom is increased, and effective community and parental involvement to improve pre-primary literacy skills is increased, then pre-reading skills among pre-primary students will increase. Further, if pre-reading skills among pre-primary students increase, then literacy outcomes for children in primary 1 grade will increase.

The input of the Itegure Gusoma is, therefore, multi-dimensional, and includes provision of books and establishment of a reading centre in the pre-primary classroom; training and capacity development of teachers, head teachers, and other education stakeholders; and engagement of parents and community members.

The research should be designed to capture data on key project inputs.

The methodology should be in accordance with the methodology outlined in the methodology chapter below. The institution is expected to follow questionnaires appropriate for quantitative and qualitative impact research and implement the survey in the field.   All survey instruments should be based on a thorough review of relevant literature and existing questionnaires, including the Early Grade Reading Assessment (EGRA) and Multiple Indicator Cluster Surveys (MICS). 

The research should focus on and include a wide array of beneficiaries and stakeholders. These include children, teachers, head teachers, parents and community members.    

The baseline study is to support the end-line programme evaluation aiming to assess the impact of the programme on early grade reading results. For this reason, the baseline study should establish an objective measure of capacities of parents and teachers in ensuring quality learning of pre-primary children.

Thus, the study should establish the robust baseline information on impact and outcome indicators following the programme results framework impact and outcome indicators.

The baseline research will be formed around the following areas of intervention:

1. the quality of pre-reading instructions;
2. community and parental involvement in supporting children’s pre-primary literacy skills;
3. pre-reading skills among pre-primary students;
4. current literacy outcomes among children in primary grade 1.

The main indicators will include, but not be limited to:

• Direct child level indicators such as: EGRA sub-skill tests including listening comprehension, oral vocabulary in Kinyarwanda, letter names, syllable reading;
• School-level indicators such as:  use of reading materials, school leadership involvement;
• Knowledge, attitudes and awareness such as parents/caregiver and community knowledge, beliefs and expectations of early reading; Parent/caregiver (family) and community engagement of early reading outside of the classroom;  

The OECD DAC Criteria for Evaluating Development Assistance will be the guiding reference in establishing the questions for end-line research; the baseline research should consider evaluation questions around relevance.

4 Methodological Approach & Process

The research needs to adhere to the Government of Rwanda’s research standards. Further key policies and performance standards to be referenced in evaluating the programme are described in the United Nations Research Group (UNEG) “Standards for Research in the UN System” and in UNICEF Research “Policies and Principles”. The basics of human rights-based approach and results-based approach to programming are described in the UNICEF Programme Policy and Procedure Manual.

The research methodology will be guided by the norms and standards of the UNEG, and the UNEG guidelines on integrating Human Rights (HR) and Gender Equity (GE) in Research. In order to be responsive to HR and GE aspects, special consideration will be given to gender, sex, distance from service locations and wealth when stakeholders and beneficiaries’ view are sought in data collection. In the design phase of the research framework, careful considerations will be given to such inclusion aspects. In the analysis phase, appropriate disaggregation will be attempted to shed light on HR and GE elements.

The research institution will work with UNICEF and key stakeholders involved in Pre-Primary education to develop and finalize the design and will conduct the research. The research team will be responsible for designing tools, field visits, data collection, data analysis and drafting of the report. The research team will work with stakeholders to coordinate the work, conduct interviews, conduct the data collection and analysis, and disseminate the findings of the research.

4.1 Methodological Approach

The suggested methodology is as follows:

Type of Study: The proposed methodology (cross-sectional, quasi-experimental, longitudinal, etc…) should be the most appropriate to assess impact and attribute results to the intervention.  The methodology should include quantitative and qualitative data; while the qualitative component will draw on the understanding and perception of the main stakeholders involved in the Itegure Gusoma programme.

Data Source: Primary data will be collected from intervention pre-primary schools (and control schools, dependent on methodology) and primary schools (tracking children into P1 from pre-primary).  Moreover, data will be collected from teachers and parents/caregivers of children at the pre-primary age. Primary data will be collected at institutional level from primary service providers and representatives of the administrative structures responsible for service delivery. 

Suggested qualitative data analysis

The qualitative analysis will focus on obtaining key information from a wide array of pre-primary stakeholders in the selected sites, including pre-primary district focal points, pre-primary head teachers and teachers and parents/caregivers of children in the baseline intervention and control groups. Specifically, the qualitative data analysis will focus on knowledge, attitudes, and practices of parents and community members, as well as key education stakeholders, in regards to their engagement and understanding of early reading amongst their pre-primary aged children.

Suggested modes of inquiry include focus group discussions and semi-structured interviews.

The qualitative data collection will be conducted simultaneously to quantitative data collection.

As defined under the scope and objectives of the Study, the methodology should provide a robust baseline following the programme objectives and critical programme components including assessing knowledge, attitudes, and behavior of teachers and parents.   The methodology should take into consideration is that the baseline should be a solid base for assessment of the impacts of the programme at its end.

Suggested quantitative data analysis

The quantitative analysis will include both descriptive analysis and advanced multivariate regression analysis. It will provide information on all pre-intervention indicators for the intervention/treatment and control groups.

Appropriate questionnaires developed using the literature review and the lessons learned from already implemented studies will be used to develop questionnaires, adjusted to Rwandan context. The quantitative analysis will include both descriptive analysis and advanced multivariate regression analysis. It will provide information on all pre-intervention indicators for the intervention/treatment and control groups (if the methodology includes control groups). In addition, the multivariate regression analysis may be used to test the soundness of the theory of change by identifying main determinants of pre-primary reading, at baseline level.

4.2 Sampling

The consultancy institution will develop the research sampling strategy and design the sample, in line with the proposed and accepted methodology.

The sample should be representative, while responding to certain limitations of the programme designs, in that the programme covers 16 out of 30 Districts in Rwanda and 80 schools surrounding the teacher training colleges (TTC).   But, the sample should be representative for the project taking into consideration the need to disaggregate data by gender and geography (urban/rural).

The overall methodology and sampling strategy needs to take into account how to differentiate between the impact of the various pre-primary interventions being rolled out (including those besides Itegure Gusoma) and include maturation effects on the children in the study.  In addition, the methodology and sampling strategy needs to consider a potential attrition rate. 

The Itegure Gusoma sampling strategy will based on random sampling of primary caregivers within selected sites.   The eligibility criteria will be that participants will have to be the primary caregiver and legal guardians of a child between the ages of 3 – 6 years of age who is enrolled in pre-primary school (N3), and live in the same household as the child.

4.3 Process

The research process and methodology will include three phases:

Phase 1. Inception:

• Research Plan development – draft work schedule to be submitted to the UNICEF for review and approval.
• In-depth desk review of available information on early reading, Itegure Gusoma programme documents, pre-primary data from MINEDUC, Education Sector Strategic Plan, and any other relevant information – to guide the development of the research questions. Review the current status of children in Rwanda with focus on pre-primary components and an integrated picture of pre-primary education particularly in the selected sites.
• Preliminary discussions with UNICEF Rwanda, VSO and the Pre-Primary Technical Working Group, to facilitate a common in-depth understanding of the conceptual framework, refining the research questions and adjusting data collection methods, tools and sources. The consultancy team will have the opportunity to discuss and propose amendments to the methodology, as long as the purpose of the study is maintained and expected deliverables are produced at the required level of quality. Any proposed changes will be discussed at planning meetings with the Task Force at the beginning of the process.
• Drafting of Inception report (deliverable 1), including the details of the methodology to be used, sampling approach, revised list of key indicators for qualitative and quantitative measurement, as well as an Research Matrix for each finally agreed research question and a detailed analysis plan, to be presented to and approved by the members of the Task Force. The proposed methodology needs to be sufficient to capture all the indicators agreed for this research purpose.

  Phase 2. Baseline Data collection:

• Develop data gathering tools. The battery of instruments are to be developed based on internationally recognized tools, such as the EGRA. Measures are to be selected that determine key areas of the Itegure Gusoma intervention and are intended for use as part of the end-line research to assess the impact of the Itegure Gusoma program.
• The development of data gathering tools, interview and observation guides using a consultative participatory process, will include the following steps:
  • Facilitate a meeting to share the tools for comments with the Task Force overseeing the consultancy.
  • Pre‐test the tools and facilitate a meeting with the Task Force, to validate the pre‐ testing data obtained and fine tune the tools based on the experiences from the team of interviewers/enumerators.
  • Train research assistants /enumerators for data collection in all the selected sites to ensure that standardised methodology and application of the tools are fully understood.
  • Conduct collaborative planning for field work with the Task force, including development of agenda and activities.
• Work with team of interviewers/observers to collect data using quantitative, qualitative observations and other appropriate methods (measurable indicators) from children, parents, teachers, head teachers, and community members:
  • In alignment with the agreed methodology, the consultant firm will collect quantitative data at intervention/treatment and control sites.
  • The consultant firm will conduct focus group discussions and in-depth interview with parents and community members in the baseline intervention and control groups
• Convene a briefing meeting with the supervising task force on the on‐going survey for corrective measures to be undertaken if need be.
• Ensure the quality of information collected from fields, cross check with the validity and reliability of information collected and verify
• The consultant firm will submit regular progress report and field reports throughout data collection period and provide a final report on the baseline data. (deliverable 2)

 

Phase 3. Analysis and reporting phase:

• Process data (data entry, cleaning, and analysis) using the SPSS/STATA/ EpiInfo or any acceptable statistical data analysis package for data gathering and analysis.
• Following the data collection and analysis phase, the evaluation team will make a presentation of the preliminary key findings (First and preliminary findings report - deliverable 3) to the Task Force.
• A detailed draft technical report will be shared with key partners for review, and once all feedback of Task force members have been addressed, the consultant firm will share a final report (deliverable 4) for review and validation by the Pre-primary Evaluation Task force.
• A concise (approximately 40 pages) summary report (deliverable 5) of the key findings of the validated technical report drafted and shared with the Task force
• All outputs from the research are subjection to approval by the Task force.

5 Major Tasks, Deliverables & Timeframe

 

Tasks

Expected Deliverables

Timeframe

1. Desk review of available information on early reading and pre-primary education in Rwanda  

Feedback meeting on findings from desk review

June 2017

2. Design of the sampling strategy, data collection and relative tools and preparation of inception report.

3. Application for all relevant research permits for Rwanda

Inception report (deliverable 1) including work plan, methodological approach, instruments to be used, interview and field visit protocols, annotated outline of final report, to be presented and approved by the Task force.

July 2017

(1st payment, 20%)

4. Data collection, cleaning and analysing quantitative data for the baseline

Quantitative data analysis progress report and end of field data collection report / baseline final report (deliverable 2)

September 2017

(2nd payment, 20%)

5. Analysis of findings and draft report preparation

Preliminary findings report and presentation of key preliminary findings to the Task force (deliverable 3)

November 2017

(3rd payment, 20%)

6. Validation of technical report / Incorporate comments from  Task force and finalize report

Draft technical report[7] shared with Task Force for review and validation (deliverable 4).

Meeting report confirming the validation of findings, including how the feedback/recommendations from the committee have been thoroughly addressed.

December 2017

7. Draft a concise, user-friendly for non-technical audience, summary report of key finding from technical report

Summary report (deliverable 5)

January 2018

(4th payment, 40%)

 

The final report should be in line with the UNICEF research standard and very focused on practical and implementable recommendations. Specifically, the report should include at least the following sections: executive summary, description of the research methodology (as per agreed inception report), assessment of the methodology (including limitations), findings, analysis, conclusions, lessons learned and recommendations for improvement. The Annexes to the report should contain: the TOR, the approved data collection instruments, and any other relevant information.

 

The final reports for the baseline research should be in line with agreed Government and UNICEF research standards and very focused on practical and implementable recommendations. The report template should include, but not limited to:

• Title page and opening pages
• Executive summary
• Programme description
• Role of UNICEF, VSO and other stakeholders in programme implementation
• Purpose of Research
• Objectives
• Research design
• Methodology, including sampling strategy and methodological limitations, and research criteria
• Stakeholder participation
• Ethical issues
• Major findings
• Analysis of results
• Key Constraints
• General Conclusions
• Recommendations
• Lessons learned
• Annexes TOR, tools of data collection used

 

The report should be provided in both hard copy and electronic version in English. Complete data sets (tools, database, filled out questionnaires, records of interviews and focus group discussions etc.) should also be provided to UNICEF at the end of the research, and will remain the property of UNICEF.  Any use of datasets and publications emanating from the research is subject to prior approval of UNICEF Rwanda.

Dissemination of Results: Findings of the research will be summarized and discussed with the members of the Pre-Primary Technical Working Group. Findings will also be made available to relevant stakeholders, in consultation with the Technical Working Group.

6 Stakeholder Participation

Key stakeholders include the members of the UNICEF, VSO, USAID, REB and MINEDUC focal persons, as well as the pre-primary technical working group.  USAID is the primary donor to the programme, with UNICEF as fund recipient.   UNICEF is providing technical and financial oversight to the entire programme.  VSO is one of the key implementing partners, providing technical and financial support to the programme.  And, the Itegure Gusoma is being developed and implemented in partnership with MINEDUC and REB.

The stakeholders will:

• Plan and design the research through consultation with the main parties involved and final approval of research’s terms of reference;
• Provide technical inputs to the design of the research;
• Provide guidelines to evaluators and monitor the research implementation;
• Review the evaluators’ inception report (including proposals for desk review of documents, research instruments, country visits, annotated outline of the report);
• Review preliminary findings for validation of facts and analyses, and help generate recommendations
• Approve the preliminary report;
• Review and approve the final report, verify the evaluators’ findings and propose management response;
• Ensure that the research findings are used for future planning and pre-primary programmatic interventions as well as advocacy purposes.

The variety of stakeholders will ensure that different opinions are represented and objectivity is achieved.

UNICEF Rwanda will be responsible for selection of the institution to conduct the research.  The research will be managed by UNICEF. The management of the research will involve drafting the terms of reference, initiating research selection process, liaison between the research team and other key stakeholders, as well as quality assurance of the report.

The consultancy institution will be responsible for organizing field visits, logistical support related to data collection, and organizing meeting with different stakeholders, including convening meetings of the Pre-Primary Technical Working Group and the Basic Education Working Group.  UNICEF Rwanda will be responsible for providing relevant information at country level, providing access to relevant reports/statistics, and providing inputs for methodology and for data analysis.

7 Existing information sources

The following information sources are available:

• Ministry of Education. (2016 October). 2016 Education Statistical Yearbook.
• Education Development Center. (2016 January). USAID/L3: National Fluency and Mathematics Assessment Midline Report. Available at http://pdf.usaid.gov/pdf_docs/pa00kzs9.pdf
• Education Development Center. (2017 January). USAID/L3: National Fluency and Mathematics Assessment End-line Report. Available at http://pdf.usaid.gov/pdf_docs/pa00kzs9.pdfEducation Development Center. (2015 March). Innovation for Education: Final Outcome Report. Available at http://idd.edc.org/resources/publications/innovation-education-final-outcome-report-march-2015.
• Save the Children. (2013 August). Emergent Literacy: Investing Early for Exponential Outcomes. Available at https://rwanda.savethechildren.net/resource-centre
• Save the Children. (2015 June). Early Literacy & Maths Initiative (ELMI) Rwanda End-line Report. Available at https://rwanda.savethechildren.net/resource-centre
• Ministry of Education. (2016 June). 2015 Education Statistical Yearbook. Available at http://www.mineduc.gov.rw/resource/statistics/
• Ministry of Education. Education Sector Strategic Plan (2013/14 – 2017/18). Available at http://www.mineduc.gov.rw/resources/policies/.

8 Ethical Consideration/confidentiality

Adequate measures should be taken to ensure that the process responds to quality and ethical requirements as per UNICEF Research Standards. As per United Nations Research Group (UNEG) Standard and Norms, and in particular adhering to Rwandan standards and norms, the consultants should be sensitive to beliefs, manners and customs and act with integrity and honesty in relationships with all stakeholders. Furthermore, consultants should protect the anonymity and confidentiality of individual information. Consultants should respect the confidentiality of the information which is being handled during the assignment. Consultants are allowed to use documents and information provided only for the tasks related to the terms of reference of this research. Data will be stored in a secure location, kept confidential with access restricted to principal investigators. The study data will be used only for the purpose of this study.

All research will be undertaken with the appropriate clearances within Rwanda, including methodology approval from the National Institute for Statistics (NISR) and ethics approval from the Rwandan National Ethics Committee (RNEC), with support from the Ministry of Education.    

9 Research team composition / qualifications and requirements

The selected research institution will be responsible for the creation of a research team. The minimum request is that the team consists of at least two experts (one expert in quantitative research and impact research, and a further expert team members for qualitative research). The team composition should include national (Rwandan) experts. The exact division of work will be decided by the institution, but in general, the team leader will be responsible for discussions, negotiations, final decisions, shape of the research, while further team members will be tasked with more technical issues (revision of technical reports, in-depth interviews with service providers, decision makers, parents, revision of existing research reports etc.).

The team will preferably include the following profiles: technical expert and team leader; qualitative research expert; Statistician ‐Data entry and analysis staff; Data collection Assistants.

The qualifications and skill areas required include:

Technical expert & team leader:

• Extensive quantitative research and impact research expertise and experience, including expertise in data collection and analysis; demonstrated skills in similar research; demonstrated technical report writing skills
• Demonstrated experience and expertise in designing and implementing multi‐sectoral initiatives in partnership with a wide range of stakeholders including government and communities

• Minimum five years of relevant work experience of which two at national and international levels in field programmes relevant to Education.

• Understanding of technical aspects of early reading and pre-primary education.

• Advanced university degree in one or more of the disciplines relevant to the following areas: Research expertise, economics and social sciences.
• Knowledgeable on institutional issues related to the provision of global public goods;
• Experience working with/in the UN or other international development organizations in the social sector or in national level development assistance and partnership support to government programmes and priorities is an asset.
• Fluency in English a must and knowledge of French and Kinyarwanda an advantage.

Qualitative research expert:

• Extensive qualitative research expertise and experience, including data collection skills; demonstrated skills in similar research
• Knowledge of technical aspects of similar programmes
• Knowledge of the areas of intervention

All members of the team:

• Language proficiency: excellent writing skills in English;
• Advanced university degree in related field or social science;
• Work experience in different countries globally: at least 8 years of field experience for team leader and research expert; at least 3 years of field experience for all other team members. Experience in working with UN agencies (desired);
• Experience in research: knowledgeable on UN research policy, recommended by UNICEF regional or global research advisors or other senior managers, skilled in performing structured interviews and facilitating focus group discussions;
• Analytical skills: Demonstrated analytical skills related to the use of quantitative and qualitative data for decision-making;
• Process management skills: Demonstrated skills and experience in conducting and presenting researchs;

• Good communication and advocacy skills: Ability to communicate with various stakeholders, and to express ideas and concepts concisely and clearly in written and oral form;

Evaluators should be sensitive to beliefs and act with integrity and respect to all stakeholders. Evaluators should protect the anonymity and confidentiality of individual interviewees.

Research and selection criteria of the consultancy institution:

A two stage procedure shall be utilized in evaluating proposals, with research of the technical proposal being completed prior to any financial proposal being compared. A 70/30 assessment model for the technical and financial proposal respectively will be adapted. Cumulative weighted average methodology will then apply in determining the best value for money proposal.

Applications shall therefore contain the following required documentation:

a. Technical Proposal: Consultant institution should prepare a proposal on the basis of the tasks and deliverables (as per the ToR). The proposal should include:

1. Description of past experience, reflecting why the institution is suited to fulfil this scope of work. 
2. The Technical Proposal shall also include updated CVs, including their intended role in this consultancy and the days available.  Copies of 2 reports of previous research conducted by the consultants should also be included.
3. Approach and methodology with detailed breakdown of inception phase, proposed scope and data collection methodology and approach that will be used by the consultant.   Moreover, a brief explanation of the data analysis and report writing and possible dissemination plan should be included along with a draft work plan and timeline for the research.

b. Financial Proposal: Expected financial offer with cost breakdown of consultancy fee, travel, daily subsistence allowance (DSA), and any other costs, during the field work in Rwanda.  The proposal should include a breakdown of prices, for each component of the proposed work, based on an estimate of the time needed.  The financial proposal shall be submitted in a separate file, clearly named financial proposal. No financial information should be contained in the technical proposal as this will lead to proposal cancellation. Financial Proposals should be filled as per table below:

 

For each Deliverable provide :

Number of person  days

Delivery date

Costs

 

Deliverable:

Inception report (Deliverable 1)

End of data collection report - baseline (Deliverable 2)

Preliminary findings report (Deliverable 3)

Final report and summary report (Deliverable 4 and 5)

Operational Costs (a detailed addendum budget required)

Total

 

The evaluation of bids will be based on the following:

Technical Proposal

Points

1.  Experience of the institution (Points 30)

The successful consultancy firm will be able to demonstrate skills in:

• Experience in evaluating education programmes;
• International exposure to developing countries;
• Prior work with government and non-government partners;
• Excellent qualitative and quantitative research analytical skills.
• Past experience and/or capacity to work in Rwanda (preference given to Rwandan consultancy firms or firms bidding with national experts)
• Demonstrate experience in convening multi-stakeholder processes and building consensus on research.

 

Team members should demonstrate the following:

• At least 10 years, combined, experience on research and analysis with quantitative and qualitative data.
• At least 10 years, combined, experience in Education. 
• Fluency in English, and Kinyarwanda (preferred). 
• Excellent writing skills.
• Demonstrated experience in developing reports.

1. Proposed Methodology and Approach (Points 30)

• Quality of previous work done in research, and education research, specifically.
• Quality of proposed methodology and approach (including sampling framework).
• Quality of the proposed process and timeline towards the final report. 

1. Technical capacity for Management (Points 10)

• Realistic and timed plan of action;
• Provide ability to conceptualize, plan, and execute research.
• Good management, interpersonal, planning, and coordination skills.
• Experience working in Sub-Saharan African, with preference for experience in Rwanda.

Financial Proposal

(Points 30)

• The price should be broken down for each component of the proposed work, based on an estimate of time taken.
• Separate cost table.
• Budget narrative to explain the assumptions behind all cost estimates.

 

10 Supervision

The research will be supervised by UNICEF Social Policy and Research Section and UNICEF’s Chief of Education and Education Officer. 

The key stakeholders, named above, will provide technical inputs to the design of the research, provide guidance to the evaluators, and monitor the research implementation process.

11 Terms and conditions

Procedures and logistics

Evaluators are expected to use their own hired vehicles, equipment, including computers. Consultant institution is required to ensure translation to Kinyarwanda during field trips as well as translation of documents from Kinyarwanda to English if required.  UNICEF will be under no operational obligation to pay for operational costs related to this consultancy, all costs required to operationalise this consultancy shall be borne by the hired institutional firm and should be included into the proposed financial proposal.

 

Terms of payment

The payment will be in four (4) instalments as follows:

• 20% of the total payment upon completion of the desk review, submission of inception report with work plan and methodology, theory of change and research instruments and protocols.
• 20% of the total payment upon completion quantitative and qualitative data collection and analysis, including field visits for the baseline research;
• 20% upon the submission of the draft final report and PPT of the research;
• The remaining 40% will be paid upon completion of all deliverables, as per the above schedule (validated final report of the research; a set of Power Point slides (25-30 slides) with key salient features of the research; a summary report.

All the deliverables need to meet UNICEF requirement and quality standards. Payment will only made for work satisfactorily completed and accepted by UNICEF. UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if work/outputs is incomplete, not delivered or for failure to meet deadlines.

All materials developed by the firm will remain the copyright of UNICEF, who will be free to adapt and modify the materials for future use.

Other Considerations

Standard UN clauses including, but not limited to, liability, indemnification, and copyrighting will be included into the contract. 

12 How to apply

Qualified institutions are requested to submit a full proposal, consisting of two separate parts (technical and financial), to Rwasupply@unicef.org.

 

All submissions must be received by 23 June 2017.   Queries should be submitted to smcginty@unicef.org and will only be accepted until 16 June 2017.  

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Prepared by:                                                                                                      Reviewed by:

               

Name: Sara McGinty                                                                                      Name: Erna Ribar                            

UNICEF Chief of Education                                                                           UNICEF Chief of Social Policy and Research

 

 

Signature: ______________                                                                     Signature: _____________

 

 

Date: _____/____/____                                                                             Date: _____/____/____

 

 

Approved by:

                                                                                               

Name: Oliver Petrovic

UNICEF Deputy Representative

 

Signature: _____________                                                                       

 

 

Date: _____/____/____