LRPS-2019-9150491 to conduct Evaluation of UNICEF’s work for Accelerating Action towards Ending Child Marriage in Bangladesh

TERMS OF REFERENCE FOR INSTITUTIONAL CONTRACT

 

Title of the assignment

Evaluation of UNICEF’s work for Accelerating Action towards Ending Child Marriage in Bangladesh

Purpose

The overall objective of the evaluation is to assess the progress made by BCO towards and achievement of results of its ECM programmes. The evaluation will assess the relevance of BCO’s work on ECM,  effectiveness and added value of the joint UNFPA UNICEF GPECM and will also assess BCO’s ECM interventions outside the GPECM. It will identify opportunities and barriers to the implementation of cross sectoral evidence-based intervention strategies. The evaluation will assess the efficiency of UNICEF’s ECM work, its relevance, effectiveness, impact and sustainability.  It will also identify good practices of convergence among sectors and lessons especially from the Phase I of the GPECM 

Location

Bangladesh: Dhaka and programme areas

Estimated Duration

6 months (September 2019-February 2020)

Reporting to Technical Supervisor of this assignment

Research and Evaluation Specialist and Gender Specialist

 

1. Background and rationale:

 

With a prevalence rate of 52.3% of women between the age of 20-24 married before their 18th birthday, and 18% marrying before 15 years married (MICS 2013), Bangladesh has one of the highest rates of child marriage in the world. While the practice of child marriage has slowly decreased in Bangladesh over the last 30 years, it remains unacceptably high nationally, with more married children in rural areas and urban slums, and particularly among poor and less educated girls.

 

Ending child marriage is a priority for both the Government and its development partners, who recognize the need to preserve childhoods, secure children’s rights to education and life, reduce their exposure to violence and exploitation and contribute to ending intergenerational poverty. In 2014, UNICEF and its partners’ support to programmes aiming at harnessing adolescents’ energy and skills to improve their lives while supporting the country’s development objectives resulted in the Prime Minister’s commitment to accelerate the eradication of child marriage in Bangladesh by 2035. The Ministry of Women and Children’s Affairs, is the GoB’s selected line ministry for coordinating all government response on ending child marriage.

 

BCO initially received funding in 2015, based on a multi-sectoral proposal, which included building on existing initiatives across different sectors of work and in some cases retrofitting to include child marriage in existing work.  In 2016 with a new CPD there was an opportunity to sharpen the approach on ECM and strengthen the links between adolescent programming and gender. In addition, BCO’s 2017-2020 Country Programme adopted a life-cycle approach to programming, with a standalone outcome on adolescents and under this outcome ECM was placed as a high-level result in the CPD Output 3.7: “By 2020, adolescent boys and girls from selected communities are engaged to develop capacities as agents of change and facilitate action to eliminate harmful social norms and practices with a focus on ECM”. The indicator for this output is as follows: “Targeted parents/primary caregivers reached by programmes addressing child-rearing practices; Number and proportion of adolescents reached through Life Skills Education (LSE) packages on gender, sexual harassment and early marriage nationally that will enable them to influence development agendas”. The CO also developed a Strategy Note on ECM and has in place a BCO ECM proposal from March 2016 which was used as a basis for the GPECM when Bangladesh became one of three GPECM countries in ROSA for Phase I of the UNFPA UNICEF Global Programme to Accelerate Action to End Child Marriage from 2016-2019. It is important to note that a multi-stakeholder meeting happened in December 2014 developed an extensive analysis of the situation of child marriage in Bangladesh and developed a theory of change for the work around ECM in the country, which was in turn the basis of the BCO strategy note for ECM. This TOC, along with the TOC of the GPECM, is presented as an annex to these TOR. Therefore, the key response to ECM has been through the GPECM which offered a framework promoting the right of girls to delay marriage, addressing the conditions that keep the practice in place, and caring for girls already in union.

 

There is no national Theory of Change but GoB has utilized elements from three sets of documents that were developed by Plan/ICRW/UNFPA, the Girls Not Brides Alliance and Maxwell Stamp (study commissioned by UNICEF) and the main findings from these documents along with the ToC framework have provided the basis for consultations and formulation of the National Action Plan to End Child Marriage (2016-2020). Moreover, UNICEF is planning to hold a multi-stakeholder meeting late June 2019 to validate the existing problem tree and ToC for ECM. It is important to note that the strategies identified by BCO correspond well with the strategies of the National Action Plan.

 

At the onset of the GPECM, BCO pulled together all interventions aimed at ECM and that were aligned with the global strategies under the GPECM TOC and brought them under the umbrella of the joint programme with UNFPA. These interventions were then funded through the GPECM funds that UNICEF received for Phase I. There is a cross sectoral task force on ECM coordinated by the Gender Specialist with focal points from all sections, who are also part of the more general group that works on adolescents.

 

This initial phase had an indicative four-year budget (2016-2019) estimated at $246.7 million with contributions from Canada, the Netherlands, Italy, the United Kingdom and the European Commission. Close to 10.5 million dollars have been received to date. The first phase of implementation is aimed at targeting adolescent girls (ages 10-19) at risk of child marriage or already in union, particularly adolescent girls.

 

The prevalence of child marriage varies across districts in Bangladesh. At subnational levels, the north and western parts of the country show higher child marriage prevalence than other parts of the country (MICS 2013). An overall objective of the programme is to accelerate action to address child marriage by enhancing investments in and policies and frameworks promoting positive change and improving the data and evidence base.

 

In Bangladesh, an ‘Area-Based Programming Approach’ was adopted for efforts to end child marriage in the country, using high prevalence rates of child marriage from MICS 2013 as the key variable for determining the area selection. The following areas were included in the GPECM according to the global programme proposal.

 

Division

District

Upazilla

In GPECM?

Mymensingh

Jamalpur

Islampur and Dewanganj;

Yes

Netrokona

Kolmakanda and Khaliajhuri

Yes

Sherpur

Sreebordi ad Nalitabari

Yes

Mymensingh

Sadar

No

Dhaka

Gazipur

City Corporation

No

Barisal

Bhola

Char Fassion, Lal Mohon, Monpura, Sadar

Yes

Patuakhali

Kalapara,

Yes

Barguna

Patharghata

Yes

Barisal

City Corporation

No

Khulna

Khulna

Dacope and Koyra

Yes

Kustia             

Daulatpur and Bheramara

Yes

Bagerhat

Shoronkhola, Morolgonj

No

Khulna

City corporation, Rupsha, Terokhada, Digholia, Phultala, Botiaghata, Paikgacha

No

Rangpur

Nilphamari

Dimla, Domar and Kishorganj

Yes

Kurigram

Sadar, Char Rajibpur

No

Gaibandha

Sadar, Fulchuri

No

Sirajgonj

Belkuchi, Shahjadpur

No

Chapai Nawabgonj

Sadar, Shibgonj

No

 

 

The ECM Strategy note mentions a combination of strategies that are being used to address both demand and supply side constraints in order to leverage results in accelerating an end to child marriage.

1. A key strategy is the empowerment of adolescent girls with interventions aimed at mobilization of out-of-school adolescents (especially girls) into adolescent clubs, alongside those who are in school, with the aim of providing them with leadership, life skills development and peer education training with a focus on prevention of child marriage and adolescent development issues (health, WASH, HIV/AIDS, nutrition).
2. Another strategy is Community Engagement and Behaviour Change with interventions aimed at community mobilisation, dialogue, mobilising and engaging families and communities to challenge prevailing beliefs and attitudes, build ownership and facilitate collective action to ensuring girls are not married before 18 and attend school etc. It also includes social reinforcement of abandonment of these norms and uptake of new attitudes and actions through mobilisation and engagement of key community level influencers such as religious and local leaders, as well as service providers to achieve a ‘critical and irrevocable’ mass of population that have changed their beliefs and norms.
3. Institutional strengthening, advocacy & law enforcement is another strategy that involves interventions around advocacy, evidence building and technical support to policy and legal reform. Under this strategy UNICEF and UNFPA have worked together on advocacy regarding the Child Marriage Restraint Act (CM RA), and support to the development of the National Action Plan (NAP) on ECM and evidence generation. 
4. Another important strategy is enhancing the accessibility and quality of formal schooling for girls which includes integrating life skills in the curriculum, teacher training, MHM, vocational training component with ILO targeting technical TVET and apprenticeship, job placement and job matching mechanisms, etc. 
5. Lastly, provision of livelihood support & financial Incentives is a strategy aimed at advocacy and technical support to the GoB, at national and local levels to develop or reinforce appropriate forms of financial incentives that would encourage families to keep their adolescent girls in schools and also perceive a value and relevance of education and formal/non-formal vocational trainings that are linked to livelihoods.

 

So far, the reach of ECM activities has been: More than 800 dropped out adolescents (640 girls and 160 boys) went back to school after getting stipends to run small income generating activities. More than 10,000 community people comprising parents, lawyer, teacher representatives, woman ward / union members, representatives from Child Led Organizations (CLOs) gained knowledge on referral mechanism during emergency and any kind of violence including child marriage. Around 150 child marriages were stopped through the support of CBCPCs in the ECM districts during the reporting period. (Ref: GPECM Report 2018, page 4). However, there are no current models or estimations in place for the number of marriages averted as a result of all strategies and interventions that are part of the programme. How to best do this will be part of the questions asked as evaluation questions later in these ToR.

 

BCO’s work on ECM also includes generating evidence that supports the strategies mentioned above. So far, the following evidence products have been produced, and will be shared with the winning bidder.

• The work education is leading for keeping girls in schools is accompanied by an impact evaluation design (RCT) and an operations research. The baseline data collection has already finished, and a report will be delivered in May.
• The ECM mass media campaign and edu-drama is accompanied by a baseline, midline and endline data collection that aims to understand impact. Baseline and midline reports will be made available.
• The adolescent health interventions are accompanied by baseline studies, as well as ongoing operations research on adolescent friendly health services. The baseline report is finished.
• Exploratory work on social norms was undertaken based on a nationwide survey. A discussion paper titled What Matters Most for Ending Child Marriage was developed, and will be made available to the public soon.
• MICS 2019 data on child marriage prevalence will be available in August 2019.
• A budgetary scoping analysis on expenditures on child marriage was also undertaken and can be found in the public domain.

 

Within the current CP ECM related work is going on in geographic areas that are not part of the GPECM. In terms of the GPECM, the Evaluation Offices of UNICEF and UNFPA commissioned an evaluability assessment of the GP which was conducted between April and September 2017 and covered the GPECM from its launch in January 2016 to August 2017.  A joint evaluation of this joint UNICEF UNFPA GPECM has been finalized in end 2018. It took stock and captured the changes the programme has gone through in this learning phase, to test the Global Programme theory of change, to assess progress made in different country contexts, support learning among key stakeholders to inform current implementation, and feed into the potential next phase of the Global Programme. The results of this evaluation along with the Evaluability Assessment are key documents that are feeding into the development of the Phase 2 of the GPECM including a revised ToC.

 

The global evaluation of the GPECM did not go in-depth into the UNICEF Bangladesh’s ECM related work and especially our overall work that goes beyond the GPECM. Therefore, this evaluation will take a deeper look into how the GPECM is working in Bangladesh, beyond the general findings from the global evaluation that are applicable to all GPECM countries. This evaluation will also look at BCO’s work beyond the GPECM in terms of successes and challenges, coherence and complementarities with the GPECM. In addition, it will also assess to what extent being part of the global programme has supported BCO’s efforts and to what extent has the joint programme achieved joint results.  Finally, aspects of efficiency, value for money will also be explored.

 

One further point to note is the meaning of sustainability for the ECM programme and thus for the evaluation. As ending child marriage will largely depend on a shift of social norms, sustainability of the results will largely depend on the sustainability of the change in the norms. This includes reaching a critical mass of people who oppose child marriage, as well as ensuring a supportive environment for girls to choose their own destiny. In this scenario, sustainability must go beyond sustainability of resources to keep the programme running, or sustainability of resources to ensure that services continue to be available to girls even if UNICEF funding declines over time. This needs to be in the background of the sustainability questions of the ECM programme evaluation.

 

The main users of the evaluation will be UNICEF, UNFPA and UNWOMEN as part of the GPECM, the government of Bangladesh and in the Prime Minister’s office and the Ministry of Women and Children Affairs including all Development partners and CSO working on ECM .. These will benefit from the evaluation in terms of lessons learnt and for managerial decisions, namely informing the next phase of the GPECM, and the joint UNICEF-GoB Country Programme Document 2021-2025. The evaluation will also be used by the donors who support the work being done for accelerating an end to child marriage, serving a purpose of accountability.

 

1. Purpose/Objective of the assignment:

 

The overall objective of the evaluation is to assess the progress made by BCO towards and achievement of results of its ECM programmes. The evaluation will assess the relevance of BCO’s work on ECM,  effectiveness and added value of the joint UNFPA UNICEF GPECM and will also assess BCO’s ECM interventions outside the GPECM funded through OR of different sections. It will identify opportunities and barriers to the implementation of cross sectoral evidence-based intervention strategies. The evaluation will assess the efficiency of UNICEF’s ECM work, with a focus on understanding. It will also identify good practices of convergence among sectors and lessons especially from the Phase I of the GPECM.

 

1. Specific objective:

 

1. Assess results against the strategies outlined in the Strategy Note on ECM and the results framework and workplan of the GPECM.
2. Assess cross sectoral convergence and collaboration and if the comparative advantage of each sector was leveraged.
3. Assess to what extend scale and sustainability have been built into the interventions.
4. Assess the extent to which equity and gender equality have been addressed in programme implementation.
5. Assess whether the programme was gender transformative in its design and whether it has led to gender transformative results
6. Assess how UNICEF BCO used it standing with the GoB to push the ECM agenda and how it leveraged the LCG WAGE platform for national level advocacy.
7. Assess how BCO built and promoted partnerships with all stakeholders especially the women’s organizations and civil society.
8. Assess the extent to which policies and laws at the national level have been translated and used at the district and upazilla level.
9. Calculate the cost-effectiveness of the programme
10. Develop a methodology that uses existing information to estimate the number of child marriages that are averted and prevented

 

1.  Scope of the evaluation

 

Temporal coverage: The evaluation will cover the ECM related work covered under the current Country Programme (2016-2020), from its onset until the time of the conduct of the evaluation (Q3 2019).

 

Geographical scope: The evaluation will cover the upstream work undertaken at the national level and the downstream work that happens in the ECM districts, either through GPECM interventions or other ECM activities. 

 

Content: The evaluation will cover all activities planned and/or implemented during the period under evaluation. The evaluation will focus primarily on the progress towards achieving outputs and contribution to outcomes in the Theory of Change and results frameworks presented. Moreover, the evaluation will focus on both streams of ECM work currently ongoing in the country, namely the work happening under the GPECM and that happening outside of the GPECM. When evaluating the GPECM in Bangladesh, we expect the evaluators  to go beyond the findings of the Global Evaluation of the GPECM and to provide a deeper and more contextualized understanding of the situation in Bangladesh. The unit of analysis will be the programme as a whole, as opposed to the discrete interventions, as most of these have strong evidence generation activities. However, an area of interest of the evaluation is understanding spillover effects from programme districts to non-programme districts.

 

Stakeholders: Different groups of respondents will need to be taken into consideration for the evaluation both at the upstream and downstream level. In addition to UNICEF staff both in Dhaka and the field, the socio-ecological model that is at the heart of the ECM programme should also guide the types of respondents that need to be reached out to throughout the evaluation. Government officials directly involved with the ECM programme from the Prime Minister’s Office, MowCA and other Ministries will need to be included. Other UN agencies and in particular UNFPA, as well as donors and implementing partners will need to be reached out to. Community members, both those who are part of organized committees that support children and their rights, as well as “unorganized” community members need to be included. Families with children (adolescents and younger), both male and female should be included,  Adolescent boys and girls, including married girls, both those who attend adolescent clubs or similar and those that do not attend such groups should be included in data collection efforts. An important note is that although the programme focuses on ending child marriage involving girls, men and boys are also key stakeholders in marriage related decisions, and thus they need to be also involved as key respondents in the evaluation. Understanding how to better involve boys and men into programming is at the core of the learnings expected.

 

2.3 Evaluation questions

 

The evaluation will be informed by the OECD DAC criteria of relevance, effectiveness, efficiency, impact and sustainability.

 

The preliminary evaluation questions are articulated around the key objectives and scope of the exercise and further focus the evaluation criteria. Once the evaluation team acquires a clear understanding of the logic and rationale of the programme, as well as the extent of implementation of the Programme, the team will further refine the evaluation questions as well as develop the evaluation matrix for this exercise during the inception phase, detailing all evaluation questions, assumptions to be assessed, indicators, and sources of information. The potential usefulness as well as feasibility of each proposed evaluation question will be assessed in close collaboration with the ERG with a view to determining the final set of questions. The bidding team can suggest changes to the evaluation questions in the proposal, as long as it clearly mentions how the original question is being captured and the reasoning behind the changes.

 

Relevance and responsiveness:

1. To what extent is the programme relevant and responsive to contribute toward ending child marriage in Bangladesh? To what extent has the programme been designed in a way that is responsive to different groups in the society, their own contexts and needs? (Consider different contexts to define these groups such as urban and rural, living in high-child marriage districts as defined in the Country Programme,  in-school and out of school, unmarried and already married girls, wealth quintile, gender and ability. Also consider different decision makers and influencers on marriage decisions, such as fathers, mullahs, mothers in law, male and female community leaders, etc.).
2. To what extent do the programme strategies and activities address structural drivers of (gender) inequality?
3. To what extent is the ToC that guides the ECM work in Bangladesh well developed, aligned with the Bangladesh context, based on robust evidence and realistic in its assumptions and pathways of change? How can it be improved for the next country programme and second phase of the GPECM?
4. How well aligned is UNICEF ECM programme with the National Plan of Action strategies and pillars? Are there clear strategies of the NPA that UNICEF should focus on during the next ECM programme given UNICEF’s expertise?
5. To what extent has the programme been implemented following a human rights, equity and culturally sensitive approach?

 

Effectiveness and impact

1. To what extent has the ECM programme achieved or is on track to achieving its planned results in all its 5 strategies? Have the combination of these strategies been able to prevent and stop child marriages in programme areas? Has there been evidence of spillover effects of the programme into non-programme areas? If so, is there evidence of what is the main change factor that is being spilled over?
2. Considering the 5 strategies and specifically answer for each the following questions. When answering the questions please keep in mind and explore the following considerations: What have been supporting and constraining factors? Were there any unexpected results (positive or negative)?
   1. (S1) Are there signs that adolescent girls who have been part of the programme are more empowered and have increased skills? Have the interventions built the agency of girls as influencers and decision makers, especially on marriage decisions? If so, has this led to a decrease in child marriage? Has this led to an overall increase of adolescent wellbeing?[1]
   2. (S1) Is there evidence of spillover effects from adolescents who have been part of the programme to other adolescents in the community who have not been part of the interventions?
   3. (S1)  Have the interventions reached the most vulnerable? (out of school adolescents, those from lowest quintiles, adolescents with disabilities, already married girls)?
   4. (S2) Have prevailing mindsets, beliefs and attitudes towards child marriage and gender norms changed in the communities and among the key community level influencers described in the strategy? (please explore different community and family members, with a special focus on men, fathers, mullahs, community leaders, mothers in law, and other identified key decision makers and influencers on marriage decisions)
   5. (S2)How effective has work on multiple platforms and different levels (national, district, upazilla) been on achieving social mobilization and community engagement? Have families, community members and key community level influencers taken concrete action to protect adolescents from child marriage and to prevent and stop marriages? Have actions varied across these key stakeholders? Has saying “no” to child marriage become the common norm in the programme communities?
   6. (S3) How effective have UNICEF and UNFPA been in advocating for a normative and legislative  framework that protects adolescent girls from child marriage? Where there any missed opportunities in the advocacy strategy that need to be addressed? Has the process been timely? Has the legislative and normative framework trickled down into specific policies and programmes?
   7. (S3) What role has evidence played in UNICEF’s advocacy strategy? Are there any evidence gaps that could strengthen UNICEF’s capacity to strengthen institutions and legal and normative frameworks around child marriage? Does the existing evidence provide enough information on the different needs of adolescents depending on their various contexts (wealth quintile, location, family context, level of ability, schooling situation, etc.)?
   8. (S4) How effective has UNICEF been in enhancing accessibility and quality of formal schooling and alternative learning pathways for adolescent girls? Has a focus on girls had a negative impact on boys? Have adolescent girls who have benefited from this strategy more able to access the job market and to make decisions about their lives, including marriage? What external factors have influenced the effectiveness of this strategy?
   9. (S5) How effective has UNICEF been in supporting the government to provide financial support to families and adolescent girls at risk of child marriage so that they remain in school and unmarried? Has the government expanded its financial support and social protection schemes as a response of UNICEF’s advocacy and technical support? Has UNICEF been effective in changing the perception of families around the value of the girl child and the relevance of education for adolescent girls?
3. Has the programme been gender transformative and achieved gender transformative results?

 

Efficiency

1. Relating to the GPECM portion of work, has the joint programme and management structure with UNFPA at the country level facilitated achieving results and efficient delivery? Which areas have gained the most from this joint programming and which ones the least?
2. Relating to the GPECM portion of work, to what extent has being part of a global programme facilitate linkages and synergies to accelerate efforts to end child marriage within UNICEF and with its partners?
3. What key principles have determined where UNICEF has mainly spent its resources for ending child marriage? Have high impact areas been prioritised?
4. Has the country programme structure of a life cycle approach facilitated efficient delivery of results?
5. What has been the cost-effectiveness of the programme in terms of investments to end child marriage and the (approximate) number of marriages averted or stopped? How does this compare to other proven interventions to stop child marriage in Bangladesh and even in the South Asia region?

 

Sustainability

1. To what extent are there sustainability considerations built-into programme implementation?
2. To what extent has the programme generated/is in the process of generating robust evidence that supports the scaling up of successful interventions to allow for the government to take these over in the future?
3. To what extent has the programme been able to generate an enabling environment and strong involvement of key stakeholders at different levels (individual, family, community and local authorities) that will prevent gains in social norms to go back to pre-programme levels even if the absence of further interventions? Has the programme reached a tipping point where social norms and gender transformative results are strongly rooted in all these levels?

1. Indicative assignment dates: September 2019-February 2020

1. Duty station: Overall duty station can be the base country of the evaluation team. However, in case the winning firm is not based in Bangladesh, the team leader and key team members are expected to be in Bangladesh for the key moments of the evaluation and to ensure enough time spent in the country for understanding the programme and for ensuring high quality of the evaluation products. Bidding firms need to describe the optimal balance of work inside and outside of Bangladesh in the bid. It is expected that the team leader leads key presentations, even when done through Skype.  As the evaluation is taking place in Bangladesh, meetings, skype calls and other interactions are expected to take place within Bangladesh working week working hours (Sunday-Thursday), 8:00-16:30 BST.

1. Methodology:

 

The ECM programme evaluation will need to be conducted using a mixed methods approach that reaches out and covers various sources of both quantitative and qualitative data. The bidders will need to show in the proposal outstanding command of different methods available that will ensure a high quality evaluation, and thus this section is under-described on purpose. However, some elements that are expected to show up in the evaluation and that should be expanded upon in the proposal are the following:

 

1. Methods for data collection

 

Desk review: Of existing literature and grey literature, as well as of existing programme documents, policy frameworks, National Plan of Action, and other relevant documents that have been produced in Bangladesh and in relevant similar contexts. This is expected to take place during the inception period, and should help identify knowledge gaps, should start informing the evaluation matrix and should help develop the data collection tools. The winning firm needs to be in the capacity to identify key documents to review and ask UNICEF to complement when these are not available in the public domain.

 

Review of secondary quantitative data:  Bangladesh is a data rich country, especially when it comes to data that is representative to the district level. The following datasets should be used to assess outcome and impact level indicators, as well as access to some services:

• MICS 2013-2014: Data  are publicly available for all districts and possibly can be used as baseline or point of comparison. This would include information on child marriage indicators, as well as on access to school and formal and non-formal education indicators.
• MICS 2019: Data collection is almost complete. It is expected that key indicators can be used for in this evaluation
• ECBSS: Data on effective coverage of social services at district level. In the case of child marriage, it includes a module on social norms.

 

Quantitative data collection: It is suggested that an online survey is used to collect responses from UNFPA, UNICEF and key stakeholders/partners. The inception report will contain the questionnaire and guidance to the online survey process. If a different way of collecting data is perceived superior, or if the high risks of non-response rates are perceived to high by the bidder, alternatives can be suggested in the proposal.

 

This evaluation does not foresee any household surveys to take place, given the rich amount of information available from ECBSS and MICS. However, social media can be used to explore certain evaluation questions with adolescents. Bangladesh has a large number of facebook and u-report users which can be accessed through facebook and u-report polls. Some u-report polls have also already happened on child marriage which should also be reviewed. These are non-representative samples of adolescents, but can be used nonetheless to get a stronger understanding of some of the key areas of the evaluation, especially around outcome and impact level changes.

 

Qualitative data collection: Focus group discussions and key informant interviews with key stakeholders will be conducted. When organizing focus group discussions, attention will be given to ensure: gender balance, geographic distribution, and cultural sensitivity, representation of population groups and representation of the stakeholders/duty bearers at all levels (policy/service providers/target groups/communities). The evaluation team must detail in the technical bid the characteristics of each sample: how it is selected, the rationale for the selection, and the limitations of the sample for interpreting evaluation results. It is expected that both GPECM and non-GPECM areas are covered, and that there is variation of the sample selected.

 

Methods for data analysis

The evaluation matrix will provide the guiding structure for data analysis for all components of the evaluation. The evaluation questions will be used to structure data analysis. The following methods of data analysis and synthesis are encouraged to be used:

 

• Quantitative analysis - Different types of analysis could be explored with the existing datasets. First, MICS will show the trends and changes over time. Special caution needs to be taken when analyzing these trends, as child marriage is usually measured as percentage of women 20-24 who got married before 18. This means that not all the women under this cohort were object of the programme interventions. However, MICS also collects data at the individual level, allowing for targeted analysis to take place. The bidding firms should expand on how to best capitalize on the data. The social norms module has been analysed in a UNICEF Discussion Paper “What Matters Most for Ending Child Marriage in Bangladesh”, but additional analysis can take place, especially when in combination with MICS 2019. Additional analysis can be done with multiple data sets that could help in terms of impact and attribution. The bidder is expected to explore these options in the proposal. It is expected that the proposed methods have limitations and strong assumptions, which need to be noted and taken into consideration when any analysis is being undertaken.

 

Cross-cutting consideration: One important cross-cutting consideration when designing the tools to answer the questions and designing the analysis is taking into consideration the ecological model and to consider both “beneficiaries”, as well as decision makers. This means focus should not only be on adolescent girls, but should also consider key stakeholders, decision makers and influencers around marriage. This includes men and boys, fathers, mothers in law, mullahs, community leaders, policy makers, etc. All levels should be considered across.

Evaluation matrix: The firms are requested to present a preliminary evaluation matrix that shows how different methods will be used to answer each of the evaluation questions proposed.

 

Norms and standards guiding the evaluation

This evaluation will be held to the highest standards employed by UNICEF for the conduct of evaluations and research. This means it will abide by the following:

• United Nations Evaluation Group (UNEG) Norms and Standards for Evaluation in the UN System, 2016
• Ethical Guidelines for UN Evaluations; Ethical guidance for Research in UNICEF

The final report is expected to meet the UNICEF-adapted UNEG Evaluation reports standards as well as benchmarks used in UNICEF’s Global Evaluation Reports Oversight System (GEROS).

 

These guidance documents will be part of the contract of the evaluator/team. It is expected that the evaluator team read these guidelines and documents thoroughly and in the proposal already include a section on quality assurance and how the evaluation will abide to the UNEG norms and standards, and also a section on the expected ethical challenges and issues that the evaluation will need to overcome. The proposal will need to already take into account the need for getting IRB approval that meets the standards set out in the UNICEF Ethical Guidelines for research and evaluation. The proposal will need to spell out how the guidelines will be followed/met, rather than only mentioning that the evaluation will abide by them.

 

A Reference Group comprised of key stakeholders from the government and other partners will be set up from the onset of the evaluation. The Reference Group will be consulted on each key milestone of the evaluation and will give feedback on the TORs and deliverables of the evaluation. Evaluation results will also be presented and validated by the Reference Group. The evaluation manager will have the accountability of accepting each deliverable.

 

1. Work schedule:

 

The time-frame for the entire consultancy is 6 months.

 

 

Deliverable

Items covered/Included

Time frame

Payment schedule

1

Inception report and presentation

• Inception report including sampling design, data collection tools, data analysis and triangulation methodology, timeline, interview schedule, evaluation questions and tools consistency matrix,ethical considerations, etc.
• Secondary data review should have happened during this stage
• Presentation of the inception report to UNICEF for approval of report

1 month

20%

2

Field work report and presentation

• Conduct data collection mission (interviews, surveys, FGDs, observation)
• Presentation of aide memoire on key findings and highlights from field work

2 months

20%

3

Draft final report and presentation

• Data analysis and triangulation
• Responding evaluation questions in report, ensuring DAC-OECD evaluation criteria
• Drafting conclusions and recommendations tied with findings
• Power point presentation with main contents of the report for initial feedback

1.5 months

20%

4

Final  evaluation report

And dissemination workshop

• Incorporation of feedback into final report
• Reader-friendly and innovative policy brief
• A participatory debriefing/workshop of final report with stakeholders to finalize recommendations

0.5 month

20%

5

Methodological report for averted marriages

• Report on the methodology for estimating averted marriages using secondary data

 

1 month

20%

 

7.1  End products

 

1. An inception report that presents the complete methodology approach to conducting the work, with all tools fully drafted. The inception report will also need to fully develop the data collection and analysis strategy and triangulation methodology. The tools and analytical methods used should explicitly consider gender, human rights and equity dimensions. An evaluation matrix that includes the evaluation questions and maps these to the tools and specific questions in the tools, as well as respondent groups is expected. Ethical considerations need to be included. The inception report should present the proposed content of the final report following the GEROS reporting standards. 

 

1. The inception report will need to be accompanied by a PPT, which needs to be presented and shared with the reference group.

 

1. Draft Final Report. A final report that adheres to the GEROS reporting standards. The report needs to show a clear flow from objectives and purpose of the evaluation, evaluation questions, methods and tools used to collect and gather information, analytical approach, findings, conclusions and recommendations. Recommendations are expected to be presented and discussed with the reference group.

 

1. Draft final report PPT: The drat final report is to be presented and discussed with the reference group. Special attention to be taken to the discussion of the recommendations.

 

1. Final Report. A final report that incorporates comments and inputs given to the draft final report and that adheres to the GEROS reporting standards. The report needs to show a clear flow from objectives and purpose of the evaluation, evaluation questions, methods and tools used to collect and gather information, analytical approach, findings, conclusions and recommendations.  The report should aim for conciseness, readability, and visual appeal.

 

1. The final report will be followed by a participatory dissemination workshop, where the key stakeholders will take part in finalizing the recommendations of the report. This presentation will include maximum 10 slides in the key findings, followed by the initial recommendations that will be presented for discussion.

 

1. A reader-friendly policy brief that summarizes the key findings, conclusions and recommendations of the evaluation needs to be produced. The firm can choose the format, but it is expected that innovative formats such as infographics or an ebook are used for enhanced readability.

 

1. Data archive: Data gathered in the exercise is transferred in an organized archive that will permit follow-on users to replicate or extend the analysis. Suitable care to be taken in assuring the anonymity of respondents.

 

1. A stand-alone methodological report that develops a method for estimating the number of prevented marriages as a result of the programme using existing secondary data (MICS 2013, MICS 2019, ECBSS, etc.) and test the methodology using existing secondary data.

 

The inception report and draft evaluation report will be shared with the evaluation reference group for feedback.

 

The consulting team will refine the proposed timeframe and expected products in the inception report.  UNICEF Bangladesh reserves the right to ensure the quality of products submitted by the external evaluation team and will request revisions until the product meets the quality standards as expressed by the joint EMG.

 

1. Supervisors and management of the assignment:

The research and evaluation specialist will manage the evaluation. The Chief SPEAR will supervise the overall assignment. The gender specialist will be involved throughout the entire evaluation.

1. Payment schedule:

First payment: 20% upon approval of the inception report and work plan

Second payment: 20% upon approval of fieldwork report and delivery of presentation

Third Payment: 20 % upon approval of draft final report and delivery of presentation

Fourth payment: 20%  upon approval of final report and dissemination workshop

Fifth payment: 20%  upon approval of a methodological report for estimating averted marriages

 

1. Qualifications or specialized knowledge/experience required for the assignment:

 

This evaluation is to be carried out by highly qualified consultants including an evaluation and a thematic expert as well as a research/data Analysist.

The team must demonstrate a clear understanding of the UN system and ensure that the evaluation is conducted in line with the UNEG Norms and Standards for Evaluation in the UN System and abides by UNEG Ethical Guidelines and Code of Conduct as well as any other relevant ethical codes

The team is expected to be composed of three core members. In case none of the core members is from Bangladesh, the team must include a local expert, who will also need to be fully involved throughout the process. 

1. Team Leader – Evaluation Expert

The team leader must have an extensive experience in leading evaluations or programmes of a similar complexity and character, as well as technical expertise in areas related to child marriage, education, adolescent health, gender equality and women’s empowerment, human rights, behaviour and social change. His/her primary responsibilities will be:

• reviewing and consolidating the team members’ inputs to the evaluation deliverables;
• representing the evaluation team in meetings with stakeholders;
• delivering the inception reports, and evaluation report in line with the requested quality standards.
• Leading the methodological design for estimating averted marriages
• Lead the cost-effectiveness analysis

 

Minimum qualification required:

• Master’s degree in Development Studies, Sociology, Economics, Social Studies, International Relations or other related field.
• 10 to 15 years of experience conducting evaluations in relevant thematic areas such as adolescents and youth, child marriage, child protection, gender equality and education.
• Experience working with United Nations agencies, particularly UNICEF and UNFPA.
• Awareness of ethical risks in programming around sensitive issues, both in programme delivery and in all aspects of M&E.
• Proven skills in evaluation methodology and mixed methods approaches including quantitative and qualitative data collection and analysis techniques.
• Demonstrated expertise/experience in developing results frameworks, tools or guides for monitoring and evaluation;

• Be fully acquainted with results-based management orientation and practices;
• Excellent report writing skills, analytical skills, and computer skills.
• Excellent command in written and spoken English
• Experience leading teams.

 

1. Senior thematic expert

 

Minimum qualification required:

1. One Research/Data Analysist consultant

 

The data/research analyst will support the team leader and thematic expert in data collection, undertaking an in-depth documentary review and preparing inputs to the evaluation report deliverables.

 

In close cooperation, and under the supervision of the team leader, the data/research analyst is expected to:

 

• Conduct the data collection and assemble relevant data and information.
• Conduct the analysis of the secondary data sources
• Undertake desk-review of data sources and materials relevant to the independent evaluation and undertake analysis;
• Prepare matrices and other formats required for systematic analysis and synthesis of the material reviewed;

 

The data/research analyst will have the following qualifications:

 

• a university degree (MA) in one of social sciences, economics, or in evaluation, statistics, or research methods.
• at least 10 years of experience performing quantitative data analysis with proven expertise in quantitative analysis software (e.g. Stata, SPSS, R)
• preferably research or evaluation experience in gender evaluations and gender-analysis;
• understanding of gender equality and the empowerment of girls and women, equity and child rights as an area of policy and practice;
• excellent analytical, writing and communication skills (English and at least working knowledge of French and/or other UN languages);
• experience with literature/document search and analysis.

 

1. One local expert (in case none of the above are from Bangladesh)

 

Minimum qualification required:

 

EVALUATION CRITERIA FOR TECHNICAL PROPOSAL

 

 CATEGORY

POINTS

OVERALL RESPONSE

* Understanding of, and responsiveness to, UNICEF Bangladesh Office requirements;

* Understanding of scope, objectives and completeness of response;

* Overall concord between UNICEF requirements and the proposal.

5

METHODOLOGY AND DETAILED TIMELINE

* Quality and suitability of the proposed approach and methodology including for the additional methodological report on estimation of averted marriages (detailed description of overall approach, draft evaluation matrix, initial sampling design, initial proposed methods, analysis and triangulation, etc.);

* Quality of proposed implementation plan, i.e how the bidder will undertake each task and time-schedules;

*Risk assessment and ethical considerations - recognition of the risks/peripheral problems and methods to prevent and manage risks/peripheral problems, and inclusion of ethical considerations.

* Timelines proposed must be detailed and realistic;

(35)

20

 

 

 

5

 

5

 

5

 

 

ORGANISATIONAL CAPACITY and PROPOSED TEAM

* Professional expertise of the firm/company/organization, knowledge and experience with similar projects, contracts, clients and consulting assignments

* Team leader: Relevant experience, qualifications, and position with firm;

* Team members - Relevant experience, skills & competencies;

* Organization of the team and roles & responsibilities;

 

(30)

10

 

10

5

5

    TOTAL MARKS

70

 

For this RFP, the Technical Proposal has a total score of 70 points. Bidders must score minimum of 49 points to be considered technically compliant and in order, for the Financial Proposals to be opened. Financial proposal has a total score of 30 points.

 

The final selection of the bidder will be based on a quality and cost basis as specified in the RFP.