The aim of an Evaluation is that of assessing the relevance and fulfilment of objectives, developmental efficiency, effectiveness, impact and sustainability of the intervention. The evaluation of a project must be impartial and independent, credible, useful, and carried out with the participation of the project's stakeholders. The scope of the evaluation includes review of all resources mobilised and activities implemented in the framework of the project.
In case the partner intends to request EC's funding for the external evaluation of the project, Terms of Reference (including methodology, duration and foreseen calendar) will have to be provided with the project proposal (or later on in the Interim or Pre-Final stage) and require the official agreement of EuropeAid.
To be considered eligible, evaluation costs have to be incurred (committed) before the end of the implementing period of the project (i.e. during the period running from the date of eligibility of expenditure of the operation established in the Special Conditions of the Grant Contract to the end of the implementation period). The evaluation itself must be carried out within three months of the end of the operation. Evaluation reports have to be submitted to EuropeAid with the final report.
The external evaluation team should be engaged via the applicable tendering/contracting procedure. The candidate contractor shall prepare a proposal in response to the ToR issued by the commissioning service. The contents of the proposal shall be as follows:
- understanding of the context, purpose and intended users of the evaluation;
- understanding of the themes or questions to be covered;
- indicative methodological design;
- core evaluation members, their field of expertise and their role;
- time scheduled;
- detailed budget;
- CVs in the standard format and declarations of absence of conflict of interest;
- CV of an expert from outside the evaluation team who will be in charge of quality control.
During the inception phase, which starts when the evaluation team is engaged and lasts a few weeks, team members collect basic documentation:
- project documents;
- ex-ante evaluation/situation analysis;
- EC documents setting the policy framework in which the project takes place (e.g. relative EC development and external relations policy, country strategy paper;
- Government strategy (e.g. PRSP).
The team shall also consult all relevant management and monitoring documents so as to acquire a comprehensive knowledge of the project covering:
- full identification;
- resources planned, committed and used;
- progress of outputs;
- names and addresses of key informants;
- ratings attributed through the results-oriented monitoring system;
- availability of progress reports and evaluation reports.
The evaluation team then reviews the project logical framework as set up at the beginning of the project cycle and identifies the points which need clarification and/or updating. The analysis of the project logic covers:
- context in which the project has been launched, opportunities and constraints;
- needs to be met, problems to be solved and challenges to be addressed;
- justification of the fact that the needs, problems or challenges could not be addressed more effectively within another framework;
- objectives (general and specific);
- nature of inputs and activities.
Particular care shall be given to the identification of the different levels of objectives and their translation into various levels of intended effects:
- activities expressed (operational objectives) in terms of results for direct beneficiaries and/or outputs (tangible products or services);
- specific objective (project purpose) expressed in terms of sustainable benefit for the target group;
- overall objective(s) expressed in terms of wider effect.
The logical framework preferably includes “Objectively Verifiable Indicators (OVIs)” and “Sources of Information” which are useful for structuring the evaluators' work. In so far as OVIs have been properly monitored, including baseline data, they become a major part of the factual basis of the evaluation. Indicators may also be developed in the framework of the evaluation as part of a questionnaire survey, an analysis of a management database, or an analysis of statistical series. Indicators may be quantitative or qualitative.
Once the analysis has been performed on the basis of official documents, the evaluation team starts interacting with key informants in the project management entourage. The first to be interviewed are generally the persons who are or have been involved in the design, management and supervision of the project. Interviews cover project management, EC services, and possibly key partners in the country/ies concerned. Other evidence is collected through direct observation of facts (including track records, photographs and alike); statements by informants who have been personally involved; identification and reference to proxies; indirect reporting on facts by informants who have not been personally involved. The evaluation team makes use of different participatory tools for research and dialogue and shall insure an ethical behaviour in collecting data.
The core evaluation report shall be based on seven criteria identified by the OECD/DAC and the EC itself. These criteria are the following:
- Relevance: the extent to which the objectives of a development intervention are consistent with beneficiaries' requirements, country needs, global priorities and partners' and donors' policies
- Effectiveness: the extent to which the development intervention's objectives were achieved, or are expected to be achieved, taking into account their relative importance.
- Efficiency: a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.
- Sustainability: the continuation of benefits from a development intervention after major development assistance has been completed. The probability of continued long-term benefits. The resilience to risk of the net benefit flows over time.
- Impact: positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.
- Coherence/complementarity: this criterion may have several dimensions: - Coherence within the Commission's development programme; - Coherence/complementarity with the partner country's policies and with other donors' interventions; - Coherence/complementarity with the other Union policies.
- Union value added: the extent to which the project/programme adds benefits to what would have resulted from Member States' interventions in the same context.
Once a self-assessment of the reliability of data and validity of analyses has been carried out and the quality control guaranteed, the evaluation team reviews the findings with the view of confirming them, and formulates a series of conclusions, recommendations and lessons learnt. The evaluation team then finalises the drafting of the evaluation report following the standard EC format.
The evaluation team leader attaches a quality certificate to the draft final report, indicating the extent to which:
- Evaluation questions are answered.
- Reliability and validity limitations are specified.
- Conclusions apply evaluation criteria in an explicit and transparent manner.
- The present guidelines have been used.
- Tools and analyses have been implemented according to standards.
- The language, layout, illustrations, etc. are according to standards.
Sources and useful links: