Baseline study of DEval and its environment (concluded)
As a new player in the field of German development evaluation, DEval sets the highest standards for itself and its own work. For this reason DEval is undertaking a baseline survey of various dimensions of its own objectives, to serve as a benchmark for assessing the impact of its work in the years to come. By conducting this baseline survey, DEval seeks to record the status quo of evaluation practice in the field of German development cooperation at the outset of its activities.
The "Baseline study of DEval and its environment" consists of two main areas of enquiry: a survey to record DEval's baseline and a monitoring exercise to review the implementation of the last system review.
The baseline study will determine the status quo at the outset of DEval's work in two areas of performance: "strategic evaluation" and "methodology research". These areas are derived from the institute's objectives as outlined in the articles of association with its sole shareholder (the German government, represented by the BMZ):
• Performing strategic evaluations: DEval is tasked with conducting strategic evaluations that are relevant for decision-making, in particular for the BMZ, but also for other organisations in the field of German development cooperation.
• Methodology research: DEval is tasked with investigating methods and methodologies, as well as with the further development of evaluation designs, methods and standards.
The aim of the baseline study is to provide DEval with a reference point for these performance areas. This will enable the institute to later identify the changes and impacts that evolve from its own work. Furthermore, the study's results will clarify the institute's position and will be drawn into its strategic planning.
The second area of enquiry consists of monitoring the implementation of measures recommended by the last system review. The system review mentioned here refers to a study carried out in 2008 by Axel Borrmann (Hamburgisches WeltWirtschaftsInstitut) and Reinhard Stockmann (CEval) on evaluation in German development cooperation. Commissioned by the BMZ, the consultants investigated the evaluation practices of German development aid agencies and recommended actions to make improvements. The BMZ has asked DEval to monitor the implementation of these recommendations.
Moreover, in order to draw conclusions about the progress made at a systemic level within the field of evaluating Germany's development cooperation, the study will also assess the current status quo of how evaluation practice among German development cooperation organisations can be characterised. The resulting insights will support the future work of DEval by offering the institute a better understanding of the environment in which it operates.
Due to the large number of results and the variety of stakeholders in this study, the two main areas of the study – surveying the baseline and monitoring the implementation of the last system review’s recommendations – will be covered by two separate reports.
Following a Europe-wide call for tenders, DEval chose to engage Rambøll Management Consulting GmbH in November 2013 as its partner for conducting the baseline survey and monitoring the implementation of the system review recommendations. The methods chosen for the study were then presented to representatives of German development cooperation organisations at a kick-off workshop in December 2013. The inception report was issued at the beginning of April 2014.
The inception report was followed by a phase of extensive surveying conducted together by DEval and external consultants from Syspons GmbH (who were provided by Rambøll Management Consulting GmbH) up through August 2014. DEval was widely supported by the actors and organisations participating in the survey and their provision of a variety of relevant documents. In gathering the data, a number of survey instruments were used, each chosen for the specific areas of enquiry of the study:
• Online questionnaire survey of German development cooperation organisations (April/May 2014, 34 organisations)
• In-depth interviews in German development cooperation organisations (June/July 2014, 17 organisations)
• Meta-evaluation of existing evaluation reports (June-August 2014, 45 evaluation reports)
• In-depth interviews in the BMZ (June-August 2014, 11 interviews)
• Expert interviews with researchers and professional organisations (July/August 2014, 9 interviews)
The collected data was collated, analysed and synthesised in August and September 2014. A wrap-up workshop was held at the end of October 2014, which brought together the involved development organisations to discuss the results and raise any additional points to be considered in the final written reports.