In practice, the processes that underpin the need for relevant, accurate and timely monitoring and evaluation (M&E) information involve iterations of an “M&E Data Cycle” comprising the following stages:
- Identify: nominating/defining the nature of the information required by the various internal and external stakeholders
- Capture: obtaining the identified data in practical and efficient ways
- Analyse: subjecting the captured data to appropriate and meaningful analysis
- Disseminate: ensuring that meaningful information relevant to the needs of the various stakeholders is supplied in a timely and useable form
- Utilise: implementing responsive decision-making, critical inquiry and reflection, and transparent disclosure in order to foster accountability and learning
- Assess: reviewing and judging the extent to which the identified M&E data has indeed fostered improved performance (i.e. M&E of M&E—or ‘meta-M&E’)
The M&E Data Cycle should be defined for each level in the project design logic (i.e. each stage of the ‘theory of change’).
Each stage of the M&E Data Cycle is potentially problematic for aid project implementing organisations, thereby eroding the anticipated benefits of M&E. The following assessment tool can be used to pinpoint areas of weakness at each stage of the M&E Data Cycle.
Question Criteria |
(1) | Consistently fails to meet expectations |
(2) | Occasionally fails to meet expectations |
(3) | Consistently meets expectations |
(4) | Occasionally exceeds expectations |
(5) | Consistently exceeds expectations |
Identification:
1 | A coherent conceptual framework (e.g. logframe) is applied to ensure that all M&E data needs are comprehensively met | 1.2.3.4.5 |
2 | Project implementation team are involved in defining the M&E system | 1.2.3.4.5 |
3 | Direct beneficiaries (‘boundary partners’) of the project are involved in defining the M&E system | 1.2.3.4.5 |
4 | Ultimate beneficiaries of the project are involved in defining the M&E system | 1.2.3.4.5 |
5 | Collaborating/strategic partners are involved in defining the M&E system | 1.2.3.4.5 |
6 | Data to verify the quality of project deliverables is identified | 1.2.3.4.5 |
7 | Data is identified to verify the efficiency of project implementation work | 1.2.3.4.5 |
8 | Data to verify the efficacy of project interventions is identified | 1.2.3.4.5 |
9 | Data is identified to verify the effectiveness of the approaches/strategies that underpin project designs | 1.2.3.4.5 |
10 | Data is identified to verify the social, economic and ecological sustainability of interventions | 1.2.3.4.5 |
11 | Data is identified to determine the prevalence and impact of risks affecting the work of the implementation team (‘management risks’) | 1.2.3.4.5 |
12 | Data is identified to determine the prevalence and impact of risks affecting the initial success/uptake of initiatives by direct beneficiaries (‘intervention risks’) | 1.2.3.4.5 |
13 | Data is identified to determine the prevalence and impact of risks affecting the overall impact of the project among the ultimate beneficiaries (‘development risks’) | 1.2.3.4.5 |
14 | A M&E plan identifies the sources/subjects of all M&E data | 1.2.3.4.5 |
15 | A M&E plan defines the persons/roles responsible for capturing the identified data | 1.2.3.4.5 |
16 | A M&E plan defines the nature of analysis to which all captured data will be subjected | 1.2.3.4.5 |
17 | A M&E plan clearly articulates the persons/roles to which analysed data will be supplied/disseminated | 1.2.3.4.5 |
18 | A M&E plan defines the schedule for data capture, analysis and dissemination | 1.2.3.4.5 |
19 | A M&E plan clearly articulates how all M&E data will be utilised | 1.2.3.4.5 |
Capture:
20 | Appropriate/informed ‘subjects of inquiry’ (data sources) are sampled | 1.2.3.4.5 |
21 | Data capture tools and protocols are unambiguous for all people involved | 1.2.3.4.5 |
22 | Data capture tools and protocols are simple to implement | 1.2.3.4.5 |
23 | All staff have been adequately trained to comply with reporting requirements | 1.2.3.4.5 |
24 | Staff compliance with defined data capture is strong | 1.2.3.4.5 |
25 | The amount of time and effort invested by staff in data capture and reporting is appropriate | 1.2.3.4.5 |
26 | Where data is captured directly from beneficiaries/communities, methods used are ethical and unimposing | 1.2.3.4.5 |
Analysis:
27 | All data captured is analysed promptly | 1.2.3.4.5 |
28 | Key persons/roles are assigned the responsibility for data analysis | 1.2.3.4.5 |
29 | Persons/roles who analyse data have the necessary skills | 1.2.3.4.5 |
30 | Analysis is accurate | 1.2.3.4.5 |
31 | Analysis draws out meaningful features and trends in the data | 1.2.3.4.5 |
32 | Analysis includes identification of variance between planned and actual performance | 1.2.3.4.5 |
33 | Analysis examines both states (absolute measures) and trends (relative changes) | 1.2.3.4.5 |
34 | Lessons learned are systematically documented | 1.2.3.4.5 |
Dissemination:
35 | Analysed data is disseminated to pre-defined stakeholders | 1.2.3.4.5 |
36 | Analysed data is disseminated promptly within agreed timeframes | 1.2.3.4.5 |
37 | Disseminated data is in a format that is relevant/accessible | 1.2.3.4.5 |
38 | Reporting staff receive feedback on data captured | 1.2.3.4.5 |
39 | Feedback includes value-adding such as trend analysis | 1.2.3.4.5 |
40 | Analysed data is segmented to ensure relevance to data clients | 1.2.3.4.5 |
Utilisation:
41 | M&E findings inform operational decisions (project implementation) | 1.2.3.4.5 |
42 | M&E findings inform tactical decisions (program management) | 1.2.3.4.5 |
43 | M&E findings inform strategic decisions (organisational direction) | 1.2.3.4.5 |
44 | Key persons/roles are held accountable for utilising disseminated data | 1.2.3.4.5 |
45 | Individuals within the organisation reflect on M&E findings | 1.2.3.4.5 |
46 | Teams within the organisation collectively reflect and debate M&E findings | 1.2.3.4.5 |
47 | Plans are adapted/modified in response to M&E findings | 1.2.3.4.5 |
48 | Project M&E findings are shared with other projects to promote organisational learning | 1.2.3.4.5 |
49 | Feedback of analysed data is used to inform field-based practice | 1.2.3.4.5 |
50 | M&E findings are used to inform planning of future projects | 1.2.3.4.5 |
51 | M&E findings are used to comply with stakeholder reporting requirements | 1.2.3.4.5 |
52 | M&E findings are widely available to ensure transparency | 1.2.3.4.5 |
Assessment:
53 | Time and resources are set aside to periodically review the M&E system | 1.2.3.4.5 |
54 | The time and resources consumed by M&E processes are reviewed | 1.2.3.4.5 |
55 | The extent to which M&E data is serving its intended purpose is reviewed | 1.2.3.4.5 |
56 | The extent to which M&E findings are utilised to improve accountability and learning is reviewed | 1.2.3.4.5 |
57 | M&E tools or protocols are periodically revised/improved | 1.2.3.4.5 |
58 | The usefulness of M&E findings to each class of data client is periodically reviewed | 1.2.3.4.5 |