what is implementation monitoring and evaluationwhat is implementation monitoring and evaluation

+353 1 416 0500 World Bank Development Impact (blog). Due to directly implementing the program, implementation staff can provide valuable feedback on why it is or is not working. Implementation, monitoring and evaluation - Your Way To Behaviour Change A group of people living in an area or having characteristics in common (e.g. A structure, overview, outline, or system consisting of various descriptive categories and the presumed relationships between them. Are the findings of the original situational analysis still relevant? Progress is, therefore, achieved in an incremental manner over time. We haven't found any reviews in the usual places. Enter monitoring and evaluation (M&E) - programs and systems designed to identify what works and what doesn't in policy design and implementation. To what extent did the programme meet its intended goals and objectives? city, neighbourhood, organisation, service, business, professional association); the larger socio-political-cultural context in which an intervention is intended to operate. Our global office is based at the Department of Economics at the Massachusetts Institute of Technology. What has changed in the environment (politically, administratively, structurally, programmatically etc.) It provides a shared understanding to support the design, management, monitoring and evaluation of interventions for sustainable development. While monitoring and evaluation is an accountability mechanism to ensure and report that activities are being delivered as planned, it should also be used for learning and adaptive planning. The Hawthorne Effect: What Do We Really Learn from Watching Teachers (and Others)?. Are there any new risks that need to be mitigated? If use of privately owned automobile is authorized or if no Government-furnished automobile is available. var sproutencodeemailRot13String = "avbssvpr@rssrpgvirfreivprf.bet"; Front Matter. This helps clarify what outcomes are useful to measure. Interim outcomes you will achieve by the end of each reporting period. Implementation Monitoring and Process Evaluation by Ruth P. Saunders is a practical guide that helps readers understand and use the steps that program planners and evaluators take in implementing and monitoring a new program, policy, or practice in an organizational setting. Choose Implementation Monitoring Methods and Compile the Comprehensive Implementation Monitoring Plan, Step 9. What are the significant changes and achievements? #1 Bad Data. The action or process of formally discussing something with stakeholders, generally asking stakeholders a relevant question and receiving answers to that question. For example, the number of people who will be trained, number of hardware produced, or number of committees formed. The carrying out of planned, intentional activities that aim to turn evidence and ideas into policies and practices that work for people in the real world. The action of leading a group of people, or the ability to do this. "The View from Above: Applications of Satellite Data in Economics." Implementation monitoring and process evaluation. Use the definitions and checklist for logframe development to help the development process. A better understanding of the intricacies of intervention development, delivery and evaluation is an important part of many student's development. In the correct quantity? Available at: [Accessed 1 Jul 2023]. However, interviews and focus groups can be more costly (in terms of both monetary and time costs) than other forms of monitoring. www.sagepub.com. Much of this information may already exist thanks to routine national and district data collection and should be collated and analysed to arrive at a baseline. There can be no doubt that the only thing worse than no data, is bad data. Monitoring and Evaluation Assistant (re-announcement); Good Neighbors Results monitoring. Are there barriers to implementation? Monitoring and Evaluation (M&E) is a combined term for the processes set up by organizations such as companies, government agencies, international organisations and NGOs, with the goal of improving their management of outputs, outcomes and impact. To inform ex-post generalizability and scaling-up collect data on the costs of the intervention and contextual details. Ideal for practitioners, researchers, and students, this book can be used as a primary text for a process evaluation or an implementation monitoring course or as a supplemental text in a broader program evaluation course. ), Depending on the indicators captured by the partner,may not provide information on why a program is or isnt working, May not provide real-time feedback and instead cause delays in analysis, Can provide valuable insight toinformresearch design, Can provide insight into why an intervention is or isnt working, Can check for respondents perceptions of treatment quality, Can provide qualitative, anecdotal evidence, May be more costly than using admin data or adding survey questions, Can be difficult to administer and moderate, Can be deployed rapidly and implemented at a low cost, Can be used at a high frequency over a period of time (especially between major survey waves) to constantly monitor dynamic programs, Can be adapted swiftly to changing circumstances, (If using mobile phones) May be less effective due to under-coverage of groups with poor network connections or limited access to phones, Known to be affected by high levels of non-response and attrition, Can be limited by the practical length of the interview, Can be low-cost if built on an already existing system, Researchers may have limited control over variables collected, Some data (sensor data, photographs) may be difficult to work with in analysis, Research staff observe firsthand how the intervention is being implemented, Can check for problems with treatment quality, implementer effects, Can be more costly (financially and in terms of time) than using admin data or adding questions to an endline survey or back-checks, Of little value if not done properly (e.g., implementers notified of visit in advance). Other projects have used remote sensors to monitor fuel emissions from cookstoves. and the specific research questions that the project is trying to answer. Monitoring and Evaluation is an approach that has mainly been developed to measure and assess the success and performance of projects, programs or entire organizations. Monitoring and evaluation use data to determine if indicators are being met and outcomes achieved. Implementation monitoring | The Abdul Latif Jameel Poverty Action Lab While randomization ensures internal validity at the beginning of the study, there are several threats (such as attrition, spillovers, non-compliance, and evaluation-driven effects) that can arise over the course of the experiment and bias estimates of the treatment effect. Researchers can use a variety of subjective and objective methods to monitor implementation. As pointed out in Holla (2019), collecting this data may be challenging: partner organizations may not be willing to share this data, or may fear how it is used. what constitutes a household? While monitoringaccording to Merriam-Websteris used to "warn and instruct," evaluation is used "to judge the value or condition of something in a careful and thoughtful way.". Monitoring the implementation of an intervention can help mitigate threats to internal validity. Involving relevant stakeholders and effective planning are crucial aspects of monitoring and evaluation. In addition to daily control over operations, deadlines, and any problems that may arise, the institution promoting a development program, or an administration responsible for policy implementation, should also perform periodic evaluations of the program or policy as a whole. Good planning, combined with effective monitoring and evaluation, can play a major role in enhancing the effectiveness of development programs and projects. Care should be taken to ensure that research staff do not elicitor social desirability biases. Airplane*. The programme risk analysis template should help undertake this process. Are there lessons and insights on why progress is or isnt being achieved? Resources needed to carry out activities and outputs. Maps are powerful tools to visually represent need and progress over time. Focus groups with study participants may reveal valuable insights, but should be planned carefullyresearch staff may clue participants into the study design, and cause social desirability bias to influence participants' responses.7. This is where monitoring entwines with evaluation. The Measuring Outcomes Guidebook (Strengthening Nonprofits, 2010) is a guide for organisations interested in learning more about implementing or improving their procedures for measuring and monitoring outcomes. Develop Strategies for Facilitating Program, Policy, or Practice Implementation and Sustainability, Step 6. A wide range of real-world examples in the book are drawn from health, education, non-profit organizations, and public administration, and an extended case study, Your Turn boxes, and worksheet templates help readers apply concepts to their own projects. Establish Complete and Acceptable Delivery/ Installation of the Program, Policy, or Practice, Step 5. Effectiveness, Efficiency, Impact and Pertinence Indicators. Speaking with implementers and key informants can reveal information that isnt reported by respondents or enumerators, such as logistical challenges in delivering the intervention, pushback from local leaders on who receives the program, confusion about the rules of the program, or cultural issues that may affect how the program and/or research is being delivered and received and may be difficult to observe. A good logical framework (logframe) is a visual representation of the logic underlying a programmes purpose and activities. As the name suggests, organisational monitoring tracks institutional development, communication, collaboration, sustainability and capacity building within an organisation and with its partners and stakeholders in relation to project implementation. ), Can be generated at low cost, especially if the implementing partner is already generating it, Can provide real-time feedback on uptake, implementation (depending on study, data, and partners), Definitions may differ by partner organization (e.g. Implement the Program, Policy, or Practice and Use Implementation Data for Formative Purposes, Step 10. Practices, programmes, policies, strategies or other activities that have been empirically shown through scientific research and evaluation processes to improve outcomes to some degree. Implementation Monitoring and Process Evaluation Thousand Oaks, CA: SAGE Publications, Inc; 2016. doi:10.4135/9781071878736. A wide range of real-world examples in the book are drawn from health, education, non-profit organizations, and public administration, and an extended case study, Step 1. Conduct Preimplementation Planning, Step 2. Saunders, Ruth P. Implementation Monitoring and Process Evaluation. 3 . Working in Monitoring and Evaluation (M&E) can be extremely challenging at first, as you navigate your way through deep theory, and complex implementation. Established monitoring, evaluation and reporting modalities for national and local government policies, including for ageing-related policies, can be followed when designing the M&E approach for the mainstreaming . Enumerator effects occur when respondents change their behavior in response to an enumerator rather than the intervention. April, 2019. To limit attrition, research teams can use mobile phone surveys to keep respondents contact information up to date, especially if there are long gaps between survey rounds. [1] Evaluation involves the examination of the relevance, effectiveness, efficiency and impact of activities in the light of specified objectives.[2]. Does it build stronger, more responsive local institutions? A deliberate simplification of a phenomenon. To facilitate evaluation procedures during and after activities, through the definition of specific indicators. Assessment, monitoring and evaluation are used at different stages during a humanitarian response and are closely linked to public-health decision-making and the implementation of RH programme activities. Monitoring and Evaluation is also useful in the Facilities [Hospitals], it enables the donors such as WHO and UNICEF to know whether the funds provided are well utilized in purchasing drugs and also equipment in the Hospitals. You may have heard some of these terms used in different combinations; perhaps the most common of these is "M&E" (monitoring and evaluation), although it's . Statements about the uncertainty factors that may affect the programme. While some administrative data can provide real-time feedback, sometimes it can cause delays in analysis, if it is generated infrequently. Any specific method or action aimed at overcoming barriers, increasing the pace and effectiveness of implementation, and sustaining interventions over time. - Indicators relating to changes in individual, family and community behaviours and perceptions over time, or proxy measures such as presence of hand-washing stations with soap and water Thank you for participating in this survey! Monitoring includes the continuous assessment of programmes based on early detailed information on the progress or delay of the ongoing assessed activities. That could be poverty reduction, achievement of SDG 3 targets in your country, NTD elimination or sustained control, etc. To enhance your experience on our site, Sage stores cookies on your computer. To avoid enumerator effects on the basis of enumerator characteristics, researchers can consider piloting the survey to determine if enumerator characteristics systematically influence responses. Methods or actions that aim to overcome barriers, increase the pace and effectiveness of implementation, and sustain interventions over time. Implementation, Monitoring and EvaluationNow What Do We Do?. Project monitoring is the 'monitoring part' of the monitoring and control phase. If these threats are anticipated, researchers can modify the study design at the outset to avoid themsee the Threats and Analysis lecture from our Evaluating Social Programs (ESP) course or the resource on randomization for examples. The results of assessment, monitoring and evaluation inform planning for comprehensive RH programmes because they help to: Analyze and Synthesize Implementation Monitoring Data, Step 12. Her roles in these projects, which include eight large-scale interventions, have encompassed using multi-level conceptual models to develop a comprehensive model of organizational environments, constructing scales and indexes to assess contextual factors and implementation processes, addressing challenges involved in intervention design and implementation in field-based settings, and designing and carrying comprehensive intervention implementation monitoring. Because of the way that CDD projects are designed and operate, monitoring can be challenging since: Progress against workplan (inputs, outputs), for example: You have clicked on a link to a page that is not part of the beta version of the new worldbank.org. Monitoring checks on all the activities on the last [implementation stage] unlike Evaluation which entails information on whether the donated funds are well managed and that they are transparently spent. The diagram below outlines the steps of the design, implementation, monitoring, and evaluation (DIME) process. An alternative approach is the mystery shopper model, where trained surveyors who are unknown to the implementer pose as potential clients or participants.6. They should include: effectiveness indicators, which: 8 . J-PAL MENA is based at the American University in Cairo, Egypt. It is worth investing in someone to lead this coordination. 5 . Delivery of the strategy is monitored and adjusted as required, and its effect on behaviour is evaluated. Use the Student Study Site to get the most out of your course! Provide a brief summary of the activities that must be implemented to accomplish each output, and provide a summary schedule of periodic meetings, monitoring events and evaluations. Monitoring, evaluation, and learning - also known as MEL - encompasses adaptable frameworks with processes, best practices, and tools that are strategically tailored for the uniqueness of organizations and sector-specific initiatives. Define the activities for achieving each output. Belfast BT8 7AR. This practical guide helps readers understand and use the steps that program planners and evaluators take in implementing and monitoring a new program, policy, or practice in an organizational setting. Measuring multiple results in CDD (multi-sectoral) can be difficult, Measuring social capital, empowerment can be tricky. This first sub-system of indicators should permit the monitoring of the evolution of the economic, social, cultural, institutional, and environmental contexts, and therefore the adoption, in a timely manner, of corrective measures in order to maintain the projects pertinence and relevance. Gladys Lopez-Acevedo is a senior economist with the World Bank who consults with government clients wanting to implement M&E systems. In addition to conducting surveys, enumerators and research staff can also conduct site visits and facilitate focus groups. Hello, would you like to continue browsing the SAGE website? This project is to support the development and implementation of Performance, Monitoring and Evaluation Planning for the NIAA's Policy and Strategy stream. - comparing actual costs with those budgeted for specific activities; 9 Harcourt St, Dublin 2, Company Registration Number: 451580Charity Number 19438. Key Process Issues In addition to daily control over operations, deadlines, and any problems that may arise, the institution promoting a development program, or an administration responsible for policy implementation, should also . PDF Stage 5 - Monitoring and Evaluation - Unece PubMedGoogle Scholar. The Multi-Channel Maximum-Likelihood (MCML) method for infrasound With study participants, researchers should be careful to avoid experimenter effects caused by frequent interaction with study staff. To access progress, to consolidate evaluation reports for the midterm and end of biennium performance assessment, identifying corporate achievements, lessons learnt pitfalls and required actions. While the views of stakeholders may then be used to influence decisions, there is no commitment or requirement to do so. While the midline and endline surveys may occur too late to fix implementation issues while the program is being implemented, the monetary cost of adding questions to existing surveys is low and the information generated is important to document for later analysis. The M&E is separated into two distinguished categories: evaluation and monitoring. The developed countries are using this process to assess their own development and cooperation agencies. It presents standards of good practice in conducting evaluations. Should you need additional information or have questions regarding the HEOA information provided for this title, including what is new to this edition, please email sageheoa@sagepub.com. Once the reflection has taken place, make the necessary changes to your logframe in terms of new resources, activities and outputs. It includes procedures for iterative changes in the planned activities as needed, according to the monitoring data. 3 Assessment, Monitoring and Evaluation - National Center for Conversely, evaluation is a scientific process that gauges the success of the project or program in meeting the objectives. What are the impacts on livelihoods and employment? Site visits allow research staff to directly observe the quality of a program's implementation. Like yin and yang, monitoring and evaluation go hand in hand. It serves as the head office for our network of seven independent regional offices. The monitoring and evaluation framework contained in this H a n d b o o k is therefore guided by three equally important objectives: to align the monitoring and evaluation system with results-based management; to promote evaluative knowledge and learning around results; and to simplify policies and procedures.