NEW: How do I know it worked? Helpful Tips, Ideas, and Questions about M&E
January 12, 2023
Why do we need to do it?
Monitoring and Evaluation, sometimes abbreviated to M&E, is a way to measure whether your project, programme or campaign has achieved its outcomes. Considering M&E will focus your project and set clear, achievable goals from the outset. It is also a valuable way to improve your work for the next time, and ensure that you are changing the lives of the communities you set out to help.
Do your donors or partners demand facts and figures that sometimes seem completely pointless? Nobody knows your community and project objectives more than you and your team! Instead of thinking of M&E as something you MUST do for the funder, think of M&E as a way to help you achieve the outcomes you WANT. You can design and present the M&E that best supports your grant or project proposal.
When should I do it?
M&E needs to happen in the design phase of your project rather than as an afterthought. A well-designed project will also have achievable goals that are measurable. The design of M&E goes hand in hand with your project goals and aims.
You want to design an online campaign to reduce intolerance between two groups, can the reduction of intolerance actually be measured? How? It may be that you have increased an understanding of behaviours or differences or influenced changes in attitudes, but how will you capture that? Considering these details at the outset will result in a better project.
You may collect base-level data before you implement a project so that you have an idea of whether it has made a difference. For example, you could take a survey of participants’ thoughts on certain topics before you run a training workshop or show them a campaign video. Afterwards, you could ask them the same questions again, have they changed in their views or behaviours? Read more about comparisons in this Resiliency Initiative article.
Analyzing data early and mid-way through a project can help you to make adjustments to improve your outcomes. For example, social media analytics may reveal that your campaign is receiving low engagement with your target audience, giving you a chance to adjust the campaign or think of other ways to reach your target audience, and tweak the messaging or change your media channels during the project.
How should we get the data?
Common ways to collect data both online and offline include in-depth interviews, online and offline surveys, focus group discussions, and collecting social media metrics to measure engagement and interaction with online campaigns.
Many online resources offer new ideas.
- Better Evaluation:
- USAID’s Data for Impact Project:
- Measure Evaluation:
Having several types of data, such as qualitative and quantitative, can help to piece together a clearer story of how your project is working or not working, including how participants and communities are responding to it. Numbers and statistics are useful but people’s views and opinions are important too.
Setting simple objectives and monitoring impact: testing the effect of campaigns against violence in Bangladesh
In Bangladesh, dangerous online content is one factor that led to violence including mass rioting across the country. Recently, an image posted on social media led to at least 141 religious temples being vandalized, with nine fatalities and over 300 people wounded. Against this backdrop, Save Youth Bangladesh aimed to mobilize young people to prevent tensions. The campaign expects to help them identify dangerous speech and then produce early warnings, while making sure that they are aware of the difference between free speech and dangerous speech.
By setting up clear aims and using simple methods to assess progress, the project was able to carry out clearly defined activities such as training and online campaigns. The project team got clear monitoring data by seeking feedback and tracking online metrics, enabling them to improve the campaign material and gain greater attention over time. The campaign reached over 25,000 people, and rather than just counting the number of ‘likes’, the project team considered Facebook insights data, carefully read online comments, and carried out in-person interviews. These steps, which were not complex and did not need sophisticated skills, set up a strong basis for future campaigns.
Who should do the M&E?
With some training and by building awareness, your team should be able to collect this data either in person or through online methods. If you have a very complex project or data set, you might like to bring in a data expert or analyst but this will depend on the scope of your work or your budget, and it is not always needed. Simple is often better.
How much data do I need to collect?
This is a big question and goes back to the design of your actual project. Think about the scope of your project, who are you trying to help or influence? Is your sample representative of views, behaviours, and opinions? Are there any foreseeable limitations?
It is okay for a project to be smaller or more targeted in scope. It is more important to tell the story of that data and acknowledge its limitations.
Think about pilots or focus groups at the start. Once you have a version of your project and M&E design you are happy with, consider running a pilot of your M&E tools on a small group of representative respondents, ideally before you launch!
What if the results aren’t great?
So, you conduct some interviews or surveys but get negative responses, or you see little change or impact. Here it is important to think about whether this is due to the actual project activities or because of problems with the collection of data.
It is still helpful to analyse this data and try to understand what went right or wrong. Presenting this explanation of what happened and why is sometimes just as valuable as showing glowing results and impressive statistics.
How should I present M&E?
Tell your story as clearly as possible. This may mean taking samples of your interview responses, showing numbers of engagement and reach of your campaign, highlighting uptake of an initiative you implemented in numbers or statistics, or comparing before and after results.
What is helpful is linking how activity in your project actually led to a result – a change in behaviour, action, attitudes – rather than how many people attended a workshop or read a post (though you may use this information in conjunction with the story).