Measured Steps To Social Impact

The 18^th December, 2014 tweet chat by Doers’ Labs focused on identifying the main steps to measuring social impact, and the finer points of each step.  But the first question that organizations often ask themselves is, “why is measurement important?”  One participant in the tweet chat posed this  question provocatively, asking whether measurement is really important for for-profit social enterprises or only to donors and impact investors.  There are many reasons that measurement is important, for both for-profit social enterprises and non-profit organizations.  The reasons offered during the chat were concentrated on the importance of measurement to testing an organization’s assumptions of what it needs to do to reach its goals (create impact), and consequently providing the basis for short and long-term decision-making.

The main steps to measuring social impact are formulating: 1) a logic model, 2) assumptions, 3) indicators and 4) tools.  A logic model defines the end goal (impact) that an organization wants to create, its intermediate goals (outcomes) and links them to what it does through a chain of results.  While an organization’s end goal should be less flexible than its activities, often unfortunately it’s the other way around.

The assumptions in your logic model are what an organization’s measurement system should test.  These assumptions will often lead an organization to the questions it wants its measurement system to answer.  It is good practice to also identify the decisions that will follow from answering each question, for an organization to ensure that its measurement system is providing data that will be used. 

All too many times discussions of measurement begin with indicators.  However, organizations that hesitate to state impacts that are difficult to measure forget that what you can measure evolves over time.  An organization may choose to start by measuring outcomes, and as its knowledge and resources increase include impacts at a later stage.  Nevertheless, it is important that the logic model articulates a complete chain of results, to keep the organization headed in the right direction.

In addition, indicators are only one part of a measurement system, and are only useful in so far as they help an organization answer the questions that it identified in the previous step.  The tweet chat included a conversation on how rubrics can be used to turn qualitative data into quantitative indicators, to measure phenomena like social autonomy and empowerment.  However, the greatest power of qualitative data is probably in its ability to help an organization understand why change has or has not happened.  The usefulness of indicators is limited to understanding whether change has happened or not.  Therefore, identifying questions that go beyond indicators is critical.          

There was widespread agreement during the tweet chat that unless an organization goes through the steps above, it is unlikely to be able to design effective tools to collect data.  However, once an organization goes through the above steps, there are technologies available to help it design and administer tools to collect data.  Uniphore, Social Cops and Vera Solutions were some of the companies mentioned during the tweet chat who enable organizations to collect, visualize and disseminate data.  Finally, it was suggested that the “next practice” for organizations that have robust measurement systems would be to use this data to create social impact models much as is done in the field of economics. 

Translating the "Corporate Social Responsibilty" mandate on ground

While going through the CSR reporting in India, Survey Report 2013 by KPMG I stumbled upon a heading ‘Regulation drives reporting‘. This seems so true in the common scenario. The natural human behaviour is to react to a situation than to respond knowing fully well that it is the response that is going to achieve positive result vis-a-vis reaction. Can a CSR report generated out of reaction mitigate the risks that companies face due to social and environmental impact? I read an article where the writer had been asked to recommend 7 short reports (under 20 pages). We love reports which spare us the jargon and reflect the real picture.

 Why report at all? Yes its required due to the new section 135 in Companies Act but it’s just a part of the existing Financial Report and one can do the to the point reporting to meet the legal requirement. The companies worldwide have been doing CSR reporting without any legal binding simply because it enhances the returns from their CSR efforts. The impact of a CSR project can be enhanced if it is reviewed periodically by stakeholders. To do that we need to have a CSR policy that envisages active stakeholders’ engagement. The new CSR regulation also envisages the companies to initiate the process by creating a committee which puts together a well thought CSR strategy. In words of Sun Tzu, “Strategy without tactics is the slowest route to victory. Tactics without strategy is the noise before defeat.”  The corporate sector has realised that along with traditional functions like marketing, branding, R&D etc., CSR is also a viable component of their overall business objective. The current reporting as is observed in the CSR Reporting Survey by KPMG 2013 merely indicates that the company is performing its social responsibility. There is a room for improvement so that CSR projects provide substantial benefits to all stakeholders and the business. It’s a win-win situation where the CSR can enhance impact for the society and profits for the business.

At present the reporting is higher of the positives and low of the negatives. Generally, the financial statements indicate the inflows and outflows and net result. Similarly the CSR reports should be made to reflect the efforts versus impact. Currently he CSR projects by the businesses are standalone activities without any linkages to the value chain of the business and fail to indicate the impact of the business on the social and environmental issues. To sum up the CSR reporting should use lessons from the financial reporting to develop a CSR reporting system to inform the stakeholders and yield better connect.

Creating a "culture of reflection"

Seven years ago, when I found myself actively involved with the social sector, I was deeply touched by the sheer number of interesting and passionate people I started meeting, all with a common goal of doing good. The reaction to the projects I worked on, the feedback from those who it touched made the whole shift from the corporate sector so totally worth it!

As I understood the sector better, and the process of running high impact social projects better, I realised that in so many cases, that the impact could have been so much more had it been captured well, gaps analysed and addressed to make the processes more efficient. In many cases, there is a strong "culture of doing" but an active "culture of reflection" is missing.

In one of the projects we at niiti were working on, where large scale rural livelihood initiatives have been deployed, we were amazed at the sheer scale of operations, the dedication of the staff and the commitment of the organisation for grassroots level change. Its admirable because bringing large scale change at the grassroots isn't the easiest thing. It takes a lot of hard work, unflinching commitment and selfless people to make it work.

But sometimes, "doing" can overwhelm and consume one so much that one forgets to pause to reflect. Given that almost all social sector projects are run rather frugally, determining from time to time whether processes can improved to make them more efficient may actually be more helpful in achieving the final mission.

Monitoring and evaluating (M&E) projects on a regular basis help bring in a "culture of reflection", and help identify gaps and inefficiencies in running operations, so that they can be improved or changed to achieve better results. This of course, is easier said than done. Often, to implement a good M&E design in an organisation requires change management strategies as a precursor. For, passionate grassroots workers may feel uncomfortable with the M&E team scrutinising their work, and may resist or rubbish the whole process in its entirety. These are legitimate concerns and absolutely need to be addressed. I recently chanced upon a wonderful site (http://www.unodc.org) where some examples of the right approach to bringing in this reflective culture through implementation of a good M&E process are.

Often, monitoring and evaluation as a task is thought of as an end tool to "measure" social impact. Yes, it definitely could have uses to measure social impact, but creating organisations that are open, adaptable and focussed on achieving  their social mission through processes that are most effective in a dynamic environment is a far greater output of having monitoring and evaluation programmes as an integral part of operations. 

Demystifying the Company Bill 2013 - Part 2

To create sustainable social impact, funding is important but often it is not directly proportionate to the depth of impact.  The Companies Bill, 2013 (Section 135 of the Indian Companies Act) makes it imperative for corporates in India to spend a portion of their profits on CSR (Corporate Social Responsibility). On Dec 19th, 2013, we initiated a conversation on twitter to discuss the various aspects of the law and how organisations can leverage this for creating lasting impact and execute this in the spirit that it was intended.

We have attempted to summarize the conversation that came up during the tweet chat in the form of two Q&As for ease of reading. The first one (posted here) is around the legal aspects of Section 135 of the Indian companies Act that was passed earlier this year. The one below outlines recommendations around choosing, monitoring, evaluating and reporting on CSR projects.

If you have any questions on the Bill that have not been covered here, or have a point of view to share on any aspect of doing business in a responsible manner and creating sustainable social impact, please write to us at info@niiticonsulting.com or leave a message on our twitter or facebook pages. 

Q1. Isn't it enough for us to measure how much money we’re spending, to know if we’re being effective or not?

 A1. In the social sector, it is not always the case that if you spend more money the amount of positive impact you create will increase.  Usually what people do with what they get – be it products, skills or a  sense of empowerment – determine the positive impacts/changes in their lives that they are able to realize.  You must measure these “things people do” and the impacts that they realize to know if you are being effective or not.

Q2. Isn't this something that can wait until the project is over?

A2. To know whether someone has realized a positive change in his/her life or not, you have to know what his/her situation was (the baseline) before your project began.  In turn, you need to have articulated what impacts you expect your project to create, and how, to know what baseline data to capture.  All of this should happen before the project begins.  In addition, regular monitoring during the life of the project will help you to detect problems early and make course corrections if necessary.  This requires you to know what to monitor, which again should be connected to the impacts you want to create.

Q3. What will a project evaluation tell us?

A3.Project monitoring will answer questions like, “What is the progress toward goals?”  “What are the short-term results (outcomes)?”

Q4. What will a project monitoring tell us?

A4. Project evaluation will answer questions like, “Why did these outcomes occur?” “What are the long-term results (impacts)?”  “What unanticipated results occurred?”  “What are the knock-on impacts?”  Some evaluation designs can also answer the question of whether the project caused the outcomes measured.

Q5. What are the steps to setting up an M&E system?

A5. The first step is to create a logic model that articulates what impacts you expect your project to create, and works backwards through your outcomes, outputs, activities and inputs.  The second step is to formulate indicators at all these levels: impacts, outcomes, outputs, activities and inputs.  The third step is to determine which indicators you will measure through monitoring, and which through evaluation.

Q 6. Isn't there a standard set of indicators we can just use?

A6. If your indicators are not relevant to the changes your project is trying to create, they will not tell you anything about whether you have been successful or not.  Therefore, it is very important that your indicators are directly relevant to your project.  However, once you have decided what you want to measure, standard indicators can sometimes be useful in telling you how to measure it.

Q 7. What is a good evaluation design?

A7.  A good design is one that is both feasible and is able to meet the requirements of the evaluation (in terms of what needs to be assessed, and the levels of statistical precision needed).  Some of the factors that determine feasibility are the time, data and resources available.  Without a control/comparison group, you will not be able to say definitively that any positive changes that have been measured were caused by your project and not by external factors.