Different sectors – academia, government, nonprofit, and for-profit – can learn from each other when it comes to using evaluation and data to facilitate positive change. This is a three-part series on evaluation in the for-profit sector (often called “social impact measurement”). In Part I, we explored what is going on in the for-profit sector in terms of social/environmental impact measurement and evaluation. In Parts II and III, we explore what the different sectors can learn from each other.
My previous blog post was about buckets. This post is about shoes.
What the buckets and the shoes illustrate is that the for-profit sector’s mindsets towards social/environmental impact measurement and evaluation can be a double-edged sword. On one side, the for-profit sector helpfully reminds us that you do not always need more or better data; you need just enough to inform the decision at hand and facilitate your mission. On the other side, for-profit players – even the “socially conscious” ones – often make big moves without much evidence on how these will impact people and planet. This is a pitfall and an area where the academic, government, and nonprofit sectors can offer guidance.
To explore this pitfall, let’s talk about shoes. Specifically, TOMS Shoes. You likely have heard of these; maybe you even own a pair. What makes TOMS notable is not the shoes themselves but the business model behind them. TOMS started in 2006 with a novel idea: for each pair of shoes a customer bought, the company would donate a pair to a child in need. Many businesses have since adopted this one-for-one model1, and “One for One” is now a TOMS registered trademark.
The idea that “One for One” shoes would be good for the world was plausible and profitable. For example, making sure that children have shoes is something that could promote health: over half a billion people worldwide are infected with hookworm, a parasite that can cause blood loss and anemia2, and people mainly get hookworm from walking barefoot on contaminated soil. And driven by the smart marketing psychology of providing a personal connection between “donor” and “recipient”1, TOMS grew exponentially and was valued at $625 million in 20143. To date, the company reports having given away 60 million pairs of shoes4.
But here is the issue: nobody seems quite sure what the health or social impact of TOMS Shoes has actually been.
People have opinions. To some, TOMS is a “darling” of corporate social responsibility5. To others, TOMS is an example of good intentions gone wrong. Critics denounce TOMS for profiting off of consumer “do-gooding”6 and/or point out that donating shoes could actually produce negative consequences by undermining local shoe producers/sellers1.
Both viewpoints seem mostly speculative. The TOMS website reports impressive donation metrics of shoes and other goods like eyeglasses – and then makes claims of downstream impacts (“improved health, increased economic productivity, job creation and access to education”4) without any clear metrics/evidence. And what about the effect of TOMS on local shoe markets? I am aware of only one rigorous evaluation on this topic, a study among 979 households in El Salvador that found no significant effects on shoe-purchasing behaviors7.
I do not argue that TOMS Shoes are “good” or “bad” but rather that we do not have enough evidence of their impact one way or the other. TOMS illustrates a frequent scenario in the for-profit social enterprise sector: rapid growth and massive movement of money and goods without substantial data/evidence on social or environmental impact. At a 2016 conference, Kelly McCarthy of the Global Impact Investing Network noted that many impact investors are still making decisions based on intuition. This scenario persists despite progress in the for-profit sector when it comes to impact measurement and evaluation.
Here is where traditional evaluation professionals, as well as those working in academia, government, or nonprofits, have something to offer. Two areas stand out to me:
“Theory of change” thinking. A “theory of change” explicitly describes a theorized progression from a product/service/program to its downstream impacts on people and planet. Many of you are familiar with this concept, which I will cover more deeply in a future post. I bring it up here because this type of thinking is fairly underdeveloped in the for-profit sector. While some in social enterprise, corporate social responsibility, and impact investing do incorporate theory of change approaches, more frequently I have come across laundry lists of “impact metrics” that do not clearly distinguish between efforts and results. A claim of “impact” may be based on a metric like “number of products sold/donated,” rather than on a metric like “decrease in prevalence of Disease X.” By contrast, the academic/government/nonprofit sectors have for decades been progressing towards evaluating downstream outcomes/impacts (not just inputs/activities/outputs8,9). They can offer insight to help the for-profit sector examine assumptions behind good intentions.
Importance of Context. At a 2016 conference, Kate Ruff of Social Value International insightfully noted that “impact measurement” in the for-profit sector seeks standardization and comparability whereas classic “evaluation” in other sectors emphasizes context. To illustrate: an impact investor may develop a social/environmental impact scorecard by which to assess all businesses in a portfolio on directly comparable standards, whereas a nonprofit program evaluator may dig into the contextual factors that influenced a program’s impacts. Both approaches are useful. Those working in academia/government/nonprofits can offer nuance to the for-profit sector when it comes to interpreting impact data, which is always highly dependent on context.
To wrap up this series: there is momentum for developing social/environmental impact measurement and evaluation approaches in the for-profit sector. This sector has much to offer to, and much to learn from, other sectors. There is a terrific cross-sectorial learning opportunity here that can help us make the best use of data and evaluation to promote the greater good.
- Knowledge@Wharton (The Wharton School). (2015). The One-for-one Business Model: Avoiding Unintended Consequences. Retrieved 4/13/2018, from http://knowledge.wharton.upenn.edu/article/one-one-business-model-social-impact-avoiding-unintended-consequences/
- Centers for Disease Control and Prevention. Parasites – Hookworm. Retrieved 4/13/2018, from https://www.cdc.gov/parasites/hookworm/
- Roumeliotis, G., & Oran, O. (2014). Exclusive: Bain Capital to invest in shoemaker TOMS – sources. Retrieved 4/13/2018, from https://www.reuters.com/article/us-toms-baincapital/exclusive-bain-capital-to-invest-in-shoemaker-toms-sources-idUSKBN0GK1ZZ20140820
- The TOMS Story. (2018). Retrieved 4/13/2018, from http://www.toms.com/about-toms
- Fritz, J. (2016). Lessons in Corporate Social Responsibility from TOMS Shoes. from https://www.thebalance.com/lessons-from-toms-shoes-2502511
- Jackson, L. (2013). They don’t not want babies: globalizing philosophy of education and the social imaginary of international development. Philosophy of Education(2013), 353-361.
- Wydick, B., Katz, E., & Janet, B. Do in-kind transfers damage local markets? The case of TOMS Shoe donations in El Salvador. Journal of Development Effectiveness, 6(3), 249-267.
- Eckerd, A., & Moulton, S. (2010). Heterogeneous Roles and Heterogeneous Practices: Understanding the Adoption and Uses of Nonprofit Performance Evaluations. American Journal of Evaluation, 32(1), 98-117. doi: 10.1177/1098214010381780
- Perrin, B. (2006). Moving from Outputs to Outcomes: Practical Advice from Governments Around the World Managing for Performance and Results: World Bank and IBM Center for The Business of Government.