How can corporations promote the greater good? Can they do good and be profitable? How well can we measure the good they are doing?
These were some of the questions explored at a recent Wharton School Conference entitled Creating Lasting Change: From Social Entrepreneurship to Sustainability in Retail. I provide a brief recap of the event. Then I discuss why I believe program evaluators, program designers, and corporations have a great deal to learn from each other.
The Location
The conference took place at Wharton’s stunning new San Francisco campus. By stunning I mean drop-dead gorgeous. Here is one of its many views.
An Unusual and Effective Conference
The conference was jointly organized by three entities within the Wharton School—the Jay H. Baker Retailing Center, the Initiative for Global Environmental Leadership, and the Wharton Program for Social Impact.
When I first read this I scratched my head. A conference that combined the interests of any two made sense to me. Combining the interests of all three seemed like a stretch. I found—much to my delight—that the conference worked very well because of its two-panel structure.
Panel 1 addressed the social and environmental impact of new ventures; Panel 2 addressed the impact of large, established corporations. This offered an opportunity to compare and contrast new with old, small with large, and risk takers with the risk averse.
Fascinating and enlightening. I explain why after I describe the panels.
Panel 1: Social Entrepreneurship/Innovation
The first panel considered how entrepreneurs and venture capitalists can promote positive environmental and social change.
- Andrew D’Souza, Chief Revenue Officer at Top Hat Monocle, discussed how his company developed web-based clickers for classrooms and online homework tools that are designed to promote learning—a social benefit that can be directly monetized.
- Mike Young, Director of Technology Development at Innova Dynamics, described how his company’s social mission drives their development and commercialization of “disruptive advanced materials technologies for a sustainable future.”
- Amy Errett, Partner at the venture capital firm Maveron, emphasized the firm’s belief that businesses focusing on a social mission tend to achieve financial success.
- Susie Lee, Principal at TBL Capital, outlined her firm’s patient capital approach, which favors companies that balance their pursuit of social, environmental, and financial objectives.
- Raghavan Anand, Chief Financial Officer at One Million Lights, moderated the panel.
Panel 2: Sustainability/CSR in the Retail Industry
The second panel discussed how large, established companies impact society and the natural world, and what it means for a corporation to act responsibly.
Christy Consler, Vice President of Sustainability at Safeway Inc., made the case that the large grocer (roughly 1,700 stores and 180,000 employees) needs to focus on sustainable, socially responsible operations to ensure that it has dependable sources for its product—food—as the world population swells by 2 billion over the next 35 years.
Lori Duvall, Director of Operational Sustainability at eBay Inc., summarized eBay’s sustainability efforts, which include solar power installations, reusable packaging, and community engagement.
Paul Dillinger, Senior Director-Global Design at Levi Strauss & Co., made an excellent presentation on the social and environmental consequences—positive and negative—of the fashion industry, and how the company is working to make a positive impact.
Shauna Sadowski, Director of Sustainability at Annie’s (you know, the company that makes the cute organic, bunny-shaped mac and cheese), discussed how bringing natural foods to the marketplace motivates sustainable, community-centered operations.
Barbara Kahn moderated. She wins the prize for having the longest title—the Patty & Jay H. Baker Professor, Professor of Marketing; Director, Jay H. Baker Retailing Center—and from what I could tell, she deserves every bit of the title.
Measuring Social Impact
I was thrilled to find corporations, new and old, concerned with making the world a better place. Business in general, and Wharton in particular, have certainly changed in the 20 years since I earned my MBA.
The unifying theme of the panels was impact. Inevitably, that discussion turned from how corporations were working to make social and environmental impacts to how they were measuring impacts. When it did, the word evaluation was largely absent, being replaced by metrics, measures, assessments, and indicators. Evaluation, as a field and a discipline, appears to be largely unknown to the corporate world.
Echoing what I heard at the Harvard Social Enterprise Conference (day 1 and day 2), impact measurement was characterized as nascent, difficult, and elusive. Everyone wants to do it; no one knows how.
I find this perplexing. Is the innovation, operational efficiency, and entrepreneurial spirit of American corporations insufficient to crack the nut of impact measurement?
Without a doubt, measuring impact is difficult—but not for the reasons one might expect. Perhaps the greatest challenge is defining what one means by impact. This venerable concept has become a buzzword, signifying both more an less than it should for different people in different settings. Clarifying what we mean simplifies the task of measurement considerably. In this setting, two meanings dominated the discussion.
One was the intended benefit of a product or service. Top Hat Monocle’s products are intended to increase learning. Annie’s foods are intended to promote health. Evaluators are familiar with this type of impact and how to measure it. Difficult? Yes. It poses practical and technical challenges, to be sure. Nascent and elusive? No. Evaluators have a wide range of tools and techniques that we use regularly to estimate impacts of this type.
The other dominant meaning was the consequences of operations. Evaluators are probably less familiar with this type of impact.
Consider Levi’s. In the past, 42 liters of fresh water were required to produce one pair of Levi’s jeans. According to Paul Dillinger, the company has since produced about 13 million pairs using a more water-efficient process, reducing the total water required for these jeans from roughly 546 million liters to 374 million liters—an estimated savings of 172 million liters.
Is that a lot? The Institute of Medicine estimates that one person requires about 1,000 liters of drinking water per year (2.2 to 3 liters per day making a variety of assumptions)—so Levi’s saved enough drinking water for about 172,000 people for one year. Not bad.
But operational impact is more complex than that. Levi’s still used the equivalent yearly drinking water for 374,000 people in places where potable water may be in short supply. The water that was saved cannot be easily moved where it may be needed more for drinking, irrigation, or sanitation. If the water that is used for the production of jeans is not handled properly, it may contaminate larger supplies of fresh water, resulting in a net loss of potable water. The availability of more fresh water in a region can change behavior in ways that negate the savings, such as attracting new industries that depend on water or inducing wasteful water consumption practices.
Is it difficult to measure operational impact? Yes. Even estimating something as tangible as water use is challenging. Elusive? No. We can produce impact estimates, although they may be rough. Nascent? Yes and no. Measuring operational impact depends on modeling systems, testing assumptions, and gauging human behavior. Evaluators have a long history of doing these things, although not in combination for the purpose of measuring operational impact.
It seems to me that evaluators and corporations could learn a great deal from each other. It is a shame these two worlds are so widely separated.
Designing Corporate Social Responsibility Programs
With all the attention given to estimating the value of corporate social responsibility programs, the values underlying them were not fully explored. Yet the varied and often conflicting values of shareholders and stakeholders pose the most significant challenge facing those designing these programs.
Why do I say that? Because it has been that way for over 100 years.
The concept of corporate social responsibility has deep roots. In 1909, William Tolman wrote about a trend he observed in manufacturing. Many industrialists, by his estimation, were taking steps to improve the working conditions, pay, health, and communities of their employees. He noted that these unprompted actions had various motives—a feeling that workers were owed the improvements, unqualified altruism, or the belief that the efforts would lead to greater profits.
Tolman placed a great deal of faith in the last motive. Too much faith. Twentieth-century industrial development was not characterized by rational, profit-maximizing companies competing to improve the lot of stakeholders in order to increase the wealth of shareholders. On the contrary, making the world a better place typically entailed tradeoffs that shareholders found unacceptable.
So these early efforts failed. The primary reason was that their designs did not align the values of shareholders and stakeholders.
Can the values of shareholders and stakeholders be more closely aligned today? I believe they can be. The founders of many new ventures, like Top Hat Monocle and Innova Dynamics, bring different values to their enterprises. For them, Tolman’s nobler motives—believing that people deserve a better life and a desire to do something decent in the world—are the cornerstones of their company cultures. Even in more established organizations—Safeway and Levi’s—there appears to be a cultural shift taking place. And many venture capital firms are willing to take a patient capital approach, waiting longer and accepting lower returns, if it means they can promote a greater social good.
This is change for the better. But I wonder if we, like Tolman, are putting too much faith in win-win scenarios in which we imagine shareholders profit and stakeholders benefit.
It is tempting to conclude that corporate social responsibility programs are win-win. The most visible examples, like those presented at this conference, are. What lies outside of our field of view, however, are the majority of rational, profit-seeking corporations that are not adopting similar programs. Are we to conclude that these enterprises are not as rational as they should be? Or have we yet to design corporate responsibility programs that resolve the shareholder-stakeholder tradeoffs that most companies face?
Again, there seems to be a great deal that program designers, who are experienced at balancing competing values, and corporations can learn from each other…if only the two worlds met.
EvalBlog 
Evaluation in the Post-Data Age: What Evaluators Can Learn from the 2012 Presidential Election
Stop me if you’ve heard this one before. An evaluator uses data to assess the effectiveness of a program, arrives at a well-reasoned but disappointing conclusion, and finds that the conclusion is not embraced—perhaps ignored or even rejected—by those with a stake in the program.
People—even evaluators—have difficulty accepting new information if it contradicts their beliefs, desires, or interests. It’s unavoidable. When faced with empirical evidence, however, most people will open their minds. At least that has been my experience.
During the presidential election, reluctance to embrace empirical evidence was virtually universal. I began to wonder—had we entered the post-data age?
The human race creates an astonishing amount of data—2.5 quintillion bytes of data per day. In the last two years, we created 90% of all data created throughout human history.
In that time, I suspect that we also engaged in more denial and distortion of data than in all human history.
The election was a particularly bad time for data and the people who love them—but there was a bright spot.
On election day I boarded a plane for London (after voting, of course). Although I had no access to news reports during the flight, I already knew the result—President Obama had about an 84% chance of winning reelection. When I stepped off the plane, I learned he had indeed won. No surprise.
How could I be so certain of the result when the election was hailed as too close to call? I read the FiveThiryEight blog, that’s how. By using data—every available, well-implemented poll—and a strong statistical model, Nate Silver was able to produce a highly credible estimate of the likelihood that one or the other candidate would win.
Most importantly, the estimate did not depend on the analysts’—or anyone’s—desires regarding the outcome of the election.
Although this first-rate work was available to all, television and print news was dominated by unsophisticated analysis of poll data. How often were the results of an individual poll—one data point—presented in a provocative way and its implications debated for as long as breath and column inches could sustain?
Isn’t this the way that we interpret evaluations?
News agencies were looking for the story. The advocates for each candidate were telling their stories. Nothing wrong with that. But when stories shape the particular bits of data that are presented to the public, rather than all of the data being used to shape the story, I fear that the post-data age is already upon us.
Are evaluators expected to do the same when they are asked to tell a program’s story?
It has become acceptable to use data poorly or opportunistically while asserting that our conclusions are data driven. All the while, much stronger conclusions based on better data and data analysis are all around us.
Do evaluators promote similar behavior when we insist that all forms of evaluation can improve data-driven decision making?
The New York Times reported that on election night one commentator, with a sizable stake in the outcome, was unable to accept that actual voting data were valid because they contradicted the story he wanted to tell.
He was already living in the post-data age. Are we?
6 Comments
Filed under Commentary, Evaluation, Evaluation Quality, Program Evaluation
Tagged as evaluation, evaluations, FiveThirtyEight Blog, How to Lie with Statistics, Nate Silver, obama, Program Evaluation