How to measure success?

It isn't only non-profits who have to prove that they are managing their resources well for the good of society - the government has to do this too.

herzog 298.88 (photo credit: Ariel Jerozolimski)
herzog 298.88
(photo credit: Ariel Jerozolimski)
Recently the Ministry of Social Affairs axed a pre-army project that"successfully prepared" some 250 17- to 19-year-old girls at risk. The ministry will no longer pay the salaries of 10 social workers who were employed on this project. WIZO, one of several partners in this project, is frantically looking for alternative funding so that the program can restart at the beginning of September. How can we know if such a project "succeeds" to rehabilitate youths-at-risk? How can we know if some other social intervention project successfully improves human rights, empowers women or promotes interfaith understanding? Many non-profit organizations and government-funded activities are tackling such social issues with budgets of millions of shekels. Are such projects impacting and in any way reducing the woes of our society? The answer, according to Sacha Litman of the Boston company Measuring Success, depends on the specific goals of the intervention project, as well as whether the actions to fulfill these goals are traceable and measurable. McKinsey-trained Litman came to Israel to teach how to set about measuring success and thereby improve management of social interventions. On a sweltering August afternoon, I took the train to Tel Aviv and walked into the cool, posh building of Ernst & Young to get his advice, for free. I found myself among a small group of altruistic people, all from non-profit organizations who are working for the good of society. Their activities include improving human rights in medical contexts and promoting science education. Alona Payis, marketing manager of the Israeli branch of Ernst & Young, one of the top global professional services companies, described her office's decision to target innovative social entrepreneurs. These men and women have entered a multinational competition for the title of Social Entrepreneur of the Year. The social entrepreneur, says Payis's office, is "one who has created and leads an organization, whether for-profit or not, that is aimed at catalyzing large-scale and systemic social change through the introduction of new ideas, methodologies and changes in attitude." The Schwab Foundation, also a player in this competition, announced that the winner will be a "pragmatic visionary" who combines the characteristics represented by Richard Branson and Mother Teresa. All the ordinary-looking people in the room that day apparently fitted this bill. Except me, certainly. Mr. Litman's presentation aimed to help these visionary activists make their impact on society, by teaching them to measure success. His opening quote came from management guru Peter Drucker: "You can't manage what you can't measure." Drucker pioneered "management by objective" and knew how to ask the right questions. Litman explained that a vision of the overall goal, such as promoting human rights or reducing child neglect and abuse, is not enough. We need to focus on the most important levers that will bring about the desired social change. And we need to be able to measure social change. By asking us questions, he showed us how to break down the overall goal of our social intervention into intermediate and short-term goals. Then we need to align our work, he explained, so that all our resources are swimming toward the same short-term goal. Finally, we must plan track-able actions from which we can measure if we have reached our goal. So much for the theory. Litman chose his own visit to a doctor to illustrate his point that anecdotal evaluation is not a measure of success. In a routine check-up he cheerfully asserted "I feel fine!" But the doctor nevertheless decided to take some measures of his health. All his tests were normal, except for the cholesterol test. This showed that he was not as well as he imagined. The doctor told him that he should eat less red meat, and exercise more. A year later the doctor repeated his tests and saw that Litman was not doing enough to maintain his good health: the cholesterol readings were still too high. Together they chose a more effective intervention program in the hope that in a year's time his tests would show real change so that he could hope to feel fine for many years to come. From this example, we learned the value of objective measures. We also learned that a one-off evaluation is insufficient: only ongoing measures, a year apart or a few months apart, will tell us if we are doing the right thing. I immediately recalled the recent visit of the government official in charge of youth in full-time care, who came to inspect the Wing of Love project for rehabilitating teenage youth-at-risk where I volunteer. The official asked the director questions about how our program works and the boys' backgrounds. He took a little time to talk with them too. Two-and-a half hours later he left, saying: "I can see that the boys have light in their eyes! You must be doing the right thing! We want you to open another program in the Negev for more boys at risk." His evaluation was anecdotal and cheered us, but of course it was not a measure of success. The non-profit organization that runs the innovative and enriching rehabilitation project, with funds that it raises for this purpose, has to measure the success of its efforts. This is clear to Ernst & Young and to our donors. But the government department that funds, inadequately, the boys' full-time care should also be doing just this - qualitatively and quantitatively, not only subjectively. It was a good idea to try to teach a group of non-profits how to manage and evaluate our projects effectively, especially as we all depend on a certain amount of helpful funding from donors who want to see measurable success from their investments in us. I came to the lecture because I wanted to know how I could turn my subjective feeling of success into objective statistics. I certainly came away with some ideas about how to redefine my short-term and intermediate goals and how to design track-able actions in my project. But it isn't only donors who want to see measurable success from their investments. The taxpayer also wants to see such success. It isn't only non-profits who have to prove that they are managing their resources well for the good of society. The government has to do this too. It is easy to target the non-profit managers who are only too keen to accept advice that could perhaps lead to support for their work from a global company. Those who attended the talk are open-minded and highly-motivated people, keen to apply every new idea that could stimulate or enhance our finding a lasting solution to the social problems that concern us. I came home thinking about who else should have attended this presentation. The Welfare Minister, for a start. He is responsible, among his many duties, for spending taxpayers' money on youth-at-risk in full-time care as well as on the many families on welfare. Mr. Herzog and his staff - not only the man who notices the light in a boy's eyes and the man who helps so inadequately to fund the boy's full-time care - could also profit from Mr. Litman's lessons. These government officials, and many others, still need to learn how to align, allocate and manage their resources, and work logically toward their social goals. They still need to learn how to measure success objectively as well as subjectively. Not only the Welfare Minister can learn from Mr. Litman, but others too, such as the Health Minister, the Defense Minister and the Chief of General Staff. But would these men give up an hour of their day, as we did so willingly, to listen and learn?