billions of drops dot com

Observations on the World’s First RFP for Social Impact Bonds

2012-01-10T17:32:00.000-05:00

BIG NEWS FROM DOWN UNDER

The Treasury Department of New South Wales (NSW), Australia, recently issued the world’s first Request For Proposals for a “Social Benefits Bonds Trial.” “SBB” is the latest entry in the continuing search for a commonly-accepted name for Social Impact Bonds (SIB), otherwise referred to (primarily by the U.S. and some state governments) as Pay-for-Success (PFS) Bonds or Contracts. (Thankfully, I won’t be wading into that debate and will use “SIB” here.) As a former government lawyer who has litigated more than $2 billion in public procurement and contracting disputes, I’m happy to report that the NSW Treasury has drafted an excellent RFP that deserves wide consideration.

The RFP offers an incisive approach to structuring the relationships with the SIB intermediary and nonprofit providers, phasing the contract negotiations, setting the general parameters for evaluation of SIB outcomes, and avoiding unintended consequences such as favoring programs that serve “easier” populations and displacing existing funding sources. NSW appreciates the critical differences between traditional government contracts and SIB partnership agreements, and formulates a deliberate and collaborative process for developing promising pilot projects.

(For the benefit of the uninitiated, “procurement” refers to the formal process for soliciting contract proposals and “contracting” refers to the negotiation, drafting and execution of an agreement that results from the procurement process. An RFP is often referred to as a “competitive solicitation.”)

AN OVERVIEW OF THE NSW RFP

As the Treasury Department explains in the RFP, an SBB, just like an SIB, “is a new financial instrument that pays a return to private investors based on the achievement of agreed social outcomes. Under a SBB, an investor provides upfront funds to a partner (non-governmental organisation (NGO) or intermediary) to provide services that will, if successful, reduce future costs to Government. Part of the Government savings are used to repay this investment and provide a reward payment commensurate with the outcomes achieved.”

Like SIBs, “SBBs aim to provide an expanded source of private upfront funding for early intervention and prevention services that mitigate the escalation of social problems,” and that “directly rewards outcomes rather than specifying how they are delivered.” Both instruments assume the benefits of market discipline: “It is expected that investors will require some review of the evidence behind any proposed intervention as well as ongoing monitoring of performance, increasing the likelihood of positive social outcomes.”

Importantly, SBBs and SIBs are also designed to enhance governmental performance management and accountability: “Because SBBs require a sound performance management framework to be established to measure outcomes prior to payments being released, agencies and service providers will need to develop and enhance methodologies and skill sets in program design and outcomes tracking ...” (For additional SIB tools and resources, see the Social Impact Bond Learning Hub hosted by the Nonprofit Finance Fund.)

One central issue the RFP handles well is the role of what NSW calls the “Social Benefit Partner” (SBP), which Harvard Kennedy School Professor Jeffrey Liebman calls the “Social Impact Bond Issuing Organization,” or SIBIO. The RFP defines SBP as:

“The organisation that is [1] responsible for the improvement of the outcome for the intervention cohort, [2] holds an agreement with Government by which they are paid upon measurement of this improvement and [3] issues social benefit bonds to investors.”

Since the SBP is the sole party responsible for the outcomes, holds the contract with the government and issues the SIBs, it follows that the SBP must hire and manage the nonprofit service providers directly:

“The Government intends to sign a SBB Implementation Agreement with the SBP for each pilot. The Government will not have any contractual arrangements with investors or with the service providers (if different [from] the SBP) as part of the pilots. Such arrangements would need to be concluded separately between the SBP and its investors and providers (as relevant), although the Government may assist in the negotiation process as reasonably determined by all the parties and set out in the SBB Development Agreement.”

The RFP also successfully addresses three other concerns often expressed about SIBs. First, the RFP guards against cream-skimming:

“The Government is interested in developing measures that encourage providers to work with the most challenging clients, and avoid perverse incentives. One way this will be done is through measuring performance with regard to all clients who are referred to the intervention, not simply those who engage with the services offered. This removes the incentive for providers to work only with the most motivated clients and avoids measurement bias due to self-selection.”

Second, private investment may not be used to offset current government budgets for social programs:

“SBBs are intended to attract new resources to the delivery of public services. While successful SBBs should reduce the need for future public spending on acute services, SBBs are not intended to replace existing upfront government expenditure.”

Third, while small nonprofits will likely find it difficult to handle the substantial performance demands that SIBs impose, NSW tries to facilitate their participation:

“The Government acknowledges that smaller service delivery contractors may not be able to submit a proposal and the Government is therefore open to a larger organisation or consortium submitting a proposal in relation to these contracts collectively. Any such proposal, however, would need to demonstrate that all affected service delivery contractors support the proposal and that there is no impact on overall service levels.”

KEY PRINCIPLES OF GOVERNMENT PROCUREMENT AND CONTRACTING

Procurement law is designed to promote two general policies: (1) fairness to organizations competing for the contract opportunity and (2) value to taxpayers. The first policy is served by creating a “level playing field” on which all potential responders can compete for the same contract on the same terms. For example, except in special cases, an RFP should not be written to favor a particular organization or an approach used by a particular organization, if the solicitation can be written on more inclusive terms. The second policy is served by encouraging cost-efficient, innovative proposals from capable organizations.

At times, these two policies can be in conflict. Historically, procurement law was very strict about setting an absolutely level playing field, so RFPs were required to set out all the terms of engagement in exhaustive detail, with bidders competing exclusively on the price of the good or service offered. This approach often failed to produce much taxpayer value. (The shortcomings were memorably expressed by the oil-driller “Rockhound” [played by Steve Buscemi] in the movie, Armageddon, as he and his fellow would-be astronauts sat on the launch pad waiting to lift off to keep an asteroid the size of Texas from hitting the Earth: “You know we’re sitting on four million pounds of fuel, one nuclear weapon and a thing that has 270,000 moving parts built by the lowest bidder. Makes you feel good, doesn't it?”)

Over time, the “lowest responsible and eligible bidder” requirement has been considerably liberalized so that firms could offer innovative approaches that the government could not define in detail on its own. But there are limits on how far this more enlightened approach can go without undermining the fairness criterion. For example, procurement law probably would not allow an agency to issue an RFP that said, in effect, “We want to contract with nonprofit organizations to help poor people. Please tell us how you would do that,” as there would be difficult to compare proposals in such a wide-open, apples-to-oranges “competition.”

There’s a lot of room between the two extremes of no flexibility and total flexibility, and the NSW RFP strikes a fair balance between creating a level playing field and inviting innovative approaches to developing SIB pilots.

A PHASED APPROACH

The government has certain irreducible obligations to provide a level playing field, comply with established legal requirements, and protect public safety and vulnerable populations. However, in the novel case of SIBs, it needs to simultaneously promote cross-sector collaboration in order to maximize the chances of attracting private investment capital.

Treasury explicitly structures the procurement process as a way to select partners to negotiate SIB contracts, rather than require them to accept terms imposed by the RFP:

“Since the release of the [government’s first report on SIBs], the NSW Government has conducted further development work on a potential SBB trial, including discussions with a range of potential providers, investors and intermediaries. It is clear from this work that, while there is a high degree of interest in the SBB model and the necessary market conditions may exist in NSW, significant uncertainty will remain until a sustainable data base is established on which to measure improvements in service delivery. This is an evolving market where significant collaborative work between proponents and the Government will be required to develop a viable SBB. The Government is therefore seeking to identify preferred proponents with whom to develop the two pilots through this Request for Proposals (RFP).”

The RFP then lays out a three-phased process:

an RFP which designates two priority areas (out-of-home care and recidivism);
“a due diligence and joint development process to be undertaken between the Government and up to two preferred proponents identified through the RFP process, in respect of up to two potential pilot SBBs”; and
“Contract negotiations with preferred proponents for implementation of up to two pilot SBBs.”

This is an insightful, albeit unavoidably laborious approach that will be carried out through two separate and successive agreements. Before the second phase begins, firms selected based on the RFP submissions will negotiate an “SBB Development Agreement” that will govern the working relationships throughout the third phase, which will then culminate in an “SBB Implementation Agreement.”

Although undeniably cumbersome, this approach accommodates the complexities and uncertainties of SIB development. SIBs have many moving parts, so it is essential to get and keep everyone on the same page. The SBB Development Agreement sets the stage for doing so by addressing such basic matters as:

“the dispute resolution process; payments to the preferred proponents (if any); insurance; the nature of any involvement by the NSW Government in negotiations with investors; ownership of any intellectual property generated during the joint development phase; the process for negotiating and reaching agreement on the SBB Implementation Agreement …; [and] termination and the sharing of information between the preferred proponent and Government.”

Unless these legal, administrative and housekeeping matters are buttoned up at the outset, it might prove too difficult to resolve the more challenging substantive issues to be covered in the SBB Implementation Agreement. But NSW recognizes the fundamental need for collaboration from the outset: “[d]uring the assessment of RFP proposals, the Government will discuss and agree [on] the terms of the SBB Development Agreement with parties being considered for selection as preferred proponents.”

You can appreciate why NSW breaks the contract negotiations into two separate agreements when you see the daunting list of issues “envisaged” in the RFP for the SBB Implementation Agreement:

“details of the target group, location, and referral and entry arrangements; contract duration and any extension provisions; ownership of intellectual property from the pilot; details of baselines, comparison groups and other measurement arrangements; payment triggers; a payment schedule covering all performance scenarios (below baseline, baseline, good performance and over-performance); allocation of risk between parties to the SBB; dispute resolution provisions including a mechanism for resolution of client issues; break clauses for all parties; [and] any options for recontracting at the conclusion of the SBB term details of how the program would be evaluated.”

The RFP does a nice job of balancing the need for a level playing field with progressive collaboration to promote benefits to taxpayers and program beneficiaries. Terms to be negotiated include the nature of the contractual relationships, the capital draw-down requirements, and the investor repayment schedules, subject to the non-negotiable requirement that investors bear the entire risk of loss if the agreed metrics are not met. Treasury is willing to consider financial incentives for SBPs “to share in the possible upside return,” but does require that “the bulk of the financial risk (and potential returns) should reside with investors.” Proposers can also recommend outcome measures and ranges, effect sizes, target cohorts, comparison groups, service locations, payment triggers, and measurement periods.

As SIBs are entirely dependent upon the achievement of defined outcomes, data collection and evaluation are critical success factors. However, government data collection systems are constrained by privacy laws, so it is noteworthy that the RFP states that “the Government will provide support in constructing and implementing the measurement system, including thorough access to administrative data.” Indeed, proposers are encouraged to “indicate any requirements of NSW Government agencies in the proposed identification, screening and referral process,” which could be crucial in terms of access to data about program participants. (Presumably, such measures could include removing all personally-identifying information from program participant data provided to intermediaries.)

NSW anticipates that the term of the pilots will be five to eight years, but recognizes that “SBB terms must balance the time needed to demonstrate maintenance of statistically significant effect sizes with investors’ interest in receiving a return within a reasonable timeframe.” Five- to eight-year terms are probably reasonable, if we assume that services will be delivered to multiple treatment cohorts of 6-18 months each, and that audits/evaluations of 12-24 months will follow each treatment period.

The RFP also does a good job of establishing a useful and feasible framework for measurement and evaluation. The primary requirement is that “an independent auditor (or auditors) will be commissioned and will be responsible for verifying whether outcome targets are met during the life of the SBB.” Specifically, “the auditors will approve and monitor the outcome targets, measurement approach and payment triggers for the two pilots, … [and] determine whether the agreed targets have been met and therefore whether government payments should be released.” This is a sensible approach because the outcome metrics should be expressed in unambiguous and directly measurable terms, allowing an auditor to essentially verify the data (including any backup records) to determine whether the contract conditions have been satisfied.

One area of possible concern, however, is that “the Government’s preferred measurement methodology is randomised control trial (RCT), although other approaches may be considered if RCT is not feasible or appropriate for a particular intervention.” RCTs can be a useful tool for the selection of evidence-based intervention models, although other approaches, such as “quasi-experimental” designs using matched comparison groups, have been valuable in demonstrating the effectiveness of innovative prevention programs such as permanent supportive housing. In addition, RCTs often will be infeasible or inappropriate for evaluating SIBs designed to serve the kinds of “most challenging clients” the RFP has in mind. For example, it would be extremely difficult (at best) to make random selections among chronically homeless individuals with severe mental health problems, or high-risk youth aging out of foster care, and it would be even more onerous to track such members of the comparison group who did not receive program services. In some cases, the use of RCTs might deny effective preventive services to the people who need them the most.

The RFP also provides a useful explanation of its thinking about out-of-home care and recidivism. It offers extensive discussion and analysis of each of the proposed pilot areas “that can be used in the development of proposals,” including detailed examples of SIB models based on this information. Importantly, the RFP states that “the examples are purely illustrative and the potential financial benefits have not been tested against any specific interventions – this aspect is left entirely to proponents.”

In sum, the RFP assigns the right responsibilities to the intermediary, provides the autonomy necessary to meet those responsibilities, and leaves essential contract terms open for negotiation so that the parties can jointly structure a SIB that investors will accept and intermediaries can implement with confidence. The “Service area information” section provides reasonable assumptions about participant cohorts, outcomes and measurement, and “indicative benefits” that help bring newbies up to speed and level the playing field for comparing proposals. This information also offers a practical way to get the ball rolling in a nascent market that, as yet, has no benchmarks from the field.

CONCLUSION

The particulars of drafting RFPs often depend on local market conditions, such as characteristics of the service provider industry, legally-mandated and customary procurement and contracting practices, and economic and social circumstances that give rise to the need for services that the government does not provide directly. In the case of SIBs, there will surely be a range of viable approaches that state and federal agencies will take to soliciting proposals, with no clearly superior way to bring this novel financial instrument to life. That being said, the NSW RFP is a worthy starting point.

Australia’s Treasury Department has done the embryonic market for Social Impact Bonds a great service by developing a procurement and contracting process that balances fairness, innovation and collaboration. While no government can abdicate its responsibilities on matters of legal compliance, protecting public safety, and serving vulnerable populations, it is in the government’s own interest to encourage the private and social sectors to take a seat at the drawing board to help devise more effective ways to finance proven prevention programs. NSW’s efforts show that the procurement community, broadly defined, has an important role to play in bringing SIBs to market.

The Burdens and Benefits of Evaluation

2011-02-22T14:02:00.010-05:00

William Schambra, Director of the Bradley Center for Philanthropy and Civic Renewal at the Hudson Institute, makes a fair point in his article, “Measurement is a futile way to approach grant making,” when he laments that “ever more elaborate schemes for ensuring measurable outcomes” have imposed “a substantial and growing burden of measurement for the nonprofit world.” And he’s right, too, when he observes that “even when measurements have been duly gathered, research shows that they have little impact on actual grant making, not affecting the amount of money spent on a program.” But he overshoots the mark when he acknowledges, with refreshing candor, that “I happen to believe that measurement is finally a futile way to approach grant making.” Useful evaluations of nonprofit performance are not a one-size-fits-all proposition, and the costs and value of assessing effectiveness can and should be calibrated as befits the purpose of evaluation: to get more funding with less effort to more effective organizations.

Mr. Schambra (who wrote a rather nice review of my book for the Wall Street Journal and hosted a panel discussion with the provocative title, “Do We Need a Nonprofit Capital Market,” to discuss it) leads a distinguished policy center that “aims to encourage foundations and charitable donors to direct more resources toward support of small, local, often faith-based grassroots associations that are the heart of a vital civil society.” So he speaks from experience when he says that “small, grass-roots nonprofits -- which are so often a key source of innovation -- are automatically frozen out of money by the burden of measurement.”

Alas, measurement and evaluation are expensive and difficult, and the small charities of which Mr. Schambra speaks will almost always have more vital work to do with their limited resources. Nearly all of the work going on to advance nonprofit evaluation (including such efforts as the Social Innovation Fund and the Social Impact Bonds that my organization, Social Finance, Inc., is working on) are aimed at larger, more mature nonprofits. (As I’ve written previously, one notable exception is Social Velocity, ably led by Nell Edgington, which does impressive work bringing growth capital to small nonprofits.)

[While I’m on the subject, could we please stop referring to randomized-control trials as the “gold standard” of evaluation? That’s like saying Mt. Everest is the gold standard of mountain climbing, when 99.99% of climbers will never see, much less climb, Mt. Everest.]

But just because it’s hard to justify imposing the burdens of robust evaluation on small charities doesn’t compel the same conclusion for larger organizations. In fact, Mr. Schambra identifies reasonable criteria for legitimate evaluation that aptly describe where I believe the social sector is heading. To his credit, he notes that “the burden of measurement might be endurable were we confident that all those numbers we were collecting were somehow adding up to a coherent science of grant making.” He welcomes an arrangement in which funders and nonprofits “decide jointly on a simple, coherent, user-friendly system to which we can both pay attention, which will prevail over bureaucratic inertia and political connections, and which will feed into a serious body of knowledge.”

I think Mr. Schambra has fairly described some encouraging developments that share a common purpose of trying to make measurement and evaluation more useful and more consequential. That is just as it should be: for all the reason he gives, the burden of proof must rest entirely on funders and evaluators, not on beleaguered nonprofits.

Examples abound. FSG Social Impact Advisors has taken the lead in two areas relating to Mr. Schambra’s concerns: “shared measurement systems” designed “to evaluate ... impact across multiple grants and stakeholders” and action-oriented approaches that offer “a fundamental transition in the way foundations use evaluation.” The Independent Sector, BBB Wise Giving Alliance, and GuideStar USA have launched a “Charting Impact” initiative grounded in “five simple but powerful questions” that give nonprofits “a shared vehicle for concisely conveying your plans and progress to key stakeholders.” Root Cause’s “Social Impact Research” project (on which I consulted some time ago) “aggregates, analyzes, and disseminates information to help social impact investors identify and support the most effective, efficient, and sustainable organizations working to solve social problems.” Charity Navigator has launched “Charity Navigator 2.0” (another consulting project) to transform its widely-used but much-maligned star-rating system into a more informative tool for intelligent giving.

These are not the kinds of “ephemeral metric fads” that Mr. Schambra derides. Rather, they are serious and sustained efforts to shift some, but by no means all or even most, philanthropic giving in a more performance-driven direction, one that actually would significantly affect the amount of funding that the most effective nonprofits attract.

In his seminal article, “Money to Grown On,” William Foster of the Bridgespan Group identified “a small but growing number of organizations ... involved in providing growth capital” and argued that “donors can learn how to scout out and grow the best nonprofits ... [and] certain nonprofits can ... learn how to attract cash for expansion.” These organizations (which my book calls “growth-ready mid-caps”) meet all of the following seven characteristics: they address a critical need; they have strong leadership; they have strategic clarity; their programs are demonstrated successes; their programs are cost-effective; they have grown successfully; and they have a sustainable funding model.

Most assuredly, these are not Mr. Schambra’s “small, grass-roots nonprofits,” whose contributions to civic life he rightly celebrates. But they are the kinds of organizations that have important potential for “scaling what works,” which has attracted the Social Innovation Fund and a coalition of more than 20 funders collaborating under the flag of Grantmakers for Effective Organizations. All of these efforts are aiming at developing evaluation systems that are not more trouble than they’re worth and that could actually influence the flow of money to nonprofits that are producing the greatest social impacts.

Evaluation critics aren’t far off the mark when they complain about the long-standing imbalance between the large costs and modest benefits of evaluation. But the fulcrum is shifting because we have an opportunity to leverage the tremendous social innovation the sector is generating with new kinds of financing that are specifically designed to amplify impact and support growth. While Mr. Schambra has a point when he says “devotion to measurable outcomes is hardly new,” the opportunity to combine more pragmatic and actionable evaluation tools with an emerging interest by “social impact investors” in growth-capital funding is new. Without evaluation, we can’t hope to convince impact-minded funders to make the kinds of larger, longer and less restricted grants that are needed to overcome the capital fragmentation that holds back social progress.

Mr. Schambra fears that “the burden of futile and pointless measurement will only continue to grow.” In the broadest sense, I’m not prepared to disagree with him. I do believe, however, that a strong and promising counterpoint must emerge if the sector hopes to move the needle of social change, and I’m quite optimistic that such a movement is well underway.

2011: The Year Philanthropy Starts to Become a “Long-Term Solution”

2010-12-24T14:43:00.000-05:00

An apparent disagreement between two of the sharpest minds in philanthropy is just too tempting to resist. Herewith, an unsolicited response to Matthew Bishop (co-author of Philanthrocapitalism and The Road From Ruin, and NY Bureau Chief of The Economist) and Lucy Bernholz (founding President of Blueprint Research & Design and blogger extraordinaire at Philanthropy 2173) on the subjects of predicting trends in philanthropy and the relationship of private giving to public policy.

Here’s the set up: Lucy recently blogged about a video of Matthew offering some predictions for 2011, which include (as Lucy summarized) a “rise in impact investing, ... boom year for education giving, down turn in government aid, and increased awareness of social capital markets.” While I would venture to say that he’s hardly going out on a limb here, still, it’s good to hear someone as perspicacious as Matthew validate what many of us are expecting and planning on for the year ahead.

But what’s noteworthy to me is Lucy’s response (with my emphases added), which, as I read it, seems to suggest that Matthew is all wet:

Most interesting, in my opinion, is Bishop’s sense there will be more attention on policy issues, not just from big givers but also from the rest of us.   I'd like to point out that the very fact that Bishop highlights education and maternal health as two "hot areas" for giving in 2011 is indicative of the limits of philanthropy as a solution. Global solutions don't lend themselves to "annual hot lists" - as I noted here. This blog received a comment from a reader that said: “Planned giving and philanthropic leadership are the need of the hour as many countries are crisis-ridden."

My response "... I think it is a fallacy to equate anything you would describe as “crisis ridden” with “philanthropy” as a solution. Philanthropy is – by design – episodic, donor directed, temporal, fragmented, decentralized and disaggregated. Not what any people, society, institution, community should expect to be responsible in “crisis ridden” situations."

Those same characteristics shape the kind of impact philanthropy can possibly have on ongoing issues such as education and maternal health or whatever the 2012 issues of the year will be. We don't help anyone by pretending otherwise. It's not to say that philanthropy can't make a difference. It is to say that we need to develop philanthropic strategies that draw from the strengths of philanthropy and don't burden it with being a long term, equitable, prioritized source of funding or attention.

The only word that captures my reaction to Lucy’s reaction is one that Matthew might use: gobsmacked. I’m utterly astounded not merely because I so strongly believe that transforming at least part of philanthropy to become “a long term, equitable, prioritized source of funding [and] attention” is precisely what we should try to do next year, but because I would have thought that Lucy Esmerelda* Bernholz would be the last person on Earth to suggest otherwise. [*I have no idea what Lucy’s middle name is ...]

Let’s put this in context. A recent installment of the New York Time’s “Neediest Cases” series, entitled “When Children Are Caught in the Cycle of Poverty,” illustrates what I’ve called “$100 million problems,” social problems that are so pervasive, incapacitating and intractable that much larger sums than usual must be marshaled to develop and pilot scalable responses:

The economic collapse has taken a toll on vast segments of society, but it has affected some groups disproportionately. Among those are children. The Institute for Children, Poverty and Homelessness reports that nationwide, 1.35 million children are homeless. Most of them are black or Latino. Children make up a quarter of the nation’s population, but account for 36 percent of all people in poverty, according to a report from the National Center for Children in Poverty, at Columbia University. In New York City, 30 percent of children are living in poverty. One out of every five children relies on local food banks or pantries for sustenance, and of these children, 79 percent rely on the National School Lunch Program. Poverty stymies performance in school and negatively affects mental and physical health, experts say. Poor children have higher rates of asthma, are more likely to suffer a higher rate of cognitive delays and developmental disorders. Absent intervention, these children will face great difficulty in transcending the disadvantages of their early lives and, as adults, are likely to perpetuate a cycle of poverty that has consumed generations in areas like East New York, Brooklyn; Jamaica, Queens; Morrisania in the Bronx; East Harlem; and Port Richmond on Staten Island. Such an outcome is not acceptable to advocates like Richard R. Buery Jr., president and chief executive of the Children’s Aid Society, who said, “Those who love our country, and believe in its ideals, cannot be satisfied until the promise of equal opportunity is made true for all of our children.”

Now, we have any number of outstanding charitable organizations that have developed innovative solutions to just these kinds of problems, many of which have achieved significant levels of incremental growth over the course of two decades or longer. But we simply cannot make meaningful headway against the massive scope of these problems -- 1.35 million homeless children, 36% in poverty, one-fifth experiencing food insecurity, with all the attendant impairments of health and educational and economic opportunity -- because we don’t know how to make those innovations much, much, MUCH more widely available.

In my book, I argue that, as it stands today, philanthropy can’t increase the availability of proven social innovations by orders of magnitude. Much of my thinking about this was informed by none other than Lucy Cleopatra Bernholz. Lucy wrote the very first and extremely insightful book on this critical subject way back in 2004, called Creating Philanthropic Capital Markets: The Deliberate Evolution (Wiley), which, by the way, hasn’t become outdated in the least.

Early on in her book, Lucy argues that “the goal of a new philanthropic system should be to demonstrably accelerate the effective application of private resources to achieve public good.” So she asks, “How do we maximize the impact of hundreds of thousands of individual charitable acts and experience the full potential that private giving can bring to our shared society?” Good question, to which she provides an equally good answer:

“The greatest professional challenge in philanthropy and the mark of success is the ability to attract other people’s money to an issue and apply the joint resources of many to make change happens.”  
“The book argues for the potential -- and the necessity -- for philanthropy to reorganize into a more rational system for social good.”

I like where this is going. So what will it take?

“If elements of the industry were deliberately designed, streamlined, creatively conjoined, and aggregated we could indeed build new philanthropic capital markets that could significantly improve the quality of community life.”

Any difficulties expected?

“The strength of American philanthropy is its diversity and personal nature. Its weakness is its dispersed size and diminishing value in the face of ever-widening wealth gaps and public revenue shortfalls.”  
“If we intend to make philanthropy a stronger, more vital participant in social health and problem solving, the most efficient places to focus are on those elements of the industry that broadly affect many individual players. Simply put, the industry is too dispersed and deliberately isolated to effect much influence on it by moving from organization to organization.”

So philanthropy needs to be “redesigned” and “aggregated” to accomplish “greater social good”; as it stands now, it’s too “dispersed” and “isolated” to accomplish its potential. I couldn’t agree more.

But then who’s this other person arguing that philanthropy was “designed” to be “fragmented, decentralized and disaggregated,” so we shouldn’t “pretend” that philanthropy can be a “long term, equitable, prioritized source of funding or attention?” Is there an “old Lucy” and a “new Lucy”?

I think my confusion began when Caroline Hartnell, editor of Alliance magazine (on whose Editorial Advisory Board Lucy sits), offered this on Lucy’s behalf:

No one should have to rely for their long-term basic needs on something that is by its very nature ‘episodic, donor directed, temporal, fragmented, decentralized and disaggregated’. No one can force foundations or philanthropists to act in a particular way over the long term. If we want to do that, we can tax the wealthy more. Social justice philanthropy doesn’t aim to meet people’s basic needs but to support marginalized groups to change the conditions that make them marginalized and to ensure that their long-term needs are met by the state as a matter of right.

I respectfully disagree. The charitable sector has long seen government -- naively, in my view -- as an “exit” or “take-out” strategy in which nonprofits develop an important social innovation, which the public sector then adopts and scales with tax dollars. What’s missing, I believe is an appreciation of just how profoundly the public sector has been recast and its role permanently diminished.

Not only has the legislative process become almost completely dysfunctional, but almost everyone agrees that our national and state budget and debt structure have become unsustainable, with severe cutbacks in governmental expenditures as far as the eye can see. There are no realistic prospects for significant expansions of government programs to address $100 million problems in education, healthcare, workforce development, housing, food, or any of the many other areas of significant decline. Lucy seems to agree in both her book and her Blueprint 2011:

“Public funds for nonprofits at both federal and state levels will be down in 2011; many of the public sources of funds for nonprofits in 2009 [including the American Recovery and Reinvestment Act and the Department of Education’s Investing In Education fund] were one-time shots and state budgets have been decimated.” 
“We are in the midst of a public revenue crisis at every level of governance that will not allow philanthropic assets to remain unaccountable or disconnected.”  
“Changes in public funding practices, primarily the devolution of funding and decision making from the federal government to state and local jurisdictions, constitute the fourth driver of the [philanthropy] industry.”

I see decades of governmental retrenchment ahead that cannot possibly cope with the array of debilitating social problems we already face. If government remains the primary institutional response to those problems, we will not make sustained progress in overcoming them. Other sectors, including business and nonprofits, have to take up the slack.

A more plausible approach, presaged by the Social Innovation Fund and NYC Deputy Mayor Stephen Goldsmith’s Governing by Network model, is forming new public-private partnerships (such as Promise Neighborhoods based on the Harlem Children’s Zone) to share ongoing responsibility and funding for the development of new initiatives that straddle boundaries between sectors by combining the strengths of both. Old Lucy seemed to agree:

“A stronger philanthropic system will integrate private funds with public strategies....” 
“There is no universal equation for the best relationship between the marketplace, government, and nonprofit providers -- the challenge is making the mix work. As the balance of players -- commercial, nonprofit, and public -- shifts, the public benefit sector presents a shocking new environment for philanthropic action.”  
“Philanthropy is also changing as a result of new public-private partnerships. There are many catalysts for those partnerships but the devolution of public budgets and decision making from the federal to the local level is a major reason for their growing popularity.”  
“Co-production of social services may become the norm, as municipal governments scale back and individual residents and philanthropic entities band together to provide core community services such as recycling, safety, and recreational services.”

This evolution is happening right before our eyes. Consider some of the more significant developments of 2010 in nonprofit growth capital markets and scaling social impact:

Three grantees of the Social Innovation Fund recently awarded the first round of subgrants to support 61 community-based nonprofit organizations working to provide workforce training, job placement, financial literacy services, and other resources to help over 32,000 individuals and families find jobs, reduce their debt, gain financial literacy and build assets. 
In June, the Social Impact Exchange held its 2010 Inaugural Conference on Scaling, where more than 450 funders, investors, high net-worth individuals, philanthropy advisors, academics and nonprofit leaders focused entirely on planning, evaluating, and financing the scaling of high impact, nonprofit organizations. In December, the Exchange announced several follow-up capital projects to be developed in time for the next annual event, including the formation of collaborative working groups focusing on Social Impact Bonds, donor-advised funds, private banks and financial institutions, and multi-family offices. 
Social Finance in London has launched the first Social Impact Bond to raise private investments for early intervention programs to reduce downstream governmental expenditures resulting from prison recidivism. The Centre for Social Impact in New South Wales, Australia has announced plans to develop a similar initiative in areas such as juvenile justice, mental health or services assisting young families at risk. 
Nine NFF Capital Partners’ “philanthropic equity” campaigns -- “the type of capital needed to explore better business models, scale impact, and create lasting change” -- have grown average annual program delivery by a factor of 3.1x, with a compound annual growth rate of 57%; annual business model revenue for these nine organizations has doubled, with a compound annual growth of 36%. 
SeaChange Capital Partners launched “transformational fundings” totaling more than $11 million for outstanding nonprofits that have “the opportunity, capacity, and ambition to increase their impact significantly on behalf of low-income young people in the United States.” 
The third annual Social Capital Markets (SOCAP) conference added a “Tactical Philanthropy” track curated by Sean Stannard-Stockton to “examine the way in which philanthropy is an integrated part of the social capital markets, not a separate activity.” 
Charity Navigator, Philanthropedia, Root Cause, GiveWell, New Philanthropy Capital, Keystone Accountability, Great Nonprofits, Guidestar, Independent Sector, BBB Wise Giving Alliance, and the Alliance for Effective Social Investing are developing and collaborating on various initiatives to advance performance-based philanthropy and measuring social impact. 
Social Venture Partners International launched its first “mezzanine fund” to “take investees to national scale.”

I could go on, but it appears to me that we’re on the verge of something seismic. In fact, over the last few weeks, I’ve asked quite a few people who’s views I respect whether they think (as I do) that a significant shift is taking place in the nonprofit capital marketplace and that major funders and intermediaries are ready to support growth capital investments and scaling initiatives. While no one thinks that we’ve arrived at the promised land just yet, they all agreed that a lot more people seem to believe that such a place exists and that a consensus is emerging about the most likely ways to get there. Having talked the talk for several years now, the possibility of walking the walk feels imminent.

Philanthropy is experiencing the kind of periodic “paradigm” changes that Thomas Kuhn described in which radically new approaches to social and scientific problems finally displace long-accepted explanations (such as belief that the sun revolved around the Earth) that clearly no longer work. The idea that philanthropy is supposed to be “episodic, donor directed, temporal, fragmented, decentralized and disaggregated” has been eroding for some time, and the realization has taken hold that “new philanthropic capital markets ... could significantly improve the quality of community life,” if, but only if “elements of the industry were deliberately designed, streamlined, creatively conjoined, and aggregated.” When we finally acknowledge (as the old Lucy urged us to do) that declining social mobility cannot be corrected by diminishing public revenues, and that the nonprofit sector has become impressively good at spawning effective social innovation, we must accept Lucy’s conclusion that the time has surely come “to demonstrably accelerate the effective application of private resources to achieve public good.”

Philanthropy as “long term, equitable, prioritized source of funding [and] attention,” here we come. But who knows? As Yogi Berra said, “It's tough to make predictions, especially about the future.”

I Beg to Differ

2010-11-30T15:42:00.000-05:00

I take a back seat to no one in my admiration of the Nonprofit Finance Fund, so it pains me to dissent from Rebecca Thomas’s thoughtful article, "New Ways to Rate Charities Don’t Help Donors Make Smart Bets," published in the Chronicle of Philanthropy, but dissent I must. The simple answer to her important questions – “So why the rush to rate and rank? Why not provide information and let donors decide?” – is that donors aren’t willing to wade through unprocessed information about nonprofits to make more informed decisions about which charities to support. Given the choice between making steady improvements in sites like Charity Navigator and continuing to leave donors without any meaningful guidance about which nonprofits are most effective, I’ll take better rankings every time.

First, a disclosure: I've been both a consultant to Charity Navigator and a member of its Advisory Panel that's helping CN enhance its rating methodology.

Second, the disconnection between performance and funding is one of the most significant problems facing the nonprofit sector. We urge charities to undertake the difficult and expensive work of measuring and managing their performance, knowing all the while that doing so is likely to have little or no effect on donor support. Why? Because, as Hope Consulting’s research shows, donors don’t look for or use performance information. Why? Because it’s not available in a form that’s useful or convenient to them. Lecturing donors about how they “should” evaluate charities, as so many articles (particularly at this time of year) do, is no way to help donors become more thoughtful philanthropists.

In a recent New York Times column, Nicholas Kristof offered what seemed like sensible advice for holiday charitable giving: “donations could accomplish far more if people thought through their philanthropy, did more research, and made fewer, bigger contributions instead of many small ones that are expensive to handle.” He’s right, of course, but his sound advice is virtually impossible to follow. For most donors, there’s no simple or direct way to find which charities do the most good. As a result, charities that don’t accomplish measurable objectives can still attract funding from uninformed consumers simply by telling engaging anecdotes.

I don’t agree that “it is far from clear that the new systems are any better than the ones they seek to replace.” Clearly, Charity Navigator’s evolving methodology is a major advancement in thoughtful analysis. Another pioneering site, Philanthropedia, is presenting donors with a tremendously helpful tool. And I would argue that sites like these are doing exactly what Ms. Thomas suggests they do: “provide donors with a truly meaningful blend of information about an organization’s leadership, direction, revenue model, capital needs, and program results.”

Critics of new rating sites fail to recognize the importance of providing mass-market information tools. Millions of donors are donating billions of dollars to more than 1.5 million nonprofit organizations with almost no idea of how well the charities are run or what they accomplish. In such a crowded market, highly-effective nonprofits are not rewarded for strong performance because, for all practical purposes, donors have no way to find such organizations. The haystack is too big and the needles are too few.

Respectfully, it is no answer to say that donors should “take the time to do a comprehensive analysis of the context, risk, and opportunities facing each nonprofit.” Perhaps they should, but they never have and they never will. Nor can we accept that “a sophisticated consumer will look beyond a simple rating and ask data-driven questions about a broad range of ingredients that lead to success in achieving an organization’s mission.” There are no such consumers. Foundation program officers and the skilled professionals at places like GiveWell, SeaChange and New Profit certainly go to those lengths, but ordinary donors who provide 75% of the total donations in this country do not.

Ms. Thomas makes legitimate points about selection bias and disclosing expert affiliations. But I think she goes too far in claiming that new standards of “cost effectiveness” and “financial sustainability” are “arbitrary, inconsistent and misleading.” They’re not perfect and they’re not as rigorous as full-blown due diligence, but they’re vast improvements over the information donors have now. The added value of the new ratings far outweighs their shortcomings.

Ms. Thomas urges Chronicle readers “to be mindful of the many highly effective nonprofit groups we may overlook in the process.” The fact is that existing information about nonprofits systematically overlooks virtually all highly-effective charities. For the first time, the emerging wave of new rating sites is likely to make it much easier for donors to find and fund charities that they think produce the most social impact. If so, performance-focused nonprofits will finally have genuine financial incentives to publish meaningful and reliable information about their actual accomplishments. And donors can decide for themselves whether they wish to continue to support charities that are unable or unwilling to do so.

SIF and the Office of Inspector General: Time to Chill

2010-11-12T13:14:00.000-05:00

Before we jump the gun once again, perhaps it would be a good idea to understand what the OIG does first.

The Nonprofit Quarterly, which stirred up unjustified hysteria about the supposed lack of “transparency” in the Social Innovation Fund’s selection of intermediary grantmakers (see my previous posts here and here), is now reporting that the OIG at the Corporation for National & Community Service (CNCS) is conducting an “investigation” of the selection process. NPQ claims that their previous stories “resulted in the publishing of some core [SIF] documents.” As to “the results of the investigation,” NPQ now piously expresses the “hope they will also be published.”

Informed readers will recall that NPQ made lurid claims about potential “conflicts of interest” without even acknowledging that the responsible official made complete written disclosures about his prior connections with SIF applicants and that he recused himself completely from any and all considerations of their applications. Nor did NPQ report that CNCS indicated beforehand that it intended to publicly post selection process documents as soon as it had time to do so shortly after completing a long and labor-intensive review period.

Now NPQ is characterizing OIG’s review as an “investigation” and calling for the results to be made public. Once again, it appears that NPQ doesn’t understand how this process works: what OIG is doing probably isn’t an investigation in the sense that NPQ is implying, and all OIG reviews are required by law to be made public. Remember that when NPQ claims credit later on.

As someone who practiced federal and state constitutional and administrative law for more than 20 years, let me offer some perspective on this latest development. My take isn’t as exciting as NPQ’s, but it is better informed.

By law, some 73 federal (and most state) agencies have an OIG. As explained in the Inspector General Act of 1978, the purpose of an OIG is “to create independent and objective units ... to conduct and supervise audits and investigations relating to the programs and operations” of those agencies. Right away, you can see that the OIG has two separate and quite different functions: “audits” and “investigations.” This is confirmed by Section 3(d), which states that:

Each Inspector General shall, in accordance with applicable laws and regulations governing the civil service—

(1) appoint an Assistant Inspector General for Auditing who shall have the responsibility for supervising the performance of auditing activities relating to programs and operations of the establishment, and

(2) appoint an Assistant Inspector General for Investigations who shall have the responsibility for supervising the performance of investigative activities relating to such programs and operations.

So OIG’s divide their activities between “auditing activities” and “investigative activities.” The distinction is extremely important: an investigation is an adversarial proceeding looking into possible violations of law or contractual obligations, while an audit is a non-adversarial process to improve program management and efficiency. Which one is this? I can’t say for sure, but the email I received from Paul L. Carttar, SIF Director (yes, the same dodgy character whom NPQ smeared before), makes it look like an audit:

Dear Expert Reviewer,

To ensure that we are building the most effective selection process possible, we have asked the Corporation for National and Community Service’s Office of the Inspector General (OIG) to evaluate the fairness and consistency of the grant review procedures that we used earlier this year to award the inaugural set of grants from the Social Innovation Fund.

If this were an investigation, SIF might not even know about it, and it certainly wouldn’t be allowed to go around telling everyone about it. Given that (1) CNCS asked the OIG to conduct the review for the purpose of (2) ensuring the effectiveness of the selection process, this sure doesn’t look like CNCS might be in the kind of trouble that NPQ is implying.

Federal and state agencies created OIGs because they wanted government agencies to have independent audit and investigation expertise. Government programs like SIF are complex and expensive, and it takes particular skills and dedicated resources to make sure they’re being run with integrity and efficiency. By asking OIG to review the SIF selection process, CNCS is taking advantage of resources provided by federal law for the purpose of helping the Corporation do its job as effectively as possible.

NPQ (and others) caused a lot of precipitous and unnecessary anxiety last time around because they didn’t bother to read, much less understand, publicly available information about the relevant transparency requirements that governed CNCS’s work or the procedures that SIF had followed. Rather than jump to unjustified conclusions again, let’s consider the possibility, strange as it might seem, that this might be yet another case where a responsible government agency is trying to do the right thing in the way that it manages taxpayer money.

When Aggregate Demand Met Aggregate Supply

2010-10-08T15:05:00.000-04:00

Here at the SOCAP10 conference, I'm struck by the number of deals that want to happen, are trying to happen, and, for all I know, may even be happening. I suspect, however, from the strained looks on the faces of many attendees, that the wannabes far outnumber the consummated transactions.

One connection that isn't going to come together anytime soon is between aggregate demand platforms such as the Growth Philanthropy Network's Social Impact Exchange -- a clearinghouse of some 2,500 cross-sector members presenting carefully vetted investment opportunities in high-performing, growth-ready, scalable nonprofit organizations -- and aggregate supply platforms managed by national donor advised funds (DAF) like Schwab Charitable Fund -- which, according to its Web site, "has issued over 270,000 grants to 44,000 public charities - a total of close to $1.5 billion." GPN's Alex Rossides and Schwab's Kim Wright Violich led a discussion at SOCAP10 among a circle of nonprofits, funders and investors on the subject of "The Future of Aggregate Demand Platforms." It seems to me like an obvious match waiting to happen, one that could address the social sector's desperate need for more effective capital allocation, but if Schwab's perspective is emblematic of the industry, and I believe it is, the DAFs apparently don't see it that way.

DAFs are in the aggregate supply business, and they have done an outstanding job in making it so simple and inexpensive for donors of all sizes and temperaments to take advantage of charitable tax deductions that they have been able to amass billions of dollars of assets in what are effectively mini-foundations for tens of thousands of retail customers. The DAFs have encouraged the formation of this sizable pool of money by simplifying paperwork and reducing cost, and tens of thousands of nonprofits have benefited from the ready availability and direct distribution of funds from these massive platforms.

Unfortunately, DAF innovation is apparently confined to the core business of aggregating the supply of donor-managed funds, with little or none left over to think aggregation on the demand side, that is, to serve as the flagship customer base for aggregate demand platforms like the Social Impact Exchange.

For perfectly understandable reasons, DAFs have always been ruthlessly agnostic (my term, not theirs) about the choices their customers make about which charities they want to support. While this is a defining characteristic of the DAF business model, without which many donors probably would not have felt comfortable parking their funds and creating a mass-market donation vehicle of such remarkable size, DAFs now seem disinclined to providing customers with even neutral information about the new wave of tools the nonprofit capital market is finally producing to enable more intelligent giving.

That is not to say that DAFs don't get it, because they clearly do. Kim volunteered that improving the allocation of capital is the most important challenge facing the nonprofit sector, and she knows that the social sector will not advance social progress unless more money finds its way to more effective nonprofits. But even though Schwab says it "keeps a close eye" on developments in the burgeoning market of aggregate demand platforms like Alex's, and Kim sits on the newly-formed advisory council supporting research on what drives charitable giving (funded by Gates, Hewlett and LiquidNet), the DAFs don't seem ready to flag these opportunities for their customers.

Creating a gigantic pool of charitable donations is no small accomplishment, but it seems a shame that the funds do not see themselves as having a role in guiding the deployment of the funds they've accumulated so brilliantly. Even if we only focus on charitable donations that are not already committed to specific organizations such as colleges, hospitals and arts institutions with which donors have strong loyalty-based relationships, philanthropy has tens of billions of dollars of essentially "dumb money" which stumbles around blindly looking for places to rest. Much of that money is "in play," in the sense that its final destination is potentially influenceable. The coordinated movement of that money to more rather than less effective nonprofits represents the untapped potential of informed giving. As the Hewlett Foundation put it a recent whitepaper, "Reallocating just 10 percent of the current $300 billion annual fund flow to the best performers would have a similar effect as raising billions in new funds—with nowhere near the same cost in fundraising time and energy."

I understand why, as Kim said, the DAFs don't want to build intelligent-giving platforms themselves, but they don't have to. In the past, there was no way for dumb money to get smart, because nonprofits did not provide meaningful information about how well they were run or what they accomplished. But GPN's site is far from the only platform in the business of aggregating donor demand for growth-ready nonprofits, with the advent of actionable investment and evaluation resources from Charity Navigator, Philanthropedia, New Philanthropy Capital, Root Cause, GiveWell, GreatNonprofits, and many others. As a Hewlett-funded study of online giving platforms shows, the problem we face today is there are too many aggregate-demand platforms, none of which has the horsepower to provide donors with the kinds of information they need to easily find and fund more effective nonprofits.

As a result of all this activity in measurement and evaluation, what once was an impossible task -- making informed judgments about which nonprofits can make best use of donors' generosity -- is on the verge of becoming commonplace. The "scaling what works" movement is well underway and it is ready, willing and able to move money to nonprofits that can produce the most social impact. But the transformation will not take place unless and until major industry players on the supply side -- such as national DAFs -- introduce their thousands of highly-engaged customers to the demand aggregators. Performance-based philanthropy needs the troops that DAFs have been recruiting.

Far from interfering with donor choice or favoring some nonprofits over others, as the DAFs seem to fear, the demand-side platforms are positioned to be ideal partners for empowering donors who want to maximize the social impact their donations produce. Rather than join in as intermediaries on their customers' behalf, the DAFs seem to have made a conscious decision to sit on the sidelines just when the market is on the verge of taking off.

Additional research on donor motivation makes sense, and capacity-building grants are always appreciated. But it does not make sense to defer building out a few of the leading platforms so the market mechanisms will be ready when we need them, and the most precious resource that these emerging marketplaces need is the customers base the DAFs have been building. Kim and Alex and their respective peers in the evolving but still siloed worlds of aggregate supply and aggregate demand platforms are plenty smart enough to find ways of creating additional value for their respective customers by the simple act of introducing them to each other.

The miracle of markets depends upon the mediated interactions of sufficient numbers of diverse buyers and sellers. The nonprofit capital market works the same way, with donors acting as consumers of the valuable services the nonprofits produce. Performance-based philanthropy is an organized mechanism by which highly-effective nonprofits "sell" social impact to impact-maximizing donors. Those donors are searching for the very thing that aggregate-demand platforms bring together: a ready supply of high-impact nonprofits. But unless the markets attract donor-customers, the nonprofit-sellers have no incentive to show up either.

This isn't a chicken-and-egg problem because both sides of the market already exist. They just haven't been introduced to each other. As yet, there's no philanthropy mall where lots of choosy donors know they can shop for lots of quality nonprofits. But surely Schwab and other DAFs have large numbers of retail customers who would love to shop at the aggregate demand platforms like the Social Impact Exchange, if only they knew it was there.

So what's holding back the formation of this marketplace? It's not ignorance, it's not indifference, and it's not lack of resources. I don't really know what it is. But I do know this: it's not happening, at least not yet.

How Mighty Oaks From Little Acorns Grow

2010-10-01T15:15:00.000-04:00

The chaotic nonprofit capital marketplace is trying to organize itself to deploy billions of dollars much more effectively. It has miles to go, but it’s heading in some encouraging directions. The intrepid Social Innovation Fund (SIF) and the just-launched Social Impact Bond (SIB) are two of the most promising.

At next week’s Social Capital Markets conference (SOCAP10) I’ll be moderating a panel on my favorite topic, “Scaling Social Impact,” with Shawn Bohen from Year Up, Jennifer Davis from the National Center on Time and Learning, and Lance Fors from the New Teacher Center, which rolls down the “Tactical Philanthropy” track curated by Sean Stannard-Stockton. Beyond that, I’m particularly looking forward to hearing more about Social Finance’s Social Impact Bond (SIB), which is surely one of the most promising developments I’ve seen in recent years. (Social Finance’s Emily Bolton will be leading an SIB “working session,” whatever that means, at SOCAP.) Here’s why.

I wish I had a nickel for every time I’ve quoted these wise words from Harvard Business School’s Michael Porter: “Philanthropy is decades behind business in applying rigorous thinking to the use of money.” The social sector is stunningly ineffectual in so many ways at managing the $300+ billion it takes in every year. We raise money based on telling stories about what we try to do, not on what we accomplish, thereby rewarding skillful storytelling rather than building productive organizations. We invest next to nothing in capacity building, making it impossible to systematically measure or improve performance. We spend virtually every dime we raise, with almost nothing saved for a rainy day or accumulated for long-term needs. We eschew best accounting practices, so few nonprofits are “bankable,” that is, they don’t have ready access to affordable working capital based on the quality of their well-managed balance sheets. The result is an appallingly low bang-for-the-philanthropic buck.

Still, the social sector is nothing if not resilient. We have conducted numerous experiments of some size and duration that provide encouraging and sometimes even convincing evidence that we now know how to solve some of our most difficult social problems. Philanthropy, volunteering and community engagement seem to be rising relentlessly even in the face of cyclical headwinds. The art and science of social entrepreneurship has gained an enthusiastic following in both undergraduate and graduate education and isn’t likely to let go.

Somehow, a tiny percentage of nonprofits manage to endure and distinguish themselves with mature management teams delivering proven innovations through relatively robust infrastructures and sustainable finances. Over the last ten to twenty years, these few exceptions to the rule have become sufficiently numerous that it has become impossible for forward-thinking funders to overlook their accomplishments, to the point that the crustiest segment of the social sector, institutional funding, has evolved a cadre of innovators.

If further proof were required that social innovation is becoming institutionalized in the best sense of the term (beyond the success of SOCAP itself), that proof arrived this summer with the launch of the Social Innovation Fund. To me, the significance of SIF resides in its structure. A new federal agency carves out a minuscule amount of taxpayer funds, $50 million, for multi-year deployment against defined performance objectives through experienced nongovernmental intermediaries to innovative nonprofits that can demonstrate their effectiveness from documented evidence through a competitive solicitation process. The agency oversees the intermediaries and the intermediaries oversee the nonprofits.

Even though SIF triples the corpus to $150 million through matching funds raised by both the intermediaries and the nonprofits, neither social nor financial innovation is the main object of the exercise. Rather, SIF is about “scaling what works”: expanding the delivery of proven social innovations beyond what the nonprofits could achieve on their own, even with the same amount of increased funding. As I have written before, producing innovation and spreading innovation are two quite different things, and the former does not inevitably lead to the latter, conventional wisdom to the contrary notwithstanding. If any magic should emerge from SIF, it will be found somewhere in the disciplined and purposeful application of expertise about scaling and the power of collaboration through communities for shared learning.

So SIF takes a small amount of federal money, triples it in conventional ways, and then processes it among carefully vetted collaborators to substantially extend the footprint of social innovation. The nonprofits, the intermediaries and the co-funders are all proven, brand-name entities that pose little risk individually, but collectively they’re a high-wire act trying to combine tiered expertise, evidentiary rigor, and careful performance management, to try to make 1 plus 1 equal 3, or at least 2 1/2.

Will the intermediaries add value? Will the co-funders play well together? Will the nonprofits grow more this way than they would through simpler and quicker funding increases? Can SIF mount a convincing case for continued and increased funding? Will SIF eventually produce a transformative model for scaling social impact? Given what the distinguished sponsors and participants have accomplished so far, I think the endeavor has quite a strong likelihood of success, but SIF wouldn’t be needed if scaling were easy. SIF is undeniably a commendable cross-sector initiative designed to draw on the complementary strengths and resources of public, private and nonprofit stakeholders, and it’s a lot cheaper than the Large Haldron Collider.

But, really, why go to all that trouble? Isn’t that a rather roundabout way to promote growth? What do intermediaries contribute other than matching funds? Consider one example: the joint application of the Strive Partnership/United Way of Greater Cincinnati, which received a $2 million, 2-year SIF grant.

Strive is a partnership of organizations that “represent all areas of our community ... the educators who teach; the nonprofits who support teaching and well-being; the philanthropies that provide financial support to both; the elected officials who create policy change; and the corporations who need a local, skilled workforce.” It was founded in the beliefs “that education must be holistic, because what happens outside of school is just important as what happens inside of school; providers must be accountable and make decisions based on data; is a cradle to career endeavor, and that working together is key to eliminating the ‘cracks’ that children might fall through; must be fair so that every child, regardless of circumstance, can find the support they need to achieve their dreams.”

The Strive intervention model begins, as all good intervention models must, with a robust theory of change. The “Student’s Roadmap to Success” is a breathtakingly thoughtful and comprehensive analysis, developed by University of Cincinnati researchers in the course of reviewing hundreds of evidence-based studies, of “the key experiences and milestones that are necessary along a child’s journey from cradle to career.” The Roadmap focuses on five critical “transition points” that “determine whether or not a child is successful in school and in life”: (1) entering kindergarten, and moving (2) from elementary to middle school, (3) from middle to high school, (4) high school to college or career training and (5) from college freshman to sophomore.”

The Roadmap establishes benchmarks children must reach in order to navigate each transition point, together with indicators of progress that academic and student/family support must achieve to reach those benchmarks. For example, to succeed at entering kindergarten, a newborn must have a “stable relationship with a strongly involved parent or caregiver ... who understands developmental milestones” so that the child “responds to parent/caregiver high-quality talking, reading and singing, ... uses exploration and discovery to understand surroundings, ... develops letter knowledge and reading sensitivity, ... and participates in high quality preschool.” When such a child enters kindergarten, she “exhibits learning-related skills such as self-regulation, social competence, self-esteem, and motivation.” Without those skills, children cannot succeed in elementary school and beyond.

Implementation is coordinated under the umbrella of a two-phase “Strive Endorsement Process.” Social service providers participate in “Student Success Networks” that meet informed criteria for successful collaboration: alignment with the Roadmap goals; inclusion of all appropriate organizations and willingness to expand as additional organizations are identified; wide support in the community; commitment to being data-driven; and willingness to publish progress and share data with the community.

For example, the Mentoring Student Success Network includes 13 organizations that provide some type of mentoring, whether it is one mentor to one child, one mentor to a group of children, or one mentor to a classroom of children, of which some organizations focus on certain areas of the city, certain schools, or on children in certain circumstances. Strive oversees 12 separate Student Success Networks corresponding to each of its priority strategies: Quality early childhood education; Home visitation (early childhood); School-based resource coordination; Mentoring; Drop out recovery; Health and wellness; Arts; After-school programs; Youth employment; Tutoring; College access; and College retention.

In the first phase of the Strive Endorsement Process, participating social service providers define a common problem and create a data plan for measurement; in the second phase, they collect local measurements to determine what indicators were most crucial and create a plan of action to influence those crucial indicators. Both phases follow the “Strive Six Sigma” process (developed with corporate financial support, training and expertise from GE) to “coordinate a group of very individual, independent organizations with a similar mission to becoming a cohesive Network of organizations working together towards the same goals.”

Clearly, without the Strive Partnership, the community of Greater Cincinnati social service providers could not possibly have undertaken the rigorous work necessary to unearth the essential developmental requirements captured in the Student’s Roadmap to Success. Even if such a dazzling blueprint were magically available from some other source, they could not have organized themselves to provide all of the coordinated academic and student/family supports needed to achieve them.

The Strive Partnership understands that scale is not an additive process in which good organizations do more and more of what has worked before, accumulating success and growing incrementally over time. Rather, scale is an exponential process by which good organizations acquire new capacities and develop new systems for delivering innovation in order to grow by orders of magnitude. Like metaphorical caterpillars that change into butterflies, scaling organizations transform themselves in stages that comprise differences in kind and not merely in degree. Intermediaries like Strive and the 10 other SIF intermediary grantmakers enable that transformation.

Nonprofits that grow incrementally and organically are merely larger than they used to be; nonprofits that scale are larger and different from what they were before, either by themselves, or by becoming an essential component of a functioning network in which the whole becomes greater than the sum of the parts, like the Strive Partnership than it can accomplish things collaboratively that the components could not produce themselves.

Wonderful, you say, Strive is terrific. But what does SIF bring to the party other than some additional funding that any federal agency could find under its couch cushions?

SIF is an intermediation factory.

Just as Strive creates collaboration value through the development and oversight of the Student’s Roadmap to Success, the Strive Endorsement Process and the Student Success Networks, SIF creates intermediation value through the development and oversight of systemic tools for competitive grantmaking, impact evaluation and using evidence of effectiveness as the basis for community-wide funding decisions. As explained in the Strive Partnership application, for example, even though the $2 million SIF grant represents only a tiny part of the Greater Cincinnati United Way’s annual $50 million budget, “it will provide a comparatively large influx of investible dollars to the Strive Partnership which will strengthen its systems work in the community and its ability to highlight programs that really work and make them examples for other similar programs and services.”

One of the defining characteristics of social entrepreneurship is the adoption of more business-like practices. When people say nonprofits should be run more like businesses, the SIF / Strive / United Way is a shining example of what that looks like. This arrangement is quite new in the social sector but completely familiar to high-growth business enterprises. However, for-profit companies like GE, the original developer of the Six Sigma process, don’t have the expertise or relationships to adapt or introduce the intermediation model in ways that take into account important differences between for-profit and nonprofit organizations working to solve stubborn and complex social problems. The Strive Partnership and SIF have that expertise and those relationships.

This self-organizing process is being invented as we speak, but it sure looks promising. The United Way began a major transformation in 2005 which it called the “Agenda for Community Impact,” which drastically refocused United Way’s funding strategy to support a few set priority outcomes, and required programs seeking funding to demonstrate through program evaluations not only that they were producing positive results, but that those results contributed to the achievement of the priority outcomes. The Strive Partnership was launched in 2006; just four years later, Cincinnati Public Schools have increased kindergarten readiness by 9%. Can substantially 100% readiness be far behind? I think not.

Now, in 2010, along comes SIF, which at a cost to U.S. taxpayers of just 7.5% of the $2 million grant awarded to Strive ($300,000), will enable the Cincinnati project “to build evidence of effectiveness and allow organizations to replicate and expand their effective programs.”

What SIF hopes to do for scaling social impact, SIB hopes to do for scaling social funding. As we’ve seen, SIF involves real rocket science when it comes to scaling impact, but not in terms of financial engineering, which relies on simple matching grants. That’s where the Social Impact Bond comes in. The SIB, in both its design and initial implementation, is quite modest in scope, but it is potentially seismic as an innovation of social finance.

The SIB is one of those simple but profound ideas that makes you wonder why no one thought of it before. Millions of people face incapacitating social problems that produce disastrous and expensive consequences. For a variety of reasons I explore in my book, our track record over the last several decades in responding to these problems has been poor and declining, and the adverse consequences and their associated costs have steadily increased across the board with only minor exceptions. This bleak outlook is not likely to improve in the face of the direst economic conditions that most of us have ever seen.

A common theme of effective social innovations is finding ways that failing institutions like public education, health care and the labor market can be fortified to serve many of those who currently fall through the cracks, so that people and communities that have fallen behind can become economically and socially self-sufficient. Through intensive, targeted and cost-effective interventions, at-risk students can graduate from high school and attend college, single mothers can raise healthier children and disconnected youth can become employable. Not only do the participants get back on track, but taxpayers avoid the added costs from trying to mitigate the collateral social damage.

The SIB starts with the simple premise that it should be less expensive to prevent problems rather than fix them after they occur. This is not an entirely new idea, but it has been devilishly difficult to operationalize for a couple of headache-inducing reasons.

First, government has no choice but to pay at least some of the staggering costs to respond to problems like children without health coverage, families without adequate food or shelter, and unemployed youth, but nothing except prudence requires it to pay for programs that might prevent those problems. In fact, the more money that has to be spent on the back end to deal with social and market failures, the less money is available, politically and otherwise, for preventive programs on the front end.

Second, even if prevention programs were funded, how could they prove the amount of savings they generated and that the savings were attributable to the programs?

These were some of the problems that plagued the Bush administration’s failed “share-in-savings” program, which sought to reduce federal outlays by entering contracts that would pay vendors only to the extent that they produced quantifiable savings. Contractors were understandably reluctant to assume the risks of up-front investments, particularly when the difficulty of establishing cost benchmarks would create doubts about whether and how much much savings were produced.

SIB responds brilliantly to both risks with the same answer: private investment. Instead of asking government agencies to pay up front for preventive programs that might not work (and potentially require the government to pay again later to address any failures) or asking underfunded nonprofits to assume the risk of nonpayment if savings are not realized, SIBs attract private funding for prevention programs by agreeing to pay investors (bondholders) a financial return if, when and to the extent that savings are produced.

We got our first concrete illustration on March 18, 2010, when London-based Social Finance and the U.K. Ministry of Justice announced the launch of the first SIB to reduce prison recidivism:

This first issue will fund social organisations working to reduce the re-offending rates of short sentence male prisoners leaving Peterborough Prison. The Ministry of Justice has agreed to make payments to investors in the event that re-offending is reduced below an agreed threshold.

Re-offending is an area where preventative work could lead to a better society and save the taxpayer money. Of the 40,200 adults on short term sentences, an estimated 60% will go on to reoffend within a year of release, at a significant cost to the taxpayer and society.

During the Peterborough Prison pilot, experienced social sector organisations, such as St Giles Trust, will provide intensive support to 3,000 short-term prisoners over a six year period, both inside prison and after release, to help them resettle into the community. If this initiative reduces re-offending by 7.5%, or more, investors will receive from Government a share of the long term savings. If the SIB delivers a drop in re-offending beyond the threshold, investors will receive an increasing return the greater the success at achieving the social outcome, up to a maximum of 13%.

Of course, if the project reduces re-offending by less than 7.5%, of course, bondholders will receive a lower financial return, potentially none.

The genius of SIBs lies in several factors:

The social service provider, St. Giles Trust, already has an impressive track record of reducing recidivism of this target population and generating substantial savings through a field-tested intervention. Research by Frontier Economics estimated that the program saved £10 in reduced costs for every £1 spent on prevention programs. The primary risk of SIBs is placed right where it should be: on the viability of the funding mechanism, which has been the obstacle to program growth in past efforts.
Project risk is further managed by the establishment of reasonable benchmarks against which savings will be measured. Government-funded share-in-savings contracts put public dollars at risk, so estimating both current expenditures and future cost reductions are subject to public scrutiny by regulators, legislators, auditors, and other financial watchdogs, inviting protracted debate, gridlock and second-guessing both before and after the fact. But when only private funds are at risk, the parties can negotiate any savings arrangement they deem reasonable. Here, the parties have agreed to what they believe are achievable targets for reduced recidivism and plausible estimates for corresponding savings, without getting lost in the weeds of either calculation.
Social Finance hopes to raise about £4.9 million (roughly $7.5 million U.S.) from investors, which will be managed to produce ongoing and long-term success:

Initial funds secured will fully fund operations for two years, giving the parties breathing space to launch and establish reliable operations.
While the program will be conducted over a six-year period -- a period of time that is long enough to enable the providers to build capacity and improve service delivery operations -- both the initial investment and the financial returns are drawn down and paid out, respectively, incrementally over that time span. Thus, investment capital will be drawn down over six years, and payments to investors (if any) will be made at the end of years 4, 6 and 8.
By focusing on short-term offenders and a one-year intervention model, the program can serve six successive cohorts of 1,000 offenders over the six-year project.

Social Finance is funded through a combination of sources that promote the objectives of the program:

Modest government-funded set-up costs (roughly $316,500 U.S.), to create a special-purpose financing vehicle designed to generate taxpayer savings;
An investor-funded capital-raising fee of 2.5% for a skilled financial intermediary to manage a new social investment instrument issued on terms that include reasonable and customary rights and protections for investors.
A 10% performance fee, to be paid only after investors are fully repaid, to reward the achievement of the desired program outcomes, reduced governmental expenditures.

The scale of the first bond is quite small, as it should be, but SIBs have enormous growth potential. On the demand (or buy) side, other kinds of prevention programs that SIBs could fund include some of our most pervasive and costly social problems, including truancy, family instability, overuse of emergency medical services, and drug rehabilitation. SIBs could become a critical factor in enabling the “governing by network” model advanced by Stephen Goldsmith and William Eggers in which federal, state and local agencies become “generators of public value within the web of multiorganizational, multiorganizational, and multisectoral relationships.”

On the supply (or sell) side, Social Finance’s long-term vision is “unlocking an unprecedented flow of social finance”:

Investment fund managers believe there would be considerable consumer interest in investing in Social Impact Bonds once a track record has been established and sufficient scale of investment opportunity exists. Ultimately, Social Impact Bonds could become a new social asset class, comparable to microfinance, enabling an unprecedented flow of investment into addressing social issues in the UK and elsewhere.

It does not take great powers of imagination to conceive of this new asset class as a tradable security with all that entails, including availability to both mass market customers such as donor-advised funds and institutional investors such as insurance companies and pension funds, as well as trading on secondary markets to create liquidity and reduce risk.

And if SIF and SIBs seem like exciting developments in their own right, just think about the possibilities if both experiments succeed over the next several years. The potential benefit of combining these innovations for scaling impact and scaling funding could be quite breathtaking.

Enabling Nonprofit Collaboration

2010-09-24T14:21:00.001-04:00

Despite a number of encouraging experiments in collaboration among nonprofits, it’s an idea that is (in Hamlet’s words) “more honor’d in the breach than the observance.” A recent agreement among seven leading organizations in support of “Charity Navigator 2.0” offers a new model based on advancing the mutual interests of the parties in order to make the whole greater than the sum of the parts.

I’ve been engaged in a lively online discussion with John MacIntosh of SeaChange Capital Partners and Sean Stannard-Stockton of Tactical Philanthropy on the intriguing subject of “Burning Bridges to Make Venture Philanthropy Work.” John recently responded to my suggestion that perhaps nonprofits and their funders could become more accountable and disciplined through purposeful collaboration:

My thought exercise was about the merits of “unilateral bridge burning” as a strategy to tackle problems of commitment and self-control in the absence of market forces. But you make a good point: entering into a “collective” could be equally effective (e.g. AA) and more psychologically plausible than going it alone. However to be effective such a “collective” needs be long-term and difficult to exit, must require that each participant give up some rights by ceding important decisions to “the group”, and must serve as a shared space of trust where people (or institutions) can candidly debate, discuss and declare their “public reasons” for a particular course of action (think Rawls or Habermas). My suspicion is that most funder “collaboratives” have none of these characteristics (let alone all three) and those that do were created through external pressure (like the SIF?). I hope I’m wrong.

I don’t think John is wrong about traditional ingredients for effective collectives, including long-term commitments, at least partial relinquishment of independence, and high levels of trust. However, I do think that other approaches are possible, and might be more conducive to productive collaborations among independent organizations seeking to work together without any kind of compulsion.

As many of you know, Ken Berger has been leading a heroic effort at Charity Navigator to reform CN’s outdated rating methodology, which has been the subject of heated and often justified criticism long before Ken’s arrival in 2008. On July 1, 2010, CN took the first big step forward by launching its “accountability & transparency” rating, which, as the site explains, focuses “on how the charity reports publicly – both on the IRS 990 and on its web site. We consider in our methodology whether the charity is making easily available, information regarding its governance practices, ethical practices and financial information.” Although it will take about a year to run through all 5,500 charities that CN rates before the accountability and transparency dimension can be factored into CN’s star-rating system, CN’s small but hard-working staff has published information on how 393 nonprofits did on the new methodology. This is a great development (in which I was involved as Advisory Panel member and CN consultant.)

CN hopes to revise and expand the financial dimensions of its ratings systems by the end of the year, and, in 2011 or 2012, incorporate the holy grail of “effectiveness and results” into its methodology that will weigh independent evidence of performance, not just self-reporting. When CN completes this work, it will have a three-dimensional rating system -- financial strength; transparency and accountability; and effectiveness and results -- that will finally enable CN to become the simple and complete “guide to intelligent giving” that donors and nonprofits alike so desperately need.

At the same time, fixing CN’s rating system will not, by itself, transform the nonprofit capital marketplace into a mechanism for guiding more money to the most effective charitable organizations. As I show in my book, there is almost no connection between funding and nonprofit performance: good nonprofits don’t raise more money and weak nonprofits don’t raise less, because fundraising is based on building relationships and telling engaging stories about what nonprofits are trying to do, not what they accomplish. (New Philanthropy Capital has just released an outstanding paper on this subject, “Talking About Results.”)

There are many reasons for the disconnection between funding and performance, none of which include laziness or ineptitude by nonprofits. Collecting and reporting meaningful and useful information about nonprofit accomplishments is difficult, time-consuming and expensive, and funders typically won’t pay for such efforts or the expertise required. When I began writing my book in 2005, the idea that charitable funding should recognize and reward strong performance was a foreign concept to all but a few forward-thinkers such as Clara Miller and George Overholser at Nonprofit Finance Fund, Paul Brest at the Hewlett Foundation, Martin Brookes at New Philanthropy Capital, Lucy Bernholz at Blueprint Research, Mario Marino at Venture Philanthropy Partners, Bill Shore at Community Wealth Ventures, Andrew Wolk at Root Cause, and Katherine Fulton and Andrew Blau at Monitor Group. (Apologies to anyone I omitted from the honor roll.)

There is now a growing consensus that social progress will be held back until nonprofits can attract greater and more sustainable funding by the simple act of accomplishing their objectives. As I put it in my book (with no small amount of jargon that I explain in plain English in the book), “We need a financing system that helps highly-engaged social impact investors to direct third-stage growth capital to the best mid-cap nonprofits, instead of one that forces those nonprofits to spend all their time looking for more drops to fill more buckets.”

Charity Navigator’s expanding rating methodology will become an essential part of this emerging performance-based nonprofit capital market, but it will not be the case that (unlike the movie, Field of Dreams) “if you build it, they will come.” As Hope Consulting’s recent study shows, almost no donors conduct comparative research to inform their giving decisions. Although CN’s three-dimensional rating will make it immeasurably easier for donors who want to maximize the impact of their donations to do so, performance-based philanthropy represents a massive change in social behavior akin to civil rights and public health campaigns. Merely providing new tools will not change long-held habits overnight.

I believe that collaboration will be a critical ingredient of shifting the paradigm and it is something that the social sector is much better at talking about than actually doing. Katherine Fulton and Andrew Blau have presciently observed that “philanthropy itself is not a system”:

“Individual institutions and givers in philanthropy are not in any sense reliant on one another; they exist independently and can act without much reference to what others do. Thus, there is no system where actors must respond to one another, adapt to one another, or learn from one another. This is not to say that donors and foundations don’t relate or learn from one another at all. They do, but only to the extent that they choose to. And they also compete with one another — for ideas, reputation, and credit, which can discourage the free exchange of ideas and lead to fragmentation of effort and isolation.”

It would be an understatement to say that these are not the ideal circumstances for successful collaborations. But they are a reality that must be accommodated in any serious efforts to accomplish what groups of organizations can do working together rather than on their own.

In the case of Charity Navigator, Ken and I realized that CN’s new methodology was necessary but not sufficient for a more performance-driven kind of philanthropy, and that CN did not have the horsepower to move the needle of donor behavior by itself. Thinking about what it would take to shift donor thinking toward supporting nonprofits that could demonstrate their effectiveness, we started talking to a number of respected colleagues who were addressing similar challenges and had developed approaches and assets that we thought could help the cause.

The response was tremendous, leading to the formation of something we call “CN 2.0,” a planned collaborative online platform for intelligent giving “dedicated to the proposition that if donors become well-informed, good nonprofits will get more money.” CN enlisted six “best-of-breed” partners: Keystone Accountability, Philanthropedia, New Philanthropy Capital, GreatNonprofits, Growth Philanthropy Network, and GiveWell. The Hewlett Foundation (via Paul Brest and Jacob Harold) and the Fidelity Charitable Gift Fund (via Cynthia Strauss) have provided both generous financial support and encouragement.

In order to get this thing off the ground (which remains in the planning and seed funding stage), we needed to actively engage the partner organizations, all of which had very full plates before CN 2.0 came along. Without any initial funding by which to purchase their affections, we needed to find a way of enticing these busy organizations to spend uncompensated time with us to develop the concept and build the platform.

To make an already too long story a little shorter, we basically took the opposite approach from the one that John MacIntosh offered above, at least regarding the need for the partnership to “be long-term and difficult to exit, ... [and] require that each participant give up some rights by ceding important decisions to ‘the group’.” On the other hand, we did fully embrace John’s thinking that the alliance “must serve as a shared space of trust where people (or institutions) can candidly debate, discuss and declare their ‘public reasons’ for a particular course of action.”

Ken and I circulated a “letter of intent” which all of the parties were happy to sign (copies of which are available upon request), which set forth “what we hope to accomplish and how we propose to go about it”:

1. We all agreed on a shared purpose to “advance social progress by helping more effective nonprofits grow” by helping “individual donors, who provide roughly three-fourths of all private charitable donations, to find and fund nonprofit organizations that will make the most productive use of their philanthropy.” We stated our belief that “our collective efforts to promote and even universalize informed social investing can guide sizeable amounts of funding to more effective nonprofits and ‘move the needle’ of social progress to an extent that isn’t currently possible in today’s highly fragmented nonprofit capital market.” 

2. So that everyone would clearly understand the scope of the undertaking, we summarized the basic parameters of the project in a detailed attachment entitled, “A Growth and Collaboration Plan for Guiding Individual Donations to More Effective Nonprofits.” In brief, we proposed combining “(1) the best available data about the finances, accountability & transparency and effectiveness & performance of charitable organizations with (2) cost-effective and timely analysis of that data, and (3) distribution of that data and analysis as widely as possible.” 

3. We also made clear that this was a risky and uncertain venture: “we acknowledge that the pathway to success is far from clear, and that experimentation and trial and error will be the order of the day. We believe that our partnership can develop affordable and practicable approaches that will not allow the perfect to become the enemy of the good, and provide a unique environment in which many of the most promising ideas for expanding social innovation can flourish.”  

4. We considered it imperative that the group be governed by operating principles of mutual benefit and open decisionmaking: “CN 2.0 is designed to be a collaborative and consensus-driven alliance of equals, all of whom recognize the complementary value and expertise that the other partners bring to this effort. Charity Navigator is committed to advancing the brands and market positions of the partners in the course of this project.”

5. At the same, we wanted to make real progress, so we created a collaborative governing structure consistent with the operating principles: “CN will take the lead on coordinating the efforts of the collaboration, and Steve Goldberg will serve as CN’s Project Lead (as an independent consultant) to keep our efforts productive and on track. Decisions will be made through consensus to the greatest extent possible, with the objective of enhancing the value of the collaboration to all partners. At this point, all CN 2.0 business models are ‘TBD’.” 

6. We did not want anyone to worry about whether the agreement would create potential legal risks, so the letter was drafted as an expressly non-binding “expression of our collective interest in collaborating on CN 2.0 and making it a strategic priority for our respective organizations.” We made clear that any participant was free to leave at any time, that our partnership would be “a voluntary alliance of separate and independent entities that doesn’t confer any rights or impose any obligations of any kind,” and that “nothing precludes any partner from engaging in any work on its own or with others, and we acknowledge that all of us are currently involved in related and potentially overlapping collaborations.” There was additional verbiage (I used to be a lawyer in a former life ...) about intellectual property rights, investment risks, “non-recourse” (i.e., no lawsuits), and a commitment that “We will work out any disagreements as professionals” with “appropriate professional courtesies of prior consultation with the others.” We acknowledged that, at some point, “it might make sense to negotiate a formal legal agreement that would create contractual rights and responsibilities, but we agree that would be premature at this time.”

We’re a long, long way from being able to say that CN 2.0 has accomplished anything worthwhile, but we do think this collaborative model has significant potential across the social sector for organizations looking for ways to work together without becoming distracted by side issues.

Growth Capital for Small Nonprofits?

2010-09-21T17:58:00.001-04:00

This is a response to a blog post by Nell Edgington of Social Velocity, entitled "Can’t Small Nonprofits Raise Capital Too?" which is available here.

Nell, I wanted to offer some comments on your thoughtful post. I hear this lament from small nonprofits all the time: how do we grow if we're too small to raise growth capital? It's a fair question to which the nonprofit capital market does not yet provide a satisfactory answer.

For the vast majority of nonprofits, there's only one kind of money, regardless of the particular source: funding for programs. That money is secured the old fashioned way, by raising it from donors by (1) building relationships and (2) telling engaging stories about the nonprofit's work. It's a costly and time-consuming process that never raises enough money for long enough time. Hence, the nonprofit starvation cycle is the dominant fact of life for small nonprofits.

George Overholser pioneered a new kind of money that he calls "patient capital" or "equity-like capital," and that I call "growth capital" (full disclosure: George disapproves of the way I use that term). The basic idea, as you say, is that investors fund the nonprofit's entire business plan for an extended period (say 3-5 years) rather than some or all of particular programs for a finite period of time. The goal is to enable the nonprofit to permanently grow to a new level of operations that can be sustained by traditional program funding. George observes that it's simply too costly for small nonprofits to make the case for this kind of funding, and he knows whereof he speaks. Small nonprofits are no less deserving than larger ones, but only the larger ones can undertake the kinds of planning and demonstrate the capacity to make effective use of funding designed to enable organizations to grow by factors of 2, 3 or more over the course of several years.

However, I believe there is an intermediate kind of funding between program funding and growth capital that small- and medium-sized nonprofits can raise and that is capacity-building funding. Of course, we all know what capacity-building expenses are -- computers, specialized staffing, professional accounting and fundraising systems, and so on -- and we also that almost no funders, either individuals or foundations, provide this kind of money, which we disparagingly call "overhead" or "administrative" expenses. The failure of the nonprofit capital market to provide capacity-building funding (and not growth capital) is what keeps small nonprofits locked in the Catch-22 of the nonprofit starvation cycle.

The emergence of growth capital is a recent development in the nonprofit sector and it is still very much in its infancy, even though courageous intermediaries like NFF Capital Partners and EMCF are demonstrating its importance for scaling what works. But you're completely correct that a similar effort needs to be made for capacity-building funding. As you might be aware, Ken Berger at Charity Navigator is revising his rating methodology so that effective nonprofits won't be penalized for making reasonable overhead expenditures designed to enhance their organizational capacity and extricate themselves from the starvation cycle. (Another disclosure: I consult with CN.) Hopefully, charities that would lose four-star ratings under the current CN rating system will attract greater funding when the new system goes into effect.

Growth capital is aimed at achieving true scale, but capacity-building funding is aimed simply at producing robust nonprofits that aren't held back by the starvation cycle of program-only funding. Just as small businesses provide most of the jobs in this country, we need the kind of funding that can enable many more small nonprofits to meet the everyday needs of the communities they serve in a reliable and sustainable way.

Deciding What "Works"

2010-09-14T15:44:00.001-04:00

Conventional wisdom holds that randomized control trials (RCT) are the “gold standard” of evaluation. In fact, RCTs only make sense under very strict conditions that can rarely be met in the real world. Most of the time, RCTs produce inconclusive results and simply aren’t worth the time and money. As the social sector assumes greater responsibility for improving the lives of many more people, it should focus less on pseudo-scientific “proof” that programs work and focus more on making good programs better.

Now that the Social Innovation Fund (SIF) appears to have survived the “transparency” commotion, the eleven chosen intermediary grantmakers have less than six months to select their portfolios of nonprofit grantees. As a commendable exercise in “evidence-based” grantmaking, SIF requires the intermediaries to incorporate evaluation into every step of their awards, from the initial competitive solicitations all the way through final payments and renewals. Applicants will be required to explain how their success should be measured and demonstrate their capacity to do so, and awards will be contingent upon the establishment of meaningful performance metrics, the timely collection and reporting of reliable data, and the faithful implementation of sound evaluation protocols.

The evidence-based rubric represents a significant advancement for the social sector, which has historically relied on anecdotal indicators of success, that is, entirely subjective and largely unreliable assessments of whether programs work. Now that philanthropy and nonprofits are pursuing more ambitious goals for improving educational achievement, extending economic opportunity and alleviating poverty at scale, there is growing acceptance of the need for more objective measures of effectiveness, reinforced by the fact that scarce tax dollars are going to nongovernmental organizations promising to deliver more effective solutions. As a general matter, this heightened appreciation for serious evaluation is an encouraging development.

There is a countervailing risk, however, that having ignored systematic evaluation for too long, the social sector will now overcompensate in the other direction, requiring evaluations conducted under “laboratory conditions” that generally cannot be met in the field and that are biased toward concluding that funded programs have no apparent impact. Before we set ourselves up for failure, this is a good time to pause and think carefully about how high we set the evaluation bar.

The problem arises from the seemingly innocuous assumption that judging whether an expenditure of public funds (or, in the case of SIF, a combination of public and charitable funds) was “worth it” requires a determination of whether the funding produced a measurable amount of social benefit. That is, can the grantee show that the observed results were attributable to the funded program? If so, we are told, the expenditure succeeded; if not, it failed. Otherwise, the thinking goes, we’d fall back to our old unaccountable ways in which programs that seemed at first to work later failed to produce enduring or significant benefits, particularly with increased funding and growth.

When auspicious programs fail to produce long-term or transformative benefits, the underlying assumption is that the observed short-term results were either illusory or the product of factors other than the funded intervention, or both. There is another explanation, however: the impacts were real albeit modest, but the program was not properly nurtured to reproduce and extend those benefits.

These alternative explanations comprise two quite different views of how we can bring about social progress. In one view, the burden of proof is placed squarely upon those who claim that the programs “worked”: if their effectiveness cannot be quantified, and the cause-and-effect relationship between the program and the apparent results cannot be proven affirmatively, it would be unscientific and therefore irresponsible to continue or expand their operation. That is to say, social programs are presumed ineffective unless they can be proven effective; the failure to demonstrate effectiveness is taken as proof of ineffectiveness. Let’s call this the “purist” perspective.

The fundamental problem with this approach is that it assumes we have good tools to determine whether programs are effective and how much impact they produced. If so, it would indeed make sense to place the burden of proof on program proponents to show their efforts were successful. After all, they’re the ones asking for funding and they control how the funds are spent and programs are implemented.

But if the assumption is false -- that is, if we can’t reliably verify and measure the effectiveness of social programs -- then imposing the burden in that way merely stacks the deck against social innovation and leaves program advocates without any effective means to make their case. In that case, we’d just be denying ourselves valuable innovations for no good reason.

The second and more “pragmatic” approach to evaluation starts from the premise that the causes and cures of social problems are simply too elusive and complex to measure or explain precisely. Pragmatists believe that, at best, policy makers can make informed judgments about which interventions are probably more effective and which are probably less effective, and social policy should try to identify the more effective policies and continuously improve them.

Decades of academic research have shown there is most definitely a place for the purist approach. It has usefully prevented the acceptance of ineffective and dangerous medical treatments, unsound scientific theories and spurious practices in business and industry. There are even some examples, such as the mighty Nurse Family Partnership, where RCTs have demonstrated the powerful impacts of social innovation. But it would be foolish to give that approach more credit than it deserves or to apply it where the circumstances do not warrant.

The justification for the purist approach is methodological rigor. By setting a high bar for proving that social programs work, we thereby reduce the risk of concluding that an ineffective program actually succeeded, what evaluation professionals call a “false positive.” If we mistakenly invest additional resources in a false positive, not only are we going to be disappointed down the road, but the longer we do so, the more disappointed we will be. In addition, the chorus of “we-told-you-so’s” by program opponents will become correspondingly louder, coupled with the fact that investing more resources in effective programs means that fewer resources would be available for alternatives approaches that might have been more effective. So adopting an evaluation methodology designed to minimize false positives has a lot going for it.

On the other hand, false negatives aren’t much fun, either. A false negative means that a successful or least promising program was allowed to die on the vine. It represents an opportunity lost at a time of desperate and worsening social and economic need.

From the perspective of methodological rigor, false positives and false negatives are exact mirror images of each other. Neither is inherently better or worse. In both cases, an erroneous decision is made and resources are misallocated. For false positives, money is wasted on programs that don’t work; for false negatives, money isn’t spent on programs that do (or might) work. Avoiding false positives reflects “do no harm” thinking, while avoiding false negatives, carried to extremes, can lead to “do no good.” By stacking the deck against false positives, we reduce the risk of being too gullible by increasing the risk of being too skeptical.

The problem, of course, is that it’s hard to tell the difference between false positives and false negatives, just as it’s hard to tell whether programs work or not. What evaluation science tries to do is to provide techniques that calibrate the risks of erroneous decisions in ways that make sense. Unfortunately, nostrums like “RCTs are the gold standard of evaluation” are often misused in ways that don’t make sense at all.

A control group, of course, is a means of isolating the effects of a certain “treatment.” If two groups of people are identical in every meaningful way except that one gets the treatment and one doesn’t (or gets a placebo), it’s fair to conclude that any difference in the results were attributable to the treatment. But in the real world, practical obstacles intrude.

First, even under ideal laboratory conditions, perfect randomness can be difficult to achieve. In the field, where researchers are dealing with specific communities of people with virtually unlimited and sometimes indeterminate or hidden characteristics, creating truly random experiments is maddeningly difficult, time-consuming and expensive. There’s also a political dimension: for example, it’s not so easy to explain to poor families why their children were randomly assigned (consigned would be more accurate) to a school that everyone knows is lousy so some clipboard-wielding social scientist can decide whether that hot new charter school that everyone’s talking about can help lucky kids from some other disadvantaged families avoid a life (sentence) of educational inequity.

Second, with an RCT, you can’t adjust the experiment along the way to make improvements. Suppose the charter school decides, half-way through the five-year evaluation, that it wants to adopt a terrific new curriculum for teaching fourth-grade math. Now you don’t have one five-year experiment, you’ve got two two-and-one-half year experiments with half as many kids in each, which might or might not still be random relative to the control group, and you might not have enough data in any treatment group to produce meaningful results.

The third limitation of RCTs, the misuse of statistical measurements, is both the most nefarious and the least understood. It begins with the peculiarities of the word “significance,” which means entirely different things in English and in statistics. In English, significance refers to importance; in statistics, significance relates to validity, but its claims of iron-clad validity are often doubtful.

Statistics is a mathematical science that allows general conclusions to be drawn from specific cases. If I want to find out if charter schools can improve the educational performance of fourth graders, as a practical matter I can’t conduct an experiment with every fourth grader in the country (i.e., the “universe” of fourth graders). Instead, I have to select a manageable number of students, called a “sample,” in a way that enables me to draw reasonable conclusions about how similar charter schools might help other fourth graders who weren’t part of the sample. Statistical science establishes procedures that enable such generalizations to be made from small experiments, and statistically rigorous studies follow what are called true “experimental designs,” of which RCTs are one example. (For purposes of simplicity, I’m pretending that all charter schools are the same, which of course they’re not.)

Here’s the basic problem inherent in using true experimental designs in evaluating social programs. The only way to perfectly measure the impact of charter schools on fourth grade students would be to conduct an experiment with the entire universe of all fourth-graders, in which a perfectly random half went to charter schools and the other perfectly random half went to traditional public schools. While such a study would result in 100% certainty about the charter-school treatment effect, we can’t conduct such an experiment for many reasons, not the least of which is there aren’t enough charter schools to serve that many students.

So we’re going to have to compare a manageable sample of charter students to a manageable sample of non-charter students. Even assuming that perfectly random assignments were made to the two groups so that there were no meaningful differences between them, neither sample would perfectly embody all of the characteristics of the entire universe from which it was drawn. We could choose another sample and divide them randomly between the treatment and control groups, and they would be different in some indeterminable ways from the first sample.

If we follow sound statistical practice, the differences among samples shouldn’t be large enough to invalidate the results. Not surprisingly, the primary factor in these undetectable variations among samples is the size of the sample: the larger the sample, the more it should be like the universe; the smaller the sample, the greater the chance that it will be quite different from the universe.

The genius of statistics is that it enables valid conclusions to be drawn about universes from pretty small samples, which is good because experiments with large samples are expensive and difficult to manage. And statisticians can estimate the amount of variation among different sizes of samples. This enables experimental designs to draw conclusions about treatments within identifiable probability ranges.

But when you’re dealing with probabilities, everything’s indefinite. Among all of the possible outcomes, some are more probable than others, but it’s hard to say exactly how likely any particular outcome is in any particular case. So how do you make an “evidence-based” decision when imprecision is unavoidable? When can you say, “this result is likely enough for us to say this works,” while “this result is just too speculative for us to accept”?

There’s nothing inherently wrong with estimating probabilities, as long as you acknowledge that’s what you’re doing. Which brings us back to the word “significance.”

In the statistical lexicon, an observed difference between two groups is considered “statistically significant” if the probability that the difference is due to purely random factors rather than to the treatment falls below some accepted threshold of evidence. In other words, since we can’t conduct universal experiments, there’s always some chance that a particular experiment will lead us to conclude that the treatment worked, when the difference between the treatment and the control group was actually due to some unpredictable and undetectable fluctuation in the sample we happened to pick. But that’s actually the beauty of statistics: we can use small samples to make informed judgments about how a treatment is likely to affect the universe, even though there’s always going to be some amount of uncertainty that can be reduced but not eliminated entirely.

Experimental designers make a living by conducting RCTs that have acceptably small risks of random error. But how small is small enough? Accepting practice says that sometimes it’s as small as a 1% chance of random variation (meaning there’s a 99% probability that the observed difference is due to the treatment), sometimes as small as 5% (a 95% probability that the treatment caused the difference), and sometimes a 10% difference is deemed acceptable (a 90% probability). Anything more than a 10% chance of random variation is almost always considered “statistically insignificant.”

Why? Because starting in 1925, Sir Ronald A. Fisher, the renowned English statistician and evolutionary biologist, declared that

“it is convenient to take this point as a limit in judging whether a deviation is to be considered significant or not.... A scientific fact should be regarded as experimentally established only if a properly designed experiment rarely fails to give this level of significance. [Researchers should] ignore entirely all results which fail to reach this level.”

Translation: the 99% or 95% or 90% probability thresholds for statistical significance are rules of thumb, nothing more. They’re convenient conventions that statisticians have agreed to use to distinguish between reliable and unreliable experimental results. Instead of 90%, it could be 11%, or 9%, but it’s not. So an 89% probability that a program worked is considered “insignificant,” while a 90% probability is considered significant. As Stephen T. Ziliak and Deirdre N. McCloskey note in their snarky but scholarly book, The Cult of Statistical Significance, Fisher’s reasonable but arbitrary line “appeals to scientists uncomfortable with any sort of ... indefinite approximation.... To avoid debate they seek certitude such as statistical significance.”

RCTs don’t run into trouble when they produce “significant” results, but real-world problems arise from the fact that most people don’t understand the significance of “insignificance.” Non-statisticians think that an RCT showing that a program result is “statistically insignificant” means that the study “proved” that the treatment “doesn’t work.” That’s completely wrong. All it means is that, due to unexplainable variation in the data, the study couldn’t determine whether or not the program worked based on the particular sample chosen with a probability of 90% or higher. Maybe it could make that determination at an 89% probability, or a 75% probability, but the statistics arbiters tell us that’s not good enough.

Ziliak and McCloskey offer an illustration of when focusing on "significance" can be silly. Suppose we conduct a study of body weight based on a person’s height and the amount of exercise they get, and suppose further that the data show that height is statistically significant but exercise is not. “A doctor would not say to a patient, ‘The problem is not that you’re fat -- it’s that you’re too short for your weight.'” That is, just because the exercise data from the particular sample chosen wasn’t precise enough to be considered statistically significant does not mean that exercise isn’t a factor in weight. It doesn’t mean that fat people are too short. It means that this study doesn’t enable reliable conclusions to be drawn about the relationship between weight and exercise. As Carl Sagan famously observed, “The absence of evidence is not evidence of absence.”

Statistical insignificance is a finding about the precision of a sample, not the impact of a program. Insignificance comes from too much unaccounted variation in the data, often resulting from a sample size that’s too small, poor experimental design, or other inexplicable factors that come into play when we try to conduct laboratory experiments in the real world. RCTs often produce “insignificant” results, which means nothing more than the experiment, no matter how expensive, laborious and time-consuming, was inconclusive. And inconclusiveness runs both ways: the study didn’t prove that the program worked and it didn’t prove that the program did not work.

Mistreating insignificance as proof that treatments don’t work has real consequences. Ziliak and McCloskey, whose book is subtitled, How the Standard Error Costs Us Jobs, Justice, and Lives, report on a 1980s study that found that Illinois saved $4.29 for every dollar it spent providing a training subsidy for unemployment insurance recipients, but the savings estimate was only significant at the 88% confidence level, just shy of the 90% cut-off. The program was deemed a failure, even though the defect related only to the fuzziness of the data sample. Another study hypothesized that stiffer penalties for dangerous driving in the United Kingdom could have saved 100,000 lives over ten years, but the estimated results were significant at 95% probability but not at 99%. In such an experiment, the choice of significance levels would be decisive.

The purist and pragmatist schools of thought about evaluation techniques reflect different perspectives about the purpose of evaluation. As we’ve seen, the purists focus on precision: they want the most accurate probability estimates possible, even if that leads them to reject results that look strong but don’t cross the magical but arbitrary threshold of “statistical significance.” As The Acumen Fund’s Brian Trelstad wrote in an insightful 2008 paper, “Simple Measures for Social Enterprise,” “Metrics and evaluation are to development programs as autopsies are to health care: too late to help, intrusive, and often inconclusive.”

I agree that purists are like doctors who let patients die because autopsies provide the most accurate cause of illness. Pragmatists care more about figuring out a cure than nailing down the cause. As the standard bearer of the pragmatists, Mark Kramer of FSG Social Impact Advisors says, “the real value of evaluation is its usefulness as a management tool to refine strategy and improve implementation over time.”

“Within the field of Social Entrepreneurship ... the primary goal is to catalyze change rapidly on as massive a scale as possible. The measures that matter most are practical indicators that can be tracked and acted on in real time to spread ideas or build strong organizations that can reach more people more cost-effectively.”

Like Kramer, Acumen’s Trelstad favors a “a performance management process that would ‘take the pulse’ of our work: frequent, simple measures that would allow us to refine our thinking, change our course, and diagnose problems before they become too significant,” using that word in its non-statistical sense, of course. They both agree that evaluation should focus “on the pragmatic question of how to help more people sooner” (Kramer) for “the primary purpose of supporting and scaling each enterprise.” (Trelstad)

Another limitation is that statistical methods only test one hypothesis at a time: does the variation in the data indicate with a high enough probability that the treatment caused the result or not? They don’t provide guidance among multiple treatment options, which is exactly what we need when considering alternative policy choices. Binary choices -- yes/no, true/false -- don’t help much. We need to know how well a program worked, whether it worked better than other potential approaches, and how effective programs can be improved. Acumen, for example, wants to know how programs they fund “compare more or less favorably to the ‘best alternative charitable option’ available to our donors,” that is, “how else the donor could have invested their money.” Trelstad frames this well:

“The search for absolute impact or performance measures is elusive and in my mind irrelevant. Performance is always relative to what you had been doing before (past), to what your competition did over the same time period (peers), and to what you should have done (projections).”

It’s important for conscientious social entrepreneurs, intermediaries and funders to maintain a sense of perspective as they try to shift the paradigm in the admirable direction of greater accountability for performance. Fortunately, there are many sound evaluation models that are much more practicable than RCTs which provide results that provide more than sufficiently reliable results on which to base reasonable policy decisions. For example, Kramer offers a 12-cell “evaluation matrix” which captures three different kinds of measures -- monitoring, process and impact -- for each of four levels -- grantee, donors, program area, and foundation. From these 12 combinations, he identifies six kinds of evaluation that reflect different objectives: formative, summative, donor engagement, cluster evaluation, overall foundation performance, and administrative processes.

One of the many benefits of the three-level SIF design -- (1) expert CNCS staff, (2) sophisticated intermediaries and co-funders, and (3) growth-ready mid-cap nonprofits, all with experience in data collection and evaluations -- is that the network is well-positioned to resist the siren call of the RCT “gold standard.” CNCS has already demonstrated both the willingness and the wisdom to avoid unrealistic methodological requirements in response to public comments (including mine) submitted to the Notice of Funding Availability (NOFA):

“Over 50 public comments were received on the use of evidence of effectiveness and impact in the SIF. Many of the comments encouraged the Corporation to be more inclusive about the types of evaluation that would produce strong evidence of impact. The Corporation will capture these insights in its Frequently Asked Questions (FAQ), a companion document to the NOFA. The FAQ will clarify that the Corporation expects subgrantees to demonstrate some level of impact in order to receive a grant, but does not expect that most initial subgrantees will have the strongest level of evidence. The SIF is designed to build the evidence base of programs over time using rigorous evaluation tools that are appropriate for the intervention. The Corporation is committed to ongoing discussion about evidence moving forward through learning communities and other forums.” (Emphasis added.)

Now, CNCS’s reference to “the strongest level of evidence” makes me a bit queasy as it seems to echo misplaced notions of gold standards. RCTs are often impossible to conduct in the field or are available only at prohibitive cost. As Brian Trelstad noted with some understatement, “it is impractical to spend $250,000 researching the impact of a $500,000 investment ...” At the risk of beating the dead horse yet again, I’ll just comment that it’s hard to see how evidence can be “the strongest” if you can’t actually get it. In my experience, hypothetical evidence isn’t all that strong. Rather, “the strongest” evidence is the most rigorous evidence that you can actually get at justifiable effort and cost. Putting that semantic quibble aside, CNCS seems to understand that it should not let the unattainable perfect become the enemy of the readily attainable good, particularly when it talks about evaluations that are “appropriate for the intervention.”

CNCS’s pragmatic approach makes particular sense given SIF’s focus on more mature nonprofits selected by growth-oriented intermediaries. As Kramer has observed, “Over the organizational life cycle, however, expectations for management performance, cost effectiveness, and scale of impact increase rapidly, requiring very different evaluation criteria at different stages of maturity.”

In the case of a promising but untested new innovation, it makes sense to ask “Does this work?” and “Is it better than existing approaches?” But once an innovation has accumulated some evidence of impact, as will be true for all SIF grantees, the more important question becomes, “How can we make this more widely available?”

Trust me, we’re going to hear that SIF was a waste of time and money to the extent that it didn’t use RCTs. Ziliak and McCloskey observe that RCT’s “arbitrary, mechanical illogic ... [is] currently sanctioned by science and its bureaucracies of reproduction ...” and “the sociological pressure to assent to the ritual is great.” But when it comes to making important choices about which social policies to fund and expand when families’ lives and welfares are at stake, insisting on an arbitrary 90% or higher standard of “statistical significance” is a luxury we don’t have. If there’s an 89% or 80% or 75% chance that a given program probably accounts for, say, the improved grades that one group of students received, we should think carefully about keeping and improving that program. I agree with Ziliak and McCloskey that it would be irresponsible to abandon such a program based on insignificance alone:

“Accepting or rejecting a test of significance without considering the potential losses from the available courses of action is buying a pig in a poke. It is not ethically or economically defensible.”

No less a figure than W. Edwards Deming put it plainly: “Statistical ‘significance’ by itself is not a rational basis for action.” And Gara LaMarche, chief of The Atlantic Philanthropies wrote in The Financial Times, “both funders and the organisations they support need more humility about cause and effect.” Trelstad reminds us that “the expectations for what one can measure and what one can prove diverge from the reality of practice.”

Of course, if there’s only a 50% chance, that is, if it’s just as likely that the higher grades were caused by random differences among samples, then, sure, that’s not very encouraging. But at some point, it’s foolish to believe there’s some bright line of probability that can rescue us from having to make difficult judgments about what works and what doesn’t. RCTs aren’t a silver bullet, a gold standard or some kind of “on-off switch for establishing scientific credibility.” In exceptional cases, they’re worth doing; in most cases, they’re not. Fortunately, there are many other good ways to evaluate nonprofit organizations and programs that don’t involve complete guesswork or wishful thinking. Those are generally the best techniques available and we should embrace them enthusiastically to help us make timely choices among encouraging alternatives, which is just what we need in the pursuit of “scaling what works.”

SIF Intermediaries: More Drops in Fewer Buckets, Please

2010-08-27T17:16:00.014-04:00

Producing new innovations and spreading successful innovations are two quite different things. The social sector is good at the former but not the latter. The recently selected SIF intermediaries can help fix that if they make larger grants to fewer nonprofits.

In my book, Billions of Drops in Millions of Buckets: Why Philanthropy Doesn’t Advance Social Progress, I argue that the primary impediment to social progress isn’t lack of innovation per se but our inability to spread -- or “diffuse” -- proven innovations. The main reasons are (1) the fundraising process fragments donations into amounts that are too small to pay for widespread expansion, i.e., the nonprofit sector has very little “growth capital”; and (2) even if growth capital were available, most nonprofits don’t have the capacity to put it to good use. As a result, the U.S. nonprofit sector comprises nearly 2,000,000 organizations, some of which have developed demonstrably effective innovations, but almost all of which are too small (less than $1 million in revenue) to bring those innovations to more than a tiny fraction of the people who need them.

The Social Innovation Fund (SIF) is an ambitious experiment to address this structural mismatch between funding the development of better ways of helping people and funding the expansion of those better ways. The popular phrase is “scaling what works,” the idea being that lots of social entrepreneurs are already figuring out “what works” on their own, but they can’t “scale” those solutions without substantial outside support.

Not so long ago, a heated debate raged about whether SIF should spur the development of new innovations or promote the expansion of “evidence-based,” i.e., established, innovations. The Corporation for National & Community Service (CNCS) resolved the dispute in favor of expansion, a policy choice I strongly support (although there are still many who do not). As CNCS puts it, “SIF will target millions in public-private funds to expand effective solutions across three issue areas: economic opportunity, healthy futures and, youth development and school support.” This approach echoes the sentiment expressed in the title of Wendy Kopp’s book, One Day, All Children ...

CNCS just completed its selection process of 11 “intermediary organizations” that “have strong track records of successfully identifying and growing high-performing nonprofits.” Over the next six months, the intermediaries must conduct their own competitions to find and fund growth-ready nonprofits with proven innovations. (In some cases, intermediaries competitively pre-selected qualified nonprofits, which CNCS reviewed and approved as part of the intermediary selection process.)

The final selection of nonprofit grantees by the 11 intermediaries will be crucial to the success of SIF’s mission of scaling what works. Obviously, the intermediaries will look for well-run, growth-ready nonprofits with innovative, effective and reliable intervention models. Although that’s a high standard of eligibility that most nonprofits don’t meet, there are far more nonprofits that qualify for SIF funding than there are funds available. Given the dire economy and fundraising market, plus lingering disappointments about the design of SIF and the selection of intermediaries, pressure will now likely shift from CNCS to the intermediary organizations to spread the final grants among as many nonprofits as possible. That would be a mistake.

The intermediary grantees comprise a diverse mix of national, regional, state, and local organizations with a variety of structures and business models. Six of the intermediary grants are for two years and five are for one, with the amounts ranging from $2 million to $10 million and the average annual grants ranging from $1 million to $10 million. The median annual grant is $3.6 million. All told, this is a heterogeneous mix, making it hard to offer generalizations about the ideal number of final grantees or size of grants. But it would inevitably dilute SIF’s ability to accomplish mission of scaling what works to increase the number and decrease the size of final grants.

The intermediary organizations were chosen for their enlightened approaches to scaling, and the applications I’ve read are impressive, so I have considerable confidence that they will make careful choices about the final grantees. Given how rapidly SIF has been launched, though, their grantmaking parameters are still somewhat fluid. For example, CNCS reports that Foundation for a Healthy Kentucky will focus on “6-10 low-income communities,” Missouri Foundation for Health will invest in “10-20 targeted low-income communities,” New Profit will collaborate with “five to six innovative youth-focused nonprofit organizations,” and the Edna McConnell Clark Foundation will fund “up to 10 youth development organizations.”

The selected intermediaries aren’t just passive conveyors of federal dollars to local nonprofits. Not only do they have to “select, invest in, support, and monitor the replication and expansion of grantees,” they’re also required to follow best practices for competitive grantmaking, data collection, performance measurement, impact evaluation, and stewardship of federal and private funds. Intermediaries will be held accountable for their grantees’ impact and growth, of course, but they’ll also have to show that they added much more value than the extra cost and red tape their participation required.

This last point is particularly important. SIF adopted a three-tiered structure because it believes (as I do) that this systemic approach can make the whole much greater than the sum of the parts. At the same time, the approach creates significant risks that would not arise if CNCS just made direct grants to growth-ready nonprofits. Even though CNCS, the 11 intermediaries and the nonprofit subgrantees all bring unique assets to SIF, the collaboration won’t succeed unless their respective contributions can be aligned and managed productively.

That kind of “alignment” is a key factor in producing what Robert Kaplan and David Norton (the developers of the balanced scorecard) call “enterprise value” in both the private and public sectors. Here’s how they explain the concept in the case of companies with separately-managed business units that must work together:

“The corporate headquarters does not have customers, nor does it operate processes that make products or services.... The corporate headquarters aligns the value-creating activities of its business units -- enabling them to create more benefits to their customers or to lower total operating costs -- beyond what they could achieve by themselves if they were operating independently.”

In a similar vein, NYC Deputy Mayor Stephen Goldsmith and William Eggers have devised an ingenious approach called “governing by network” in which the public sector acts as an “integrator” of outside third-parties to accomplish objectives that government agencies couldn’t do on their own: “A public agency can use its positional authority and perceived impartiality to bring the different parties together, coordinate their activities, and resolve any disputes.” That’s what CNCS is trying to orchestrate with SIF.

But very few organizations, whether private, public or charitable, have meaningful experience managing strategic alliances with independent organizations they don’t control. Governing-by-network is much easier said than done:

“As more and more agencies forge partnerships with third parties, agency performance will largely depend on how well the partnerships are managed. To achieve high performance in this environment, governments will need to develop core capabilities in a host of areas where today they have scant expertise.”

Think about this in terms of SIF. CNCS has to oversee the intermediaries, and the intermediaries have to oversee their nonprofit portfolios. Collectively, the enterprise is trying to significantly increase the impact and scale of the funded social innovations. To be deemed successful, all three layers have to manage the funding, monitor performance metrics, measure impact, and foster growth while maintaining quality, which means they have to develop and manage the tools, processes, infrastructure, and organizational capacity to carry out those functions, all of which requires dedicated funding, skilled professionals and other scarce resources. At the end of the day, CNCS will have to convince skeptics and supporters alike that the results of the experiment were worth the extra effort and expense.

As someone who strongly believes that SIF is on the right track, I encourage the intermediaries to avoid making their job more challenging than it already is. It stands to reason that, if you’re in the business of herding cats, the fewer the better. Even one or a few more nonprofits in the portfolio will multiply the complexities of intermediation, and I suspect for the really crucial grantees, the incremental increase in funding would provide a welcome increase in their margin-for-error, one that might prove much more valuable in difficult times ahead than potentially underfunding additional recipients.

These are only the first round of short-term SIF grants, and they could gain the public support needed for expansion if, but only if, their success is clear. Both CNCS and the intermediaries should have the courage of their convictions about SIF’s mission and go all in on scaling what works. Find the very best high-performers and give them as much growth funding as you can.

The Social Innovation Fund Kerfuffle

2010-08-22T20:32:00.010-04:00

This is a response to an article published in the August 19, 2010 New York Times, "Nonprofit Fund Faces Questions About Conflicts and Selection Procedures," which is available here.

The social sector has long been a recipient of federal, state and local government funding, but it has little experience working with government as an equal partner. Breaking with a past in which nonprofits acted almost exclusively as supplicants for federal largesse, either as contractors-for-hire or grantees, the Social Innovation Fund (SIF) vests shared responsibilities in a broad spectrum of leading social sector players, including foundations, intermediaries, and nonprofits with track records of success, not only for funding, but also for the development and administration of the program itself.

Contrary to isolated pot shots being leveled at SIF, the program is a real-world model of transparency that marks the advent of a new era in cross-sector collaboration -- unless, that is, we continue to indulge in an orgy of recrimination and second-guessing that amounts to much ado about nothing. In two previous posts (here and here), I’ve explained why CNCS has neither an actual conflict of interest nor the appearance of one, and why CNCS should not release the first-round reviews of grant applications by outside experts. In this post, I’ll offer some thoughts about the singular importance of SIF and the risks that unfounded and disproportionate criticisms will end up throwing out the baby with the bathwater.

[Author's note: Although I was involved in the SIF review process, I’m writing on my own behalf and I don’t speak for the Corporation for National & Community Service (CNCS) or any of the organizations that participated in SIF. I have not discussed this memo with or shown it to anyone else (including CNCS) prior to publication.]

SIF Purpose and History

In a speech to a joint session of Congress on May 25, 1961, President Kennedy made the paradigmatic statement of what Jim Collins later dubbed a “big hairy audacious goal,” or BHAG:

“I believe that this nation should commit itself to achieving the goal, before this decade is out, of landing a man on the moon and returning him safely to the Earth. No single space project in this period will be more impressive to mankind, or more important for the long-range exploration of space; and none will be so difficult or expensive to accomplish.”

In 1962, he added for the benefit of those who didn’t understand him the first time, “We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.”

If the nonprofit sector as a sector has ever had a BHAG, the SIF is it. Indeed, it was not just what SIF aspired to accomplish that was audacious, but also how it proposed to go about it, by actually engaging in two enlightened practices about which there has been much talk and little action for many years: collaborating across sectors and leveraging institutional co-investment.

We can’t appreciate the significance of SIF without understanding the extent to which it departs from accepted ways of doing things. In 2006, Harvard Business School Professor Michael E. Porter told The Economist’s Matthew Bishop that “Philanthropy is decades behind business in applying rigorous thinking to the use of money.” Even further back in the stone age of performance-based philanthropy -- 1999 -- Professor Porter and Mark R. Kramer, his co-founder of FSG Social Impact Advisors, wrote a seminal article in the Harvard Business Review, “Philanthropy’s New Agenda: Creating Value.” They argued that “foundations create value when their activities generate social value that go beyond the mere purchasing power of their grants,” and identified four ways of doing so: (1) selecting the best grantees; (2) signaling other funders; (3) improving the performance of grant recipients; and (4) advancing the state of knowledge and practice.

SIF does all four.

Porter and Kramer have not been alone in calling for a fundamental restructuring of the nonprofit capital market:

“The core problem is that our education and training systems were built for another era, an era in which most workers needed only a rudimentary education. It is not possible to get where we have to go by patching that system. There is not enough money available at any level of our intergovernmental system to fix this problem by spending more on the system we have. We can get where we must go only by changing the system itself.”

“Tough Choices or Tough Times,” The New

Commission on the Skills of the American Workforce

“The financial system [for nonprofit enterprises] we have put in place and support is the worst enemy, not only of the improvements everyone is trying to make, but of the socially critical programs and services this system is meant to sustain. All efforts to improve the sector will be merely palliative without essential, systemic reform of the way the rules of finance work.”

Clara Miller, President and CEO, Nonprofit Finance Fund

“Many organizations with the potential to grow are unable to do so because they cannot tap into an easy-to-access capital market... There are not enough organizations able to systemically expand and strengthen their work in order to really resolve social issues.”

Arthur Wood and Maximilian Martin,

“Market-Based Solutions for Financing Philanthropy”

“The forecast is for hundreds of billions, if not trillions, of philanthropic dollars to be dropped into our industry, but given the current scattered nature of our efforts, the chances of effectively channeling these resources to substantial public good is like trying to direct the floods of the Nile River by building sand castles along its banks.”

Lucy Bernholz, Creating Philanthropic Capital Markets: The Deliberate Evolution

“Any serious discussion of nonprofit capital market deficiencies has to start with an acknowledgement that there is no obvious way out of this maze. Expanding social needs are being addressed by nonprofits with very limited funding options. Grant seekers put extensive resources into navigating unclear and unpredictable restrictions. This fragmented funding landscape weakens the sector and ultimately limits impact, detracting from the very community efforts that the funds are meant to support.”

Cynthia Gair, REDF

I collected these voices in my book and added my own two cents:

“Just as experienced financial investors find the most lucrative investment opportunities before everyone else, we need to help ‘smart money’ find the most capable nonprofits that are ready to take on $100 million problems. If we really want to help ‘all children’ but we don’t want to wait forever for ‘one day’ to arrive, we need to turn the fundraising paradigm on its head: We need a financing system that helps highly engaged social impact investors to direct third-stage growth capital to the best mid-cap nonprofits, instead of one that forces those nonprofits to spend all their time looking for more drops to fill more buckets.”

The origin of SIF traces back to the formation in 2007 of the non-partisan America Forward coalition, led by New Profit, Inc. (NPI). At the time, NPI had virtually no interaction with the public sector, but it assembled “more than 90 results-driven, entrepreneurial nonprofit organizations, grant-making intermediaries, and their partners” around a shared vision that

“one day, our nation’s leaders and citizens will work together to foster innovation and high-impact results in the social sector, identify what works, and grow the most effective solutions to wherever they are needed.”

In a 2007 briefing book, America Forward proposed what it then called the “Social Investment Funds,” which would “make it easier for policymakers, and private-sector funders, to seed innovation, extract what works, identify why it is successful, and take solutions to scale.” In May, 2009, First Lady Michelle Obama announced the $50 million SIF:

“The idea of the fund is simple: find the most effective programs out there and then provide the capital needed to replicate their success in communities around the country. By focusing on high-impact, results-oriented nonprofits, we will ensure that government dollars are spent in a way that is effective, accountable and worthy of the public trust.”

Just one year later, on May 27, 2010, SIF and some of its partners made three big announcements:

Five of the country’s most forward-thinking foundations -- Eli and Edythe Broad, John and Ann Doerr, Omidyar Network, Open Society Institute, and Skoll -- pledged $45 million over two years to help provide matching funds for SIF grantees. 
140 community foundations signed a letter to President Obama from the Council on Foundations that endorsed SIF “as a tool to find and invest in more community-based solutions.” 
22 grantmakers committed nearly $5 million for three years to an independent coalition called “Scaling What Works” to “extend the reach, learning and impact of the Social Innovation Fund and other philanthropic efforts to bring effective nonprofit programs to scale.” Members include Annie E. Casey, Atlantic, Gates, Carnegie, Mott, Packard, Edna McConnell Clark, Knight, Kresge, Lumina, Robert Wood Johnson, and Kellogg. Its objectives are: (1) Serve as ongoing convener and conduit between the field of philanthropy and the public agencies involved with the Social Innovation Fund with the hope of building effective partnerships between philanthropy and the public sector that can speed the pace at which community solution scale; (2) Expand the number of grantmakers nationally who are prepared to support the evidence base, capacity and growth of promising nonprofits; and (3) Support collaborative learning and action among the network of grantmaking intermediaries funded by the SIF so they can most effectively invest public and private resources, and so the lessons they learn are translated for the larger field.

SIF Purpose and Structure

Grantmakers for Effective Organizations (GEO), which is leading Scaling What Works as part of its “GEO Action Network,” summarized the objectives of SIF:

Promote public and private investment in effective and potentially transformative nonprofits to help them strengthen their evidence base and replicate and expand to serve more low-income communities; 
Identify more effective approaches to addressing critical social challenges and broadly share this knowledge; and 
Develop the grantmaking infrastructure necessary to support social innovation in communities across the country.

This “grantmaking infrastructure” is so important for the future of social change. As Porter and Kramer pointed out, it’s not just the direct grant dollars that matter; value comes from producing “social impact disproportionate to their spending.” As summarized in today’s otherwise depressing New York Times story, “Nonprofit Fund Faces Questions About Conflicts and Selection Procedures,” the SIF is specifically organized to produce that kind of value:

“The 11 winners effectively serve as conduits to channel the grant money to other nonprofit organizations that operate successful programs that can be expanded to serve more people in more areas. The winners must match the government’s money, which also must be matched by the final recipients, potentially trebling the fund’s financial effect. The broader goal ... is to develop a network of intermediaries like the grant winners that can identify promising programs and connect them to donors and other sources of financing to allow them to expand.”

Folks who’ve never worked in or with government don’t realize how rare and precious the SIF is. Before we snatch defeat from the jaws of victory, consider what’s at stake.

The economy is still in serious trouble and the deficit is blocking far more important programs than SIF. As a result, there isn’t going to be any more government funding for the social sector, period.
It is orders of magnitude more difficult to prevent or stymie a new federal program than to successfully navigate one through the thickets of competing stakeholders. For every SIF grantee, there are nearly four disappointed applicants who know reviewers, reporters, bloggers, and friends on Capitol Hill. There’s no downside for an unsuccessful applicant to whisper any rumor that might disqualify a grantee and free up a slot; even if they take down the whole program, they’re no worse off. The media thrives on gossip, second-guessing and finger-pointing, no matter how thinly supported.
To mangle a phrase from Finley Peter Dunne, transparency, like politics, ain’t beanbag. Sausage-making is always ugly. Without exception, every large project with real money at stake that involves scores of people has an embarrassing paper or email trail. When the media and (former) congressional staff gather in the cause of “controversy,” everything is eagerly taken out of context and cooler heads almost never prevail. After all, “questions have been raised.” If I had access to the kinds of documents that critics are calling on CNCS to release, I could make any public, private, charitable, or religious endeavor look bad.
Public scrutiny is a blunt instrument. You can’t calibrate how much a process is “sullied” by unsubstantiated rumors or when the failure to satisfy a small handful of “surprised” participants becomes “stonewalling.” That’s why CNCS tried to overcompensate with duplicate panels to review every application. Transparency means providing sufficient information and inviting sufficient participation to insure fundamental fairness. It does not require unanimous agreement about each application at each and every step and every final decision. Neither perfection nor the absence of “questions” or “controversy” is the standard. Sorry, folks, but when it comes to government transparency, this is about as good as it gets.

Keep in mind that the President and First Lady of the United States invested their personal political capital in SIF. Why in the world would they do that for a tiny federal program that doesn’t come close to their top ten priorities? Why would SIF staff risk their careers on anything that challenges the status quo, no matter how valuable, if they can be pilloried even after they’ve recused themselves from any conflicting entanglements? Why would any government agency ever try another partnership-of-equals with the nonprofit sector? If SIF goes down, neither it nor anything remotely like it is coming back any time soon. For anyone who wants to strangle SIF in the crib, this is how to do it.

And the really sad thing is, we have even gotten to the hard part of SIF yet. Writing all those procedures, enlisting all those volunteers, shepherding them through weeks of reviews, and then reviewing the reviews for errors and inconsistencies is just mechanical stuff. Even the final round of grantmaking by the selected intermediaries will be a walk in the park compared to the real challenge looming ahead: showing that the SIF model can actually scale what works to an extent that isn’t otherwise possible. We’ve talked a good game about social transformation and the time will come soon -- assuming SIF doesn’t blow up first -- when we have to deliver.

And we’re being held hostage to typos and grammatical errors.

Just last week, new data showed that flawed 2008 research had vastly overstated the gains that New York City public schools had made in reducing the achievement gap between black and white students. What had been thought to be reductions in test-score differentials of about 50% turned out to be modest at best. By happy coincidence, a recent graduate of the Harris School of Public Policy at the University of Chicago, sent me an October 2009 research paper from the Federal Reserve Bank of Chicago on the Rosenwald Fund which showed how an innovative and highly risky public-philanthropic partnership not unlike SIF could scale what works to bring proven innovations to exponentially more people in need. Here’s the abstract:

“The Black-White gap in completed schooling among Southern born men narrowed sharply between the World Wars after being stagnant from 1880 to 1910. We examine a large scale school construction project, the Rosenwald Rural Schools Initiative, which was designed to dramatically improve the educational opportunities for Southern rural Blacks. From 1914 to 1931, nearly 5,000 school buildings were constructed, serving approximately 36 percent of the Black rural school-aged Southern population. We use historical Census data and World War II enlistment records to analyze the effects of the program on school attendance, literacy, high school completion, years of schooling, earnings, hourly wages, and migration. We find that the Rosenwald program accounts for at least 30 percent of the sizable educational gains of Blacks during the 1910s and 1920s. We also use data from the Army General Classification Test (AGCT), a precursor to the AFQT, and find that access to Rosenwald schools increased average Black scores by about 0.25 standard deviations adding to the existing literature showing that interventions can reduce the racial gap in cognitive skill. In the longer run, exposure to the schools raised the wages of blacks that remained in the South relative to Southern whites by about 35 percent."

At a time when our very best social enterprises reach fewer than 3% of the people who need their help, we can’t afford to treat minor misdemeanors as major felonies. But as Professor Porter once said, “Foundation scandals tend to be about pay and perks, but the real scandal is how much money is pissed away on activities that have no impact. Billions are wasted on ineffective philanthropy.”

SIF is working in a conscientious and open way to change that, and it needs our support.

Response to Nonprofit Quarterly Editorial

2010-08-20T20:11:00.007-04:00

This is a response to an editorial published in the Nonprofit Quarterly, "Social Innovation Fund Disclosures Good But Insufficient," which is available here.

Although I was involved in the SIF review process, I’m writing on my own behalf and I don’t speak for the Corporation for National & Community Service (CNCS) or any of the organizations that participated in SIF. I have not discussed this memo with or shown it to anyone else (including CNCS) prior to submission to NPQ.

The editorial states that you are “concerned whether or not SIF can demonstrate that no pre-existing relationships sullied the high profile grantee selection process.” You urge CNCS to release “a list of the names of applicants, the original applications, the review panel’s comments and ratings, and, if they were not assured of confidentiality, the names of the expert reviewers.” In fact, there is no basis at all to question whether “pre-existing relationships sullied” the process, and the release of information you request would undermine the integrity of that process. Now that the grantees have been announced, I do happen to agree that the names of the reviewers should be made public, but without any information about who reviewed which applications and certainly not the specific ratings or written comments any of us made, for reasons I’ll explain.

I’ll begin with my own disclosures. I was one of the 48 external reviewers chosen to review the intermediary grant applications in an early phase of the selection process. I received an honorarium of $700 for well over 75 hours of work. I have professional connections of varying degrees with several of the applicants, all of which I was required to disclose in writing prior to my selection as a reviewer, and I did not see any materials of any kind related to those applicants. One of the applicants that I reviewed and rated highly was named one of the 11 grantees. I also happen to be a lawyer with many years of experience in public ethics law, include actual and potential conflicts of interest. I have never met SIF’s Director, Paul Carttar.

To put things in perspective, consider this quote from of a blog by Adin Miller, “Analysis of Social Innovation Fund Update,” posted on April 23, 2010, and available here. The call to which he refers was a media advisory conference call held by senior SIF staff, including Director Paul Carttar, a press release and audio recording of which were published on SIF’s web site after the call.

“What was amazing about the call was the level of transparency applied to the Social Innovation Fund and the grantmaking process in general. As Mr. Carttar noted, the Corporation has stressed an open and transparent process, which has been carried out throughout the development process for the Social Innovation Fund. While the Corporation won’t release the names of the applicants, its willingness to shed light on the applications received and its upcoming review process must be recognized and applauded. The field of philanthropy, in which grantmakers commonly cloak such information, can learn to adopt similar standards of transparency.”

The primary “fact” upon which you rely to question the integrity of the process is the SIF’s past relationship with one of the applicants:

“One of the 11 grantees of the Fund in 2010, New Profit, is an organization with which the director of the SIF, Paul Carttar, has had close ties. Upon taking on the role at the Fund, he signed a Conflict of Interest waiver due to those ties. This creates circumstances that heighten the need for transparency.”

To the contrary, Mr. Carttar’s disclosure and his recusal from all proceedings related in any way to SIF participants with which he had prior business dealings eliminate any need for the kind of false transparency you advocate. As you acknowledge, Mr. Carttar complied with the federal Ethics in Government Act by filing a written disclosure of all of his prior business relationships with potential SIF participants before he took the job. The waiver letter dated April 5, 2010 is available online here, and it shows why your concerns are unfounded.

First of all, you are mistaken when you say CNCS’s August 9th response “explained for the first time in detail its many-layered process …” In fact, back in April, the waiver letter itself provided a detailed explanation of the process:

“First, applications will be reviewed for compliance with the NOFA submission requirements, including a determination of whether the applicant is an entity eligible for award. The initial review of eligible applications will be conducted by a four member panel to determine an applicant’s ability to select, support and monitor the performance of a portfolio of innovative and effective nonprofit organizations. At this stage of the review (the “blended review”), panelists will apply the rubric in the NOFA, which puts 45% weight on the review of Program Design, 35% on Organizational Capacity, and 20% on Budget. Next, at a first consensus meeting, senior staff at CNCS will review the top applications and evaluate them in light of the goals of the SIF. The results of this meeting will be to confirm the group of approximately 30 applicants that will be sent forward to the “expert review” stage, where panels of two expert reviewers will assess applications based on the same rubric utilized in the blended review process, but will focus more specifically on evaluation plans and potential for replication. The expert review will be followed by a second consensus meeting, at which information from both the blended and expert reviews will be assessed. This meeting will reduce the number of applicants to a final competitive group of approximately 10-15 intermediaries. These top applicants will be packaged for a pre-decision meeting with the senior Corporation official designated by the CEO1 (hereinafter “designated official” or “designated selection official”) to make the selections, which will include executive summaries of proposals, as well as summary information and statistics. After the pre-decision meeting, staff will have clarifying discussions with the applicants, informed by those questions and issues. They will present final proposals for the designated official’s consideration at the final decision meeting.”

The footnote after the phrase “senior Corporation official designated by the CEO” says, “The CEO [i.e., Mr. Carttar] will designate a selection official because his former employer [i.e., New Profit] has submitted a notice of intent to apply.”

You attach no weight either to the content of Mr. Carttar’s disclosure or to the judgment of the ethics official who wrote it, dismissing what you call CNCS’s “baby steps” as amounting to saying, “trust me.” That’s a shame because the explanation is quite informative and undermines your insistence that CNCS can “ensure its own credibility and power” only by disclosing the applications and panel ratings. First, here’s the ethics officer’s “authorization and limited waiver” to Mr. Carttar, which takes a quite different and much better approach:

“You are authorized to participate in matters of general applicability including general policy discussions and decisions concerning the operation of the SIF. This authorization and waiver is limited, however, in that you will recuse yourself from party-specific compliance or eligibility determinations, consensus meetings, discussions, and recommendations, or portions thereof, regarding any former employer or client … This prohibition shall include Monitor Group, New Profit, Inc., New Leaders for New Schools and Teach for All, and Kaboom! You are authorized to attend the two consensus meetings except to the extent described above. If, after these meetings, either your former employer or clients remain under consideration for the final competitive group, you will recuse yourself from the pre-decision meeting with the designated official and the final decision meeting with the designated official.” (The ethics officer made the underlinings.)

Second, the reasons for her decision include:

(1) “The grant application review process will include internal and external reviewers applying established eligibility criteria. These criteria were established in the Notice of Fund Availability developed by the Corporation prior to your appointment; (2) The grant application review process … will be managed by the Office of Grants Policy and Operations in accordance with regular agency procedures; (3) You have agreed to resign from Monitor and from your affiliations with New Profit, Inc., New Leaders for New Schools and Teach for All, and your board membership with Kaboom! (4) you will no longer have financial interests in any of these organizations, and (5) if any organization with whom you have a covered relationship applies for funding, you will recuse yourself from preparation for and participation in the pre-decision …, except to the extent set forth in this Authorization and Limited Waiver.”

When a government employee complies with a widely publicized ethics ruling by quitting his job, relinquishing financial claims and resigning from a prestigious board, and staying out of decisions that would otherwise be his direct responsibility, he’s entitled to the presumption of ethical propriety.

You equate transparency with publishing any and all documents having anything to do with the SIF process, what you call “fully enter[ing] the sunlight.” In your view, if there was a potential conflict of interest at the beginning of the process, anything less than 100% publication of every document later produced by anyone in a nonofficial capacity fails to remove the taint. That’s just irresponsible.

Disclosure and recusal shift the question from “does the government employee have a current or potential conflict?” to “has the government employee complied with the restrictions imposed in the waiver letter?” If he has complied, he has had no involvement of any kind – direct or indirect – with any application submitted by any party from which he has recused himself, including the review or recommendation of that application by government officials or outside experts. None. Public disclosure and recusal completely eliminate the need for further transparency because the government employee has absolutely no opportunity to improperly influence the process with respect to those parties. NPQ’s own disclosures in a related article provide a similar example.

Answering the only remaining question – has the employee complied with the restrictions imposed in the letter? – is the responsibility of specific federal regulators, namely the Office of Grants Policy and Operations and the Office of Government Ethics, as well the CNCS Office of Inspector General. Neither the public nor the media need the documents you request to make sure that Mr. Carttar hasn’t violated the terms of the limited waiver.

The specific documents you want CNCS to release have nothing to do with transparency. You are mistaken when you claim that the ratings and comments of the expert panels are “public records.” Rather, both are exempt from the Freedom of Information Act (FOIA) under the well-established “deliberative process privilege,” as explained by the U.S. Court of Appeals for the D.C. Circuit in Coastal States Gas Corporation v. Department of Energy, 617 F.2d 854 (1980):

“The Deliberative Process Privilege serves to … protect against confusing the issues and misleading the public by dissemination of documents suggesting reasons and rationales for a course of action which were not in fact the ultimate reasons for the agency’s action… The exemption thus covers recommendations, draft documents, proposals, suggestions, and other subjective documents which reflect the personal opinions of the writer rather than the policy of the agency.”

The ratings and comments are precisely the kinds of “predecisional” documents the privilege covers. They reflect the personal and subjective opinions of the reviewers as part of the “give-and-take of the consultative process,” made in an early round of a review process that had many subsequent phases. Their disclosure would surely inhibit candid assessments by reviewers, which were sometimes sharp and even caustic, and after-the-fact analysis in the public domain would convey the false impression of “suggesting reasons and rationales for a course of action which were not in fact the ultimate reasons for the agency’s action.” Confidence in SIF would inevitably be eroded by such a contrived “controversy.”

You then make the gratuitous comment that “according to our sources, even the expert panelists appear not to have understood” the process. Leaving aside your ironic reference to anonymous sources, the statement is completely untrue. In fact, the experts unquestionably understood the process, because the SIF bent over backwards to explain it to us over and over again in multiple web seminars, instructions and phone calls precisely because they wanted to make sure that we applied the same criteria to all applicants in the same way. And I’m not talking about rocket science here, just basic definitions and steps that were sufficiently clear the first time they were explained to us. But the staff insisted that each expert reviewer participate in every redundant forum going over the same material, and we complied because we wanted to make sure the inaugural process succeeded. The process had a lot of detailed parts for the purpose of maintaining tight control, but it was not confusing. If anything, we were too straightjacketed, but it made sense to err on the side of caution and consistency, as your editorial painfully demonstrates. Hopefully, the training process will be streamlined in the future.

You quote an unnamed CNCS “spokesperson” who spoke to the Chronicle of Philanthropy about why CNCS had promised not to make public the names of the unsuccessful applicants or their applications. That’s just sloppy journalism and completely unfair. As required by law, CNCS conducted an open bidding process that included publication in the Federal Register, which invited anyone to submit comments in response to the Notice of Funding Availability (NOFA). Whatever shorthand language the unnamed spokesperson might have used in a separate conversation with the Chronicle about the reasons for not publishing information about intermediaries that weren’t selected, the actual reasons – the official reasons – for deciding not to do so came out of that legally-required public comment process. Taking public comments seriously is one hallmark of government transparency, whether or not you happen to agree with the public.

Nor is there anything “confounding” about why unsuccessful applicants might not want their submissions disclosed. As a World Bank employee stated in the New York Times on August 16, 2010, “the private sector talks about failure freely and candidly,” while the nonprofit world “has to worry about donors who don’t want to be associated with failure and beneficiaries who may not benefit from admissions of failure.”

Given all of the above, you have no basis to lampoon CNCS’s careful, conscientious and open process as a “restricted access, need to know, hush-hush operation.” It is easy to talk about the virtues of transparency without accepting any of the attendant responsibilities. Unlimited transparency can interfere with thorough and deliberate decision-making, particularly when raw information never intended for public dissemination becomes fodder for sensational and one-sided cheap shots and second-guessing in the guise of serious analysis. SIF ran a painstakingly open process for selecting intermediary grantees, and it followed its rules to a fault, subject to review by numerous oversight agencies. It has promised to provide additional information soon, but it is right to draw the line at releasing more than a thousand pages of unofficial and subjective documents prepared by private experts expressing their personal opinions which may or may not have shaped the final official decisions. No one should care whether I and my colleagues gave applicant X a 2 instead of a 3 on question Y.

There is not the tiniest bit of evidence that “pre-existing relationships sullied the high profile grantee selection process,” or even the appearance of such. If NPQ wants to serve the public interest, rather than its own prurient interest in obtaining as much blog fodder as possible, it should undertake a substantive analysis of the SIF information will release about the actual basis for the final decisions made.

It is also too easy to be cynical about government and to wrap oneself in the mantle of “transparency,” but baseless cynicism does not serve the public interest. This is a case where CNCS has established a well-designed program with extensive input from the social sector, and it has been appropriately transparent within any reasonable definition of the term. The publication of peripheral documents would force CNCS to divert its small staff and budget to responding to endless second-guessing, rather than insuring that the important and extremely challenging work of SIF continues productively. That might be a desirable outcome for NPQ, bloggers and SIF critics who want to continue fighting battles decided long ago, but it would not be good for the social sector or the people who need the ultimate SIF grantees to make their proven innovations available nationwide.

Sincerely,

Steve Goldberg

Response to Paul Light re SIF "Stonewalling"

2010-08-20T08:14:00.004-04:00

Let me see if I have this right. SIF is "becoming one of the most important efforts to stimulate social change in recent history," its selection process was conducted in record time and resulted in choosing 11 of "the very best organizations in the country," with "no pressure" to favor any applicants, using "perfectly appropriate" criteria. As a fellow SIF reviewer, I agree.

But, yet, you making sweeping assertions that SIF is "becoming a study in what doesn't work in government transparency," based on "rumors," "controversy" and "surmise," a fancy word for guess. Your sole basis is that you claim to know of "at least one" "weak and nonresponsive" applicant that received a grant, although you state that you "have no idea how this applicant reached the winner's circle."

Maybe you're right about that applicant, but you're inflammatory rhetoric and eager leaps to speculative conclusions can only force everyone involved to run for cover because "questions have been raised." You mention "the applicant's impressive lobbying effort on behalf of SIF" to imply that the applicant might have done something nefarious, offering wild conjectures about "revised" applications and unexplained "clarifying discussions." Soon, no doubt, we'll be hearing sage pontifications along the lines of "doesn't the White House know the cover-up is always worse than the crime."

This is sad and unfortunate. Even with its tiny size, SIF represents and important and courageous experiment by a forward-thinking administration to promote social progress by combining what the government does well (funding programs at scale) with what the social sector does well (fostering innovative solutions to difficult and incapacitating problems). It has attracted private funding from some our best foundations that exceeds the taxpayer money committed.

You have raised questions about just 1 of 69 applications, which can be investigated by the responsible oversight agency, the Office of Grants Policy and Operations. There is no basis to cast SIF's response as "weak and non-responsive," yet you seem willing to throw SIF to the wolves and let a promising cross-sector innovation become engulfed in a feeding frenzy of speculation and second-guessing. Are you really prepared to deny the beneficiaries of the final grantees the unprecedented financial and management leverage that SIF is on the verge of producing?