The Evidence: Ostrom

What she found · Eight design principles for durable common-pool-resource institutions. When · Coding manual in 1989, synthesis in 1990, later diagnostic turn through 2010. Who cited whom · This is an independent route to multi-level governance. Evidence base · More than a thousand collected cases, a 455-question instrument, 86 coded cases, and three later reanalyses. What it costs us · Nesting is the least-supported principle in the 2010 review, and this literature published disconfirmations the VSM literature has not equipped itself to produce.

An independent route, not a rival program

Irrigation systems, inshore fisheries, mountain commons, groundwater basins: Elinor Ostrom wanted to know why some of them survive for six hundred years and others collapse in a generation. She was not studying firms. Governing the Commons appeared in 1990; the Nobel followed in 2009.^[1]

We have cited her for a long time, and warmly. Nested, multi-level governance, arrived at from irrigation ditches by a scholar whose cybernetics came from Ashby rather than from Beer,^[2] is among the better reasons to think the model’s components are not arbitrary. Chapter 10, Six Components, makes that argument, and this chapter does not withdraw it. What we have not done is read the rest of her literature with the attention we gave the part that flattered us. Her tradition wrote down what it was looking for before it looked, coded a large number of cases against that written instrument, submitted the result to independent reanalysis three times, and printed the findings that went against it. Two of those findings go against us as well. We cited this literature as corroboration and did not report where it goes against us. This chapter repairs that omission.

Ostrom’s program is no opponent, and the chapter keeps no score. Her literature is the nearest available example of what a comparable body of work looks like when a community decides to gather evidence about its own claims. The comparison with our own record is unflattering to us, and the unflattering parts should be stated precisely, neither dramatized nor waved through.

The eight principles, read with the model in hand

Her eight design principles for long-enduring common-pool resource institutions read as follows. Clearly defined boundaries. Congruence between the appropriation and provision rules and local conditions. Collective-choice arrangements, in which most of those affected by the operational rules can participate in changing them. Monitoring by monitors who “are accountable to the appropriators or are the appropriators”. Graduated sanctions. Rapid, low-cost, local conflict resolution. Minimal recognition by external authorities of the right to organize. And, for resources embedded in larger systems, nested enterprises, with appropriation, provision, monitoring, enforcement, conflict resolution and governance “organized in multiple layers”.^[3]

Four of the eight land on our own territory, and we put none of them there. The eighth is nesting, reached from irrigation ditches rather than from Beer’s derivation. It resembles recursion and is not the same claim. Beer’s recursion requires each embedded layer to be a viable system in its own right; Ostrom’s principle asks that activities be “organized in multiple layers of nested enterprises.”^[4] The eighth principle is convergence on a related but not identical object. The third is a legitimacy condition on System Five↗: those who live under the operational rules can take part in changing them. That constrains where policy may be made. Beer comes near it: System Five “consists of many people,” should include workers, and later represents all stakeholders, and divisions should preferably take part in formulating objectives.^[5] But he files participation under “the psychology of the system (and not its logic)”; Ostrom makes it a condition of endurance. The fourth is the audit channel (the Three-star of Diagnosing the System) with an ownership requirement Beer did not impose, since a monitor accountable to the appropriators is a different instrument from a monitor accountable to the center. Beer’s audit runs “by agreement with System One management,”^[6] which protects the audited unit’s consent and does not make the auditor accountable to it. Chapter 4, Channels Without Owners, takes that requirement up, because our specification of the audit channel says who may open it, how often and with whose agreement, and says nothing about whose interest the auditor serves. The seventh is a precondition the VSM places and does not regulate. In the model, the right to self-organize lies with the containing recursion, whose metasystem has “the authority to delimit the autonomy of any … System One” by the cybernetic rules.^[7] The model has no case in which that authority dissolves the system, or is held by a power the cybernetic rules do not bind. Chapter 8, The Loop That Stayed an Intention, uses it as a boundary condition on the whole apparatus. Chile in September 1973 is the proof of Ostrom’s point about Beer.

The other four do not map, and that is informative too. Graduated sanctions and rapid local conflict resolution are not variety-engineering constructs, and the model offers no guidance on how to build either. Clearly defined boundaries and congruence with local conditions come closer to the recursion criteria we do discuss, but they are stated as properties of a rule set rather than of a system’s structure. Independent arrival should produce a partial overlap. A complete one would more likely mean that we had read the mapping into her table than found it there.

How the evidence was built

The Workshop’s coding manual was built by evaluating questions against more than a thousand unpublished case studies, and it runs to four hundred and fifty-five coding questions. Eighty-six cases were systematically coded. Fourteen are read in depth in the book, and five of those fourteen are failures.^[8] That evidence is what makes this material belong in a book about our own poverty. Including the failures makes the principles falsifiable rather than an anthology of successes. Beer argued for the idea: he proposed “organizational pathologists” to do post-mortems on systems that are viable no more, and he reported failures of his own.^[9] The VSM literature has never included failures in a coded sample.

The instrument matters as much as the count. Because the coding questions were written down and published, a later researcher could recode the same cases, disagree with the original judgments in public, and show her work. That is how the reanalyses below became possible, and it is why the numbers in this chapter can be argued with at all. Our own case literature has no equivalent artifact. Beer published a diagnostic workbook whose charts readers may reproduce, and a fifteen-question identity instrument with card scoring and clustering.^[10] Neither was used as a coding instrument across cases, neither carries success criteria stated in advance, and nobody has recoded a case with either. So a skeptical reader has nothing to recode, and nobody outside a given consultancy can establish that a reported success was a success by a standard fixed before the engagement began.

How the evidence was tested

Then the principles were tested. This is the part that matters most, and the part we have no counterpart to. A 2010 review of ninety-one studies found them moderately supported overall, with a revealing gradient: the empirical studies were more supportive than the abstract ones, and the purely theoretical studies were actively unsupportive.^[11] The gradient admits more than one reading. People who watch an irrigation system for a year may see structures that a modeler working from published summaries cannot. Or fieldworkers who know the principles may simply find them. The review does not settle which, and neither do we.

A 2016 reanalysis of sixty-nine recoded cases, using qualitative comparative analysis, concluded that “no DP, alone, leads to successful outcomes… but a combination of more than eight DPs seems to greatly increase the odds of success”. The principles work configurally, and which configuration works depends on the infrastructure.^[12] That substantially qualifies the design-principle idea as it is usually quoted, and it was published by researchers working inside the tradition rather than against it.

A 2017 review of fifty-four papers across every inhabited continent found the two enforcement principles, monitoring and graduated sanctions, consistently weak in practice. It reported that the principles “do not conclusively diagnose the functionality of young and viable CPR institutions, whereas they may do so for either the short-lived (failed) or the long-lasting institutions.”^[13] Each of the three studies makes the underlying claim smaller and more conditional than the version that circulates in secondary citation. None of the three was suppressed, and the people whose work they qualify cite all three routinely.

Three findings from that literature should make uncomfortable reading for us.

The first finding: nesting is the weakest of the eleven

In the 2010 review the least supported of the principles is the nested one, the principle closest to recursion. Nested enterprises came out alone at the bottom of the eleven. That is still within the range the reviewers call moderately supportive, but nesting is the only one of the eleven whose association with success failed to reach significance at the five percent level. The support ratio is 2.0, at the very edge of the weakest band the authors use; Fisher’s exact test gives P = 0.065; the effect size, phi = 0.262, is the smallest of the set. Proportionality between what a member puts into a commons and what they take out came out near the top, and it is also one of the two conditions the 2016 configural analysis found holding across all three sectors. Nesting held nowhere in particular.^[14]

We had counted this place as support. Chapter 10 lists Ostrom’s nested governance, arrived at without Beer, among the six independent arrivals that make the model non-arbitrary, and marks it as convergence on a related object. It is an independent arrival. It is also the one her own literature supports least, and both things are true at once. Chapter 5, The Ladder Without a Ruler, works with this result at length. The result does not disconfirm recursion. Principle 8 corroborates recursion, and the 2010 result counts against it, only to the extent that Ostrom’s nested enterprises are themselves viable systems, and nobody has coded them for that; the second arm of the pilot priced at the end of this chapter would. The further problem it raises for us is that we have no scale on which recursion could be graded in the first place.

Two qualifications are owed here, and they run in our favor without rescuing us. First, a support ratio of 2.0 is a weak positive, not a negative: the cases where nesting was present and the institution endured still outnumber, two to one, the cases where it was present and the institution failed. Second, the 2017 review found principle 8 largely inapplicable to its sample, because most of the resources studied were small in scale. A principle stated only for systems that are parts of larger systems will always be tested on an unfavorable sample when the sample is dominated by systems that are not. What we cannot do is quote the eleven-principle result as evidence for recursion while quietly omitting where nesting placed in it.

The second finding: viability confirmed only in retrospect

The second is the survivorship structure of the whole exercise. The principles were derived from institutions selected for having endured, so endurance is built into the sample before any coding begins. A set of principles that can diagnose the long-lasting and the already-dead, but not the young and viable, describes what survival looks like in retrospect. It offers no instrument for the present tense.

VSM practice makes exactly the same move, and makes it without the coding. Beer did not. “The fact that it is there does not prove that it is effectively there, witness universities, nor efficiently there, witness hospitals,” he wrote, and he proposed forward-looking measures: “nobody is measuring stability. It can however be done.”^[15] The community did not build or test them. In practice we define success as viability, and viability is only ever confirmed afterwards. A firm that is still trading is evidence for whatever diagnosis was applied to it; a firm that failed is evidence that the diagnosis was not implemented properly. Chapter 9, Why None of This Ever Cost Us Anything, treats this as the central mechanism by which our claims have avoided cost. Ostrom’s literature shows the structure most clearly, because there the researchers themselves named it and printed it in a journal.

The third finding: Ostrom foreclosed the blueprint reading herself

The third is Ostrom’s own. A note in a 1992 working paper she wrote with Michael McGinnis reads: “We do not think it is possible to elucidate necessary AND sufficient principles for enduring institutions as it takes a fundamental willingness of the individuals involved to make any institution work.”^[16] The two authors foreclosed the blueprint reading themselves, eighteen years before the meta-analyses reached the same conclusion, and Ostrom spent her last decade arguing against panaceas and for a diagnostic approach.^[17] Beer hedged the status of his model more than once: “There is no ‘correct’ interpretation of the VSM,” and the choice of five systems is “somewhat arbitrary.”^[18] He also called it “a cybernetic model of the necessary and sufficient conditions of any viable system,”^[19] which is the claim Ostrom refused to make. In the texts we know, nobody has withdrawn it.

A disclaimer is no virtue in itself. This one was load-bearing. It set the expectation against which the later reviews were read, so that a moderate result could be reported as a moderate result rather than as an embarrassment. We have no such statement on record, and the absence has a cost. Because we have never set out in print what the model cannot do, every finding that the model did not do it arrives as an attack rather than as information.

Where the comparison does not transfer

Common-pool resource institutions are not firms. Membership is frequently inherited and tied to a place, exit is expensive or impossible, and the resource is a physical stock with a regeneration rate that sets the time constants of the whole system. There is no capital structure, no employment hierarchy, and often no competitor. The governance problem is appropriation from a shared stock under temptation to overuse, which is not the problem a manufacturer or a hospital or a software company is solving. And no commercial organization is measured against their success criterion: the institution and the resource are both still there in a century. Ostrom herself later took part in an attempt to carry the principles across. With David Sloan Wilson and Michael Cox she argued that they follow from more general evolutionary principles and apply to a wider range of groups than those that manage common-pool resources.^[20] The differences just listed are the ones such a generalization has to survive in a firm, and Chapter 4 asks the same question of the fourth principle.

So the support ratios cannot be carried across as though they were measurements of the VSM, least of all when they happen to suit us. The argument cuts both ways, and the symmetry is the honest part. If the weakness of nesting in Ostrom’s sample is only partial evidence against recursion in firms, then the presence of nesting in her table is only partial evidence for it, and the independent-arrival argument in Chapter 10 is correspondingly weaker than the confident form in which we have often made it. A claim cannot be strong corroboration on the way in and weak analogy on the way out. The gap between nesting and recursion narrows the argument once more in both directions, since nesting bears on recursion only where the nested units are viable.

What does transfer is the method: a published instrument, cases coded against it before the outcome is known, independent recoding by people free to disagree, and the disconfirmations printed alongside the rest. None of that is specific to irrigation. It is available to any community with archives and the willingness to open them.

The comparison that can be made

The comparison that can be made precisely is with Ostrom, and it is worse. On her side stand more than a thousand collected case studies, a four-hundred-and-fifty-five-question coding instrument, eighty-six systematically coded cases, three independent review and reanalysis studies covering ninety-one, sixty-nine and fifty-four cases respectively, and a research program that published its own disconfirmations. On ours, a 2025 systematic review retained twenty-one studies from seven hundred and sixty-eight Scopus records.^[21] Eighty-six coded cases against twenty-one studies is a factor of four, so the gap is not really one of counting. It is that one side coded its cases against a published instrument, had the result independently tested three times, and printed the findings that went against it, and the other side has done none of those things.

Part of the difference is resources, and it would be unfair to ourselves to ignore that. Ostrom had a research workshop, doctoral students, decades of funding and a discipline that rewarded the coding of cases. We have consultants with client confidentiality agreements and a small academic literature with no dedicated institution behind it. That explains the difference in counts. It does not explain the three omissions that would have cost us almost nothing: writing down what a successful diagnosis would look like before making one, letting a second person recode a case, and publishing an engagement that did not work.

What this chapter hands forward

Build the register. Ostrom’s instrument is published and could be adapted rather than invented. A registry of VSM interventions with pre-stated success criteria, published boundary judgments and reported failures is the single highest-value program available to this community. It requires no theoretical advance whatsoever, only the willingness to be seen losing.

Priced pilot · Adapt the CPR manual to ten accessible VSM engagement archives. Two researchers can define the codebook, double-code the cases, adjudicate disagreements and publish the pilot in approximately thirty working days, excluding archive permissions. A negative result — diagnoses do not predict the stated outcomes, or cannot be coded reliably — counts against the model and must be published. Chapter 29, The Two Cheapest Decisive Experiments, builds the register of interventions and the prediction test on it. This chapter prices only the retrospective pilot. An optional second arm: two coders apply Beer’s criterion of viability, “able to maintain a separate existence,” to the nested enterprises in a subsample of the coded CPR cases and report their agreement; its price is still to be set.^[22]

And take Ostrom’s seventh principle as a constraint on cybernetics itself. No amount of internal viability survives the withdrawal of the right to self-organize. That right is a precondition for any design, and the model places it in the containing recursion without regulating its withdrawal. It belongs with the question of who may draw a boundary, which Chapter 29 keeps separate from what the data support. Our best-known case study is the proof. Asked why his team could not have designed for the pressure from outside Chile, Beer answered that it was “like complaining that man, who is supposed to be an adaptive biological system, cannot adapt to a bullet through the heart.”^[23] Ostrom’s seventh principle turns the same condition into a design question: the right to organize has to be recognized and kept, not only survived. We have spent fifty years telling Chile as a tragedy and have not read that difference as a finding.