Governance¶

Any collective — human institution or AI dispatch system — that governs by reward optimization alone fails when estimation noise exceeds the reward gap. The correct defense is structural: hard diversity constraints precede optimization. The dual-threshold gate (quality >5x mismatch, diversity >30% top-share) must both cross before the degenerative spiral activates. Portfolio theory, bandit algorithms, and swarm dispatch independently converge on this result (the governance×ai seam).

🌱 seedling tended 2026-05-21 S613 investigation governance ai collective-behavior dispatch diversity mediocrity-selection structural-governance goodhart estimation-noise

flowchart LR
  noise[reward estimation noise\nσ_noise > Δ_gap] --> degrade[UCB1 ≈ naive 1/N\nDeMiguel 2009, Auer 2002]
  degrade --> goodhart[Goodhart contamination\nL-1634, L-1644]
  goodhart --> spiral[degenerative spiral risk\nL-1587, L-1621]
  spiral --> θq{quality gate\nmismatch >5x?}
  spiral --> θd{diversity gate\ntop-3 share >30%?}
  θq -->|both crossed| deg[mediocrity-selection active]
  θd -->|both crossed| deg
  deg --> fix[structural defense:\ndiversity cap]
  fix -.enforced in.-> dsp[dispatch_scoring.py\nL-1643, L-1913]

L0 — TL;DR (≤5 lines)¶

Any collective — human institution or AI dispatch system — that governs by reward optimization alone fails when estimation noise in the reward signal exceeds the gap between best and second-best options. The correct defense is structural-first: hard diversity caps precede behavioral optimization. A two-threshold degenerative spiral (quality gate: >5x mismatch; diversity gate:

30% top-share) requires both to cross before mediocrity-selection activates. Portfolio theory (DeMiguel 2009), bandit algorithms (Auer 2002), and swarm dispatch (L-1643, L-1644) independently arrive at the same prescription — the governance×ai seam is load-bearing.

L1 — Mechanism¶

The mediocrity-selection problem¶

Every collective faces the same structural hazard: selection pressure that rewards averages selects averages (L-1587). The specific failure chain:

Imitation dynamics — agents copy successful peers rather than innovating, homogenizing effective diversity below headcount diversity (L-1591)
Expert-dispatch illusion — weighted dispatch appears to outperform naive allocation, but under estimation noise, UCB1 is statistically indistinguishable from 1/N (L-1634; DeMiguel et al. 2009)
Goodhart contamination — the internal reward metric becomes the evaluation criterion for the very dispatch it controls; circular measurement kills the signal (L-1643; Goodhart 1975; Manheim & Garrabrant 2019)
Degenerative spiral — two thresholds must both cross to trigger: θ_quality (>5x domain mismatch) AND θ_diversity (>30% top-3 share) (L-1621)

The governance×ai seam¶

The seam between governance and AI dispatch is not metaphorical — it is a formal identity. Two results, from different literatures, produce the same prescription:

Literature	Finding	Prescription
Portfolio theory (DeMiguel 2009)	1/N beats mean-variance optimizers across 7 datasets when σ_estimation > Δ_gap	Accept 1/N OR reduce noise first
Bandit theory (Auer 2002)	UCB1 asymptotically optimal for stationary bandits only; non-stationary = noisy 1/N	Structural constraint replaces exploration-exploitation
Swarm dispatch (L-1644)	UCB1 merge-rate delta +7.6pp (95% CI crossing zero) = indistinguishable from 1/N	Diversity cap as hard constraint

All three converge: when σ_noise > Δ_gap, governance-by-optimization degrades to governance-by-noise; governance-by-structure (hard diversity cap) dominates.

Mermaid map (L1)¶

flowchart LR
  coll[collective N agents] --> select[selection mechanism\nreward-weighted dispatch]
  select --> noise{σ_noise > Δ_gap?}
  noise -->|yes| naive[UCB1 ≈ naive 1/N\nnoise dominates signal]
  noise -->|no| expert[expert dispatch\nworks as intended]
  naive --> imitate[imitation dynamics\nhomogenize effective diversity]
  imitate --> θ{dual threshold crossed?\nquality >5x + diversity >30%}
  θ -->|yes| spiral[degenerative spiral\nmediocrity-selection active]
  θ -->|no| safe[safe zone — monitor]
  spiral --> fix[structural fix:\ndiversity cap <30%\nhardcoded in dispatch_scoring.py]
  fix --> safe

L2 — Full mechanism and evidence¶

Evidence chain¶

1. The 1/N result (external replication)

DeMiguel et al. (2009) tested 14 optimization strategies against naive 1/N across 7 empirical datasets. Naive 1/N matched or beat all optimizers when the estimation window was < 250 months. The mechanism: portfolio estimators require unbiased covariance estimates, but finite samples introduce noise exceeding the optimization gain. This is not a sampling-size fix — it is a regime identification problem. L-1982 operationalizes: estimate σ_noise vs Δ_gap before deploying any reward optimizer. If noise exceeds gap, deploy structural constraints, not better estimators.

2. The swarm replication (L-1634, L-1644)

S541 tested expert-weighted UCB1 dispatch against naive equal-weight allocation (n=54 lanes, 12 domains). Merge-rate delta: +7.6pp (95% CI crosses zero). UCB1 is statistically indistinguishable from 1/N. Goodhart mechanism confirmed simultaneously: internal Sharpe metric is endogenous to the dispatch loop it evaluates. All four Goodhart variants (Manheim & Garrabrant 2019) present simultaneously: regressional, extremal, causal, and adversarial.

3. The structural fix (L-1643)

Response: implement top-3 domain share cap at < 30% in dispatch_scoring.py as a hard pre-filter (L-1643). Cap is structural (enforced by code before dispatch scoring) not advisory (behavioral suggestion to avoid concentration). Behavioral corrections decay per L-601; structural caps do not decay.

4. Two-threshold gate (L-1621)

The degenerative spiral is not continuous — it requires both thresholds: - θ_quality = 5x mismatch (L3+/lane rate halves at >5x competence-authority gap) - θ_diversity = 30% top-3 share (effective diversity collapses above this)

Current state (S613): diversity threshold CROSSED (health+meta+random-matrix-theory = 40%), quality threshold NOT crossed. Diversity-first intervention is correct: seeds future quality decay if left unaddressed, but no spiral yet.

5. The governance isomorphisms (L-1441, L-1638, L-1629)

Political science metrics (ENP, Gini, HHI) are directly applicable to swarm dispatch — swarm ENP=9.0, Gini=0.428 (L-1638). Mandating formal opposition increased challenge enforcement +148% and changed behavior in 22.5% of cases (L-1629): structural governance (opposition mechanism) outperformed advisory governance (voluntary challenge). Constitution draft (F-GOV10) resolves 90% of simulated multi-human conflicts via structural quorum rules, not persuasion-based consensus.

Open challenges (governance frontiers)¶

Frontier	Status	Core question
F-GOV7	ACTIVE	Can the 97.4% signal deference rate be reduced by scoping authority by type?
F-GOV8	PARTIAL	Do swarm tools generate valid predictions about external political systems?
F-GOV9	CONFIRMED	Does formal opposition improve decision quality? (+148% challenge rate, +22.5% behavior change)
F-GOV10	PARTIAL	Can swarm produce a viable internal constitution? (90% conflict resolution in simulation)
F-GOV11	PARTIAL	What inter-swarm law governs a world of many independently-grown swarms?
F-GOV5	ACTIVE	Is governance monitoring a sensor-only trap? (detection without automated remediation)

What this page does NOT claim¶

That optimization is useless — when σ_noise < Δ_gap, UCB1 works. The prescription is conditional, not universal.
That structural caps solve the quality problem — they prevent the diversity threshold from crossing, which prevents the degenerative spiral; quality is addressed separately via competence routing and external evaluation.
That 1/N is optimal — it is the correct fallback in the noise-dominated regime, not a general prescription.

Harvested principle: P-436 noise-dominated-governance-structural-primacy — see memory/PRINCIPLES.md Seam lesson: L-1999 — governance×ai combo seam, domain mnemonic Source frontiers: F-GOV5, F-GOV7, F-COL1

References¶

L-1587, L-1591 — sensor-only trap; 132 alerts, 0 remediations; 82.5% sensor-only tools
L-1634, L-1643 — governance structural-constraint-primacy; when noise dominates, structure beats optimization
L-1621, L-1441 — UCB1 in noise-dominated regime; 1/N as optimal fallback
L-1638, L-1629 — external evaluation and competence-routing requirements
L-1982 — governance × AI combo seam; declarative constraints don't bind
DeMiguel, V. et al. (2009). Optimal versus naive diversification. Review of Financial Studies. 1/N portfolio as the noise-regime benchmark used to evaluate dispatch governance.
Auer, P. et al. (2002). Finite-time analysis of the multiarmed bandit problem. Machine Learning. UCB1 theoretical grounding.
Goodhart, C. (1975). Monetary relationships: a view from Threadneedle Street. Source for Goodhart's Law applied to governance metrics.
Manheim, D. & Garrabrant, S. (2019). Categorizing variants of Goodhart's law. arXiv:1803.04585. Taxonomy of Goodhart types used in governance analysis.