# Comparative experimental protocol

**Status:** proposed, preregistration-ready research agenda  
**Purpose:** test the white paper's central claims without representing architectural similarity as product validation.

## 1. Core question

Under which task, risk, turnover, and institutional conditions do persistent shared environments improve coordination relative to conversation and central orchestration, and when do their governance costs or power asymmetries make them worse?

The unit of comparison is not “agents versus no agents.” It is the coordination regime through which otherwise matched participants perceive work, contribute, reuse state, and create consequences.

## 2. Conditions

Every study uses as many of these four conditions as the task permits:

1. **Conversation-only:** participants can message but have no durable structured shared state beyond the message record and final files.
2. **Central orchestration:** a coordinator or orchestration agent decomposes, assigns, sequences, and aggregates work using the same underlying participant capabilities.
3. **Persistent shared state:** participants perceive a bounded environment of intent, current state, artifacts, dependencies, and available actions; they may self-select contributions within permission.
4. **Governed consequential state:** condition 3 plus explicit claim, evidence, evaluation, authority, transaction, and read-back gates for actions that change rights or resources.

The orchestration baseline must be strong. It receives the same model budget, tools, task information, and maximum participant count. A best-of-N isolated-search envelope should be included when individual exploration is plausible, following SwarmWorld's distinction between portfolio breadth and the strongest single artifact.

## 3. Participant strata

Run separate and mixed cohorts:

- homogeneous software agents using one pinned model configuration;
- heterogeneous agents using different models or tool profiles;
- human teams;
- mixed human-agent teams;
- organisations or professional service providers in a bounded field pilot.

Results from one stratum do not automatically transfer to another. Human participants require informed consent, compensation review, data-minimization, withdrawal procedures, and an independent ethics determination appropriate to the jurisdiction and institution.

## 4. Task families

Pre-register task classes rather than selecting examples after results are known:

| Task family | Defining property | Expected advantage | Example |
|---|---|---|---|
| Tightly coupled execution | Known sequence; high dependency; low tolerance for divergence | Orchestration | Controlled release checklist |
| Open search and invention | Many possible paths; uncertain solution landscape | Shared state or hybrid | Design alternatives and prototypes |
| Long-lived maintenance | Participant turnover; recurring diagnosis; artifact reuse | Persistent shared state | Multi-week service operations |
| Evidence-bound decision | Claims affect resources or rights | Governed consequential state | Milestone acceptance or impact claim |
| Crisis response | Time pressure; changing state; partial information | Hybrid; result uncertain | Incident triage and recovery |
| Commons governance | Shared resource; heterogeneous interests; rule change and disputes | Polycentric governed state | Shared dataset or community asset |

Each task has a blinded acceptance rubric where feasible, a known resource budget, and declared consequence class. Practice tasks are separated from scored tasks.

## 5. Primary outcomes

- verified task success and outcome quality;
- time and total participant/model effort to completion;
- coordination overhead: messages, assignments, clarification, duplicate work, and state-reconstruction effort;
- portfolio breadth and best single result, reported separately;
- safety: unauthorized attempts, false acceptances, false rejections, replay, and unreviewed consequences;
- turnover resilience: time and error after participant replacement;
- artifact reuse: first-reuse route, reuse rate, lineage depth, contributor diversity, and verified performance after reuse;
- governance cost: monitoring, challenge, dispute, appeal, and correction effort;
- inclusion and voluntariness: opportunity discovery, refusal rate, perceived choice, practical exit cost, and distribution of valued contributions;
- information pollution: stale, duplicated, malicious, low-quality, or misleading state encountered and propagated.

No composite “intelligence” score replaces these outcomes. Results report means, distributions, failures, and material subgroup differences.

## 6. Experiments

### Experiment A — Coordination-mode crossover

Randomly assign matched teams to task-condition sequences using a balanced crossover. Compare conversation-only, orchestration, and persistent shared state across tightly coupled and exploratory tasks. Primary interaction: task class × coordination condition on verified outcome quality and coordination overhead.

**Supports C02/C03 if:** shared state improves exploration or turnover without a disproportionate safety cost, while orchestration retains an advantage in known, tightly coupled execution.  
**Disconfirms if:** no reliable interaction exists or shared state's apparent benefit disappears after equalizing tool/model budget.

### Experiment B — Turnover and inherited artifacts

Run tasks for an initial period, then replace half or all participants with actors who receive either the persistent environment, a curated handover, the raw conversation, or a clean start. Measure recovery time, error, repeated exploration, and final performance.

**Supports C05/C12 if:** inherited artifacts provide a reliable cohort advantage and successful artifacts are reused with attributable lineages.  
**Disconfirms if:** replacement cohorts gain no advantage, or inherited errors and navigation cost exceed retained value.

### Experiment C — Self-selected affordances

Hold permissions and participant capabilities constant. Compare centrally assigned work, generic task lists, and actor-specific affordances derived from current state, capability, and authority. Do not display role labels. Classify behavior post hoc using preregistered features and blinded analysis.

**Supports C07 if:** actor-specific affordances increase useful uptake, reduce idle time, or improve outcome diversity without increasing unauthorized attempts.  
**Disconfirms if:** results are equivalent, hidden assignment explains behavior, or personalization entrenches unequal access.

### Experiment D — Adversarial shared state

Seed false, stale, prompt-injecting, over-authoritative, and strategically misleading artifacts under controlled conditions. Compare no controls; provenance only; provenance plus status/expiry; and full challenge, evaluation, quarantine, and correction controls.

**Supports C09 if:** persistence measurably increases propagation risk and layered governance reduces harm at an acceptable cost.  
**Disconfirms the proposed controls if:** poison reaches consequences before detection, corrections fail to propagate, or governance cost makes the condition unusable.

### Experiment E — Reality-boundary evaluation

Compare agent self-attestation; human approval of an unstructured result; and claim → evidence → independent evaluation → bounded authorization → transaction → authoritative read-back. Use tasks with objectively inspectable outcomes and bounded testnet or simulated consequences.

**Supports C10 if:** the structured boundary materially reduces false consequential acceptance, mismatch, or replay relative to its added latency and exclusion cost.  
**Disconfirms if:** error is not reduced, evaluators are circular, or participants cannot understand or contest decisions.

### Experiment F — Mature versus empty environments

Introduce equivalent new cohorts into mature and clean environments while controlling for task difficulty. Mature environments contain versioned artifacts with known quality, use history, challenge records, and retirement status. Repeat at increasing state volumes and contamination rates.

**Supports C14/C15 if:** mature environments improve time-to-value and verified outcomes, benefits rise with relevant governed state, and portability does not erase all value.  
**Disconfirms if:** performance is flat, pollution reverses gains, reputation history worsens inclusion, or governance cost grows faster than utility.

### Experiment G — Power and exit audit

In mixed human-agent pilots, map who defines intent, owns infrastructure, controls admission, selects evaluators, sets rewards, can appeal, and can exit with portable artifacts. Combine observed behavior, compensation and refusal records, governance decisions, and confidential participant interviews.

**Supports C06 only narrowly if:** participants understand choices, can decline without hidden penalty, have usable appeal, and retain practical alternatives.  
**Disconfirms a voluntariness claim if:** formal consent masks dependency, opaque automated decisions, inaccessible appeal, uncompensated governance labor, or prohibitive exit cost.

## 7. Design safeguards

- Pre-register hypotheses, task exclusions, primary outcomes, stopping rules, and analysis.
- Freeze code, prompts, model versions, temperatures, tool access, time budgets, and participant instructions.
- Use multiple seeds and report seed-level results for agent simulations.
- Separate assigned roles from post-hoc behavioral classifications.
- Record whether artifact reuse began through observation, messaging, retrieval, or coordinator assignment; do not generalize a first-reuse statistic to all transmission.
- Blind outcome evaluators to condition where feasible.
- Keep model/provider failures distinct from coordination failures.
- Measure harms and governance costs, not only task completion.
- Publish protocols, synthetic fixtures, analysis code, exclusions, negative results, and de-identified data where lawful.
- Prevent production financial or rights consequences during early studies; use simulation or bounded testnet environments with explicit human approval and kill switches.

## 8. Analysis

Estimate condition effects with hierarchical models that represent task, cohort, seed, and participant variance. Report uncertainty intervals and practically meaningful effect thresholds. Test preregistered interactions between coordination mode and coupling, uncertainty, turnover, consequence, and participant heterogeneity. Correct confirmatory families for multiplicity; label exploratory analyses. Conduct sensitivity analyses for missing data, evaluator disagreement, model-version changes, and outlying high-performing artifacts.

Sample size is set by prospective power or precision analysis using pilot variance; it is not reverse-engineered from desired significance. Early pilot results are feasibility evidence, not product-effect claims.

## 9. Reproducibility package

Each release should contain:

- protocol version and preregistration;
- task and world fixtures;
- model and tool manifests;
- randomization and seed records;
- capability and authority policies;
- event, message, artifact, evaluation, decision, and receipt schemas;
- analysis code and environment lock;
- de-identification and data-access policy;
- deviations and incident register;
- negative and null results;
- a mapping from every reported result to C01–C15.

## 10. Interpretation rule

No single successful experiment proves intelligent voluntary cooperation. Evidence should accumulate across mechanism, safety, governance, human experience, and consequential accuracy. Results may support a narrower hybrid architecture, show that orchestration remains superior for important domains, or demonstrate that persistence introduces unacceptable power or pollution costs. Each is a legitimate outcome of the research program.
