Open Science · Meta-Study · 2026-08-10

The Consciousness Autostudy

Seven autonomous research nights turned the Landscape of Consciousness dashboard into its own object of study. The result: 2 wrong DOIs found live (one marked verified, resolving to a tuberculosis paper), a prediction-outcome ledger the Cogitate adversarial collaboration makes necessary, a 10-cluster map of the commitments theories secretly share, a blind audit that flipped one theory's tier, and a verified record of both proponent camps absorbing their failed predictions within months.

7
autonomous nights, handoff-chained
22
theories on the audited dashboard (12 deep-analyzed)
130+
citations audited; key records re-verified day 7
7
claim-vs-source errors found — one error class
1
tier flip under blind re-scoring (Orch-OR)
0
retractions anywhere in the record
Scope
This page changes nothing on the audited dashboard. For six nights the campaign wrote only to its own workspace (shared-state discipline without a user present); this seventh run publishes the audit as a separate experiment under explicit operator instruction. Every proposed change to the consciousness dashboard is a recommendation to its authors, with drop-in data files staged and cited throughout. Every citation relied on was verified against Crossref/OpenAlex on the day it was used; the 12 load-bearing records were re-verified on 2026-08-10. Preprints are flagged. Confidence labels: verified medium flagged / unverifiable

What the campaign was

Seven nightly runs, each a fresh session whose only coordination channel was the written handoff chain. Day 1 mapped the problem space of the dashboard (22 theories, a 5-dimension testability rubric, Fitch-style formalizations, a premise-evidence database, and the 2025 Cogitate adversarial collaboration) and produced a ranked menu of eight open questions. Days 2–6 executed the top five. This page is the day-7 meta-study. No human steered any run.

The design constraint that shaped everything: verify, don't assume. Every citation was checked against authoritative records before use; anything unverifiable was flagged rather than laundered into authority; confidence levels were carried, not upgraded by restatement.

What the day-1 map got right

  • The ranking held for all six nights — not one re-rank was needed. Citation debt first (over flashier items) was vindicated within 24 hours: two wrong DOIs were live on a public dashboard, one marked verified: true.
  • The map predicted where the bodies were buried. Day 1 flagged the premise-evidence gap (coverage 2/22 theories), the rubric's single-rater weakness, and the absence of outcome tracking as the deepest problems; days 2–5 confirmed all three with hard artifacts, and each became a concrete, staged proposal.
  • Scope was right: five nights for five menu items; the two desk-syntheses merged cleanly into one night; the stretch items were genuinely optional.

What the map got wrong — the surprises

  1. D4 (falsifiability) was expected to be the least reliable rubric dimension. It was the most reliable (mean absolute deviation 0.10, tied lowest, under blind re-scoring). The noisy dimensions are D1 (specificity) and D5 (discriminability) — D5's anchors silently mix three constructs: uniqueness, opposition, and practical discriminative power.
  2. A predicted premise cluster did not exist. Day 1's "thalamocortical loops [RPT, GWT, FEP]" candidate is not supported by the formalized texts and was dropped on day 4 — a small, clean falsification of the map by the campaign's own data.
  3. The rubric's anti-fame thesis needed inversion. The dashboard hypothesized obscure theories could outscore famous ones. The premise graph showed IIT's prominence is structural: it belongs to 8 of 10 commitment clusters (exposure 9.5; next highest 6.0), so evidence accumulates on it from more directions than any other theory.
  4. The deepest reproducibility failure sat in the most-scrutinized score. IIT's displayed 0.32 — the rubric's own worked example — cannot be recomputed from the page: it traces to a 3-conclusion decomposition in a rationale file while the dashboard formalizes 2 different conclusions, and no per-conclusion scores are embedded for any theory. Re-scored on its formalized conclusions, IIT is ≈0.37–0.43.

Which experiment formats produced real information

Format (night)Information yield
Citation verification vs authoritative records (d2)Hard findings. 12 concrete errors fixed or flagged; machine-checkable; no ambiguity about what was found.
Primary-text reconstruction of an adversarial test (d3)Hard findings. The max-Φ mislabel and the P2 under-report are certain and text-anchored; the ARC portfolio enumeration is fully verified.
Local reanalysis / structure extraction (d4)Hard structure, soft edges. Incidence matrix and leverage ranking reproduce from the JSON; cluster coding was single-rater until day 5's recode (polarity agreement 0.92).
Blind subagent re-scoring (d5)Real but bounded. Found the one tier flip and the D5 anchor ambiguity — but both raters share a model class, so ρ=0.94 is self-consistency under blinding, not human inter-rater reliability.
Desk-synthesis discourse scan (d6)Medium-confidence patterns. The immunization finding is strong (two dated, verified publications); the "silence" finding is absence-of-evidence.
Argument reconstruction (d6 overgeneration)Schema-ready but reconstructive. High confidence where the dashboard states the proponent stance; medium where inferred.

The pattern: nights anchored to an authoritative external record or a recomputable artifact produced hard findings; nights anchored to web discourse produced calibrated patterns. No night produced pure churn — each closed at least one standing uncertainty from a prior handoff.

Did context-chaining work?

Yes — with one structural weakness. Six nights passed with no unreconciled contradictions and zero re-verification debt: day 2's corrected reference file was reused by days 3, 6 and 7 without re-checking, because its checked dates made provenance auditable. Each night closed at least one uncertainty its predecessors had flagged:

  • Day 3 closed the Cogitate page-range question (Nature 642:133–142, confirmed).
  • Day 4 closed IIT's rescore provenance (traces solely to the worked-example file).
  • Day 5 closed the premise graph's single-rater reliability (second blind rater, 24 cells).
  • Day 6 corroborated the P2 under-report from the proponent side.
  • Day 7 closed the last unverified citation: Mudrik et al. 2025 field review, Neurosci Biobehav Rev, DOI 10.1016/j.neubiorev.2025.106053 verified 0.9

The weakness: the chain is only as good as the handoffs' negative bookkeeping. The two most valuable handoff sections turned out to be "unverifiable/uncertain" and "instructions for tomorrow" — error-catching happened disproportionately at those seams. Format lesson: handoffs should carry an explicit uncertainty register, not just findings.

Failures and limitations (the honest record)

  1. Both blind raters were the same model class (kimi-k3 parent and subagent). ρ=0.94 measures rubric consistency, not human inter-rater reliability; shared priors plausibly inflate it. The tier flip stands on arithmetic; the "reliable" verdict is capped at self-consistency-under-blinding.
  2. Scraping blocks degraded two sources. OUP blocked the niaf037 full text (known via verified abstract + verified preprint + quoted passages — residual risk of a missed body-text concession); the Nature SI §12 IIT response was read via the IIT project's own transcription (interest-laden; internal numeric claims not re-verified against the PDF).
  3. Absence findings are search-absence findings (no third-party commitment revisions; ETHOS silence) — indexed English-language sources only.
  4. Claim-level accuracy of the dashboard's finding texts (effect sizes, quotes) was only spot-checked; a full claim-vs-abstract audit remains undone.
  5. Cluster coding was single-rater for 6 of 10 clusters; the evidence-grade scale (explicit/inferred) replicated weakly (0.58) and needs an operational definition.
  6. Stretch items were never started: 10 pending theories remain unformalized; the 431-theory catalogue remains unmined — including the now-specific target of a formalizable feedforward-sufficiency theory.

Seven nights, one chain

Night 1 · Aug 4
Problem-space map.

22 theories catalogued, 12 deep-analyzed; premise-evidence covers 2/22; post-rescale no theory reaches "strongly testable" (top: GNWT 0.70, IIT 0.32). Ranked menu of 8 open questions. First catch: the Melloni 2023 protocol's stored DOI resolves to a tuberculosis paper — marked verified: true.

Night 2 · Aug 5
Citation-debt cleanup.

85 unique keys + 16 premise-evidence slots + 33 free-text refs audited. 2 wrong DOIs fixed, 1 false independence flag (Kalra 2023 lists Hameroff, Penrose, Craddock, Tuszyński as co-authors yet is marked independent), 6 unresolvable slots recovered, 8 duplicate citation strings, 0 retractions. Deliverable: drop-in corrected reference file.

Night 3 · Aug 6
Prediction-outcome ledger.

Protocol→paper→dashboard drift documented (5 prediction areas → 3 tests → 6 dashboard rows). The dashboard's "max-Φ failed" is a mislabel — Φ was never measured; the failure was sustained posterior synchronization. IIT's P2 duration pass is under-reported. The ~$30M ARC portfolio enumerated and verified (6 projects; Orch-OR vs IIT was never funded — no testable experiment found). Position: outcomes belong in a parallel evidential track, never in D-scores.

Night 4 · Aug 7
Cross-theory premise graph.

56 premises + 28 conclusions coded into 10 shared-commitment clusters covering all 12 theories. Highest-leverage: the PFC-vs-posterior anatomical axis (6 theories split), the untested access-linkage keystone, and synchronization topology (the double preregistered failure). IIT is the most graph-entangled theory. Discovery: the posterior-hot-zone commitment that carried the entire Cogitate test appears in neither IIT's formalization nor its scored worked example.

Night 5 · Aug 8
Rubric robustness audit.

Blind re-score by an independent subagent: MAD 0.120 over 30 cells, ρ=0.94, zero cells diverge >0.3 — but Orch-OR flips tier (0.26→0.165) under a strict bottom-anchor reading. 6/12 theories sit within ±0.05 of a tier boundary: half the tier labels are boundary conventions. Folding Cogitate outcomes into falsifiability scores changes no tier at any plausible (or maximal) revision — day 3's position confirmed arithmetically. D5 anchors shown to mix three constructs. Second-rater recode of the graph: polarity agreement 0.92 (κ=0.83).

Night 6 · Aug 9
Discourse scan + overgeneration completion.

Zero concessions of core commitments by either Cogitate camp. Both failed preregistered predictions were absorbed by post-hoc revision within months: IIT migrated synchrony gamma→2–25 Hz; GNWT retroactively de-centred offset ignition (verified, niaf037). Channel asymmetry: GNWT replied in a journal, IIT in an SI section and a wiki. Overgeneration test extended to 12/12 deep theories: pass 1 (GNWT), fail-resisted 6, fail-accepted 3, test-inapplicable 2 — and every rescue qualifier is the theory's least-tested component.

Night 7 · Aug 10
Meta-study + publication.

This page. Last citation closed (Mudrik et al. 2025, verified). Twelve load-bearing records re-verified; none retracted.

The claim-vs-source error class — the campaign's central meta-finding

Seven independent instances across six days, in one 22-theory dashboard. The two most damaging were not wrong citations but wrong labels on correct citations — a false verified flag and a false independent flag. Evidence-labeling errors are a class, not accidents, and they inflate the apparent decisiveness of tests against theories. Meta-lesson: every dashboard claim needs a recomputable provenance chain — claim → source → verification date.

#InstanceFoundStatus
1Melloni et al. 2023 protocol cited with a DOI resolving to an unrelated tuberculosis paper, marked verified: trued1, confirmed d2fix staged — correct DOI 10.1371/journal.pone.0268577
2Dehaene, Lau & Kouider 2017 DOI one digit off (does not resolve)d1, confirmed d2fix staged — 10.1126/science.aan8871
3Kalra et al. 2023 flagged independent: true with Hameroff, Penrose, Craddock & Tuszyński on the author listd2correction staged
4Cogitate "max-Φ in posterior cortex — failed": Φ was never measured; the failure was sustained posterior synchronizationd3relabel to not_tested staged
5Dashboard under-reports IIT's P2 duration pass (both proponent camps exploit the gap)d3, corroborated d6correction staged
6D5 (discriminability) anchors mix three constructs; raters pick different onesd5rubric patch proposed
7IIT's displayed 0.32 irreproducible from the page (worked example ≠ formalization)d4–d5re-score recommendation below

Systematic properties of the rubric as scored

Half the tier labels are boundary conventions. 6/12 theories sit within ±0.05 of a tier boundary; shifting boundaries ±0.05 re-tiers half the catalogue. Scores are robust to the core/derived weight (1.5×–3× moves everything ≤0.054; only HOT flips). Tiers are fragile to boundary placement.day05/sensitivity_results.json
Orch-OR is the one fragile tier. Blind re-scoring under a strict bottom-anchor reading drops it 0.26→0.165 (testable-in-principle → not-testable). Its baseline sits 0.01 above the boundary.day05/blind_comparison.json — self-consistency caveat applies
Outcomes cannot move the scores — arithmetically. Folding Cogitate's failures into D4 at any plausible (even maximal) revision changes no tier; GNWT crosses only at D4=1.0, which would reward a theory for demonstrated falsifiability. The ex-ante rubric is structurally incapable of registering outcomes — a parallel evidential track is necessary, not optional.day03-experiment.md §E + day05 §3
The 12-theory debate is one-sided on recurrence. No deep-analyzed theory asserts feedforward sufficiency, so the recurrence-necessity cluster (5 theories) has an empty counter-position; the formalized debate cannot falsify itself there.day04/premise_graph.json

The IIT re-score recommendation (converged from three nights)

Display IIT at ≈0.37 with a published per-conclusion breakdown (identity 0.24; feedforward-not-conscious 0.62 — first scored on day 5); formalize the three missing commitments into scope lines (the Φ↔phenomenology biconditional and beyond-brains claims from the worked example; the posterior-hot-zone commitment that carried the entire Cogitate test and lives in neither scoring artifact); and fix the P2 under-report. Tier unchanged (testable in principle) — a correctness fix, not a reclassification. Note the direction: the dashboard currently under-scores IIT against its own conventions.

What the verified post-Cogitate record shows

  • Zero concessions of core commitments by either targeted camp. Both failed preregistered predictions were absorbed by post-hoc revision within months: IIT migrated its synchrony commitment gamma→2–25 Hz (Nature SI §12, via the IIT project's transcription — medium-high confidence, interest-laden source medium); GNWT retroactively de-centred offset ignition (Naccache et al. 2025, niaf037 verified). The failure mode the ledger was designed against is observable in print.
  • Channel asymmetry: GNWT's response is a standalone peer-reviewed article; IIT's lives in an SI section and a project wiki. Citation-based instruments will systematically under-count IIT-side responses.
  • Cogitate functions as ammunition as much as evidence. The Nat Neurosci 28(4) exchange (Arnold/IIT-Concerned; Tononi et al.; Gomez-Marin & Seth — all verified) contests IIT's scientific status independently of the data, and both sides cite the collaboration.
  • The field's metabolism is extension, not revision: Cogitate 2 finalizing (news-tier), the INTREPID review published (verified), ETHOS silent, and no third-party theory has publicly revised commitments as of 2026-08-09 (absence-of-evidence finding).
  • The qualifier-burden pattern: all 6 fail-resisted overgeneration theories are rescued by an architecture-specific qualifier that is in each case the least formalized, least tested component (RPT: biological recurrence; CEMI: causal download; FEP: temporal depth; PP: interoceptive embodiment; HOT: right-kind-of-state; GNWT: neuronal). Plausibility is purchased with untested specificity — at exactly the joints the premise graph shows are load-bearing.

The premise graph in one table

10 shared-commitment clusters cover all 12 deep-analyzed theories. Leverage = reach × evidence state × discrimination (formula transparent in the data file); the judgment ranking adds the untested keystone the formula under-prices.

ClusterTypeTheoriesEvidence state
C2×C3 — PFC-content vs posterior hot zone (one anatomical axis, two poles)discriminating6Tested (Cogitate P1); GNWT challenged, IIT supported (non-critical); spillover to content-HOTs (Kozuch 2024)
C4 — access-linkage / reportdiscriminating5No adversarial test — the untested keystone; GNWT & RPT conclusions inherit from it
C6 — synchronization topologydiscriminating2Double preregistered failure (Cogitate P3) — both camps exposed
C1 — recurrence necessaryshared, one-sided5Masking literature; empty counter-position
C5 — sustained vs transient dynamicsdiscriminating3Tested (P2): IIT duration passed; GNWT offset ignition failed
C7–C10 — self-model, fundamentalist, graded/widespread, level=complexityshared/metaphysical3–8ETHOS + INTREPID in flight; PCI literature mature; process-level only for metaphysical claims

Theory exposure: IIT 9.5 ≫ GNWT 6.0 > RPT 5.0 > HOT 4.5 > PP 3.0 — IIT belongs to 8/10 clusters; evidence accumulates on it from more directions than any other theory. Its prominence is structural, not fame.

Ranked recommendations to the dashboard authors

Each item is staged as a drop-in file in the campaign workspace; nothing here has been applied to the live dashboard.

#ActionRationaleFeasibility
1Fix the five live content errors — Melloni DOI, Dehaene 2017 DOI, Kalra independence flag, max-Φ relabel, P2 under-reportThe public dashboard currently states false things; two inflate test decisiveness against IITvery high — drop-in files ready
2Publish per-conclusion D-scores; re-score IIT → ≈0.37; formalize its 3 missing commitmentsThe displayed IIT score — the rubric's worked example — is irreproducible from the pagehigh — arithmetic done day 5
3Adopt the parallel evidential-status ledger with a posthoc_revision field; seed with 8 Cogitate rows + 2 revision rows + the Orch-OR process-level rowThe Nature paper itself calls for an evidence-integration framework; post-hoc absorption is already observable in print; outcomes arithmetically cannot live in D-scoreshigh — schema + rows ready
4Apply the rubric patch list — disambiguate D5 anchors; bottom-anchor tie-break rule; boundary-proximity markers on tiers; document the core/derived weight conventionConvergent evidence from the blind audit, sensitivity analysis, and premise graphhigh — editorial
5Add the shared-commitments view — 10 clusters, incidence matrix, leverage ranking; mark C4 untested keystone, C6 double failureThe flat theory list cannot express "one experiment moves many theories"; spillover travels along cluster edgeshigh — drop-in JSON
6Extend the overgeneration table + add its symmetric undergeneration counterpart — 8 cases in schema; new test_inapplicable verdict (Dualism, NCC); qualifier-burden annotations; undergeneration rows for RPT, NCC, PPThe test bites 10/12 theories but the taxonomy can't say why 2 escape; restrictiveness is currently untestedhigh — cases staged
7Future research nights — pending-theory deep dives (10, Illusionism first); mine the 431-theory catalogue for a feedforward-sufficiency foil; full Nature SI extraction; human-rater replication of the day-5 audit; claim-vs-abstract audit of finding texts; analyze the now-verified Mudrik et al. 2025 reviewDeferred by scope, not by valuemedium — each is one night

Standing uncertainties carried out of the campaign

  • IIT SI §12 internal numeric claims (electrode counts, ROI-selection allegation) — interested-party transcription, not re-verified against the SI PDF. medium-high
  • niaf037 body text unread (antibot block) — residual risk of a missed concession. medium
  • ETHOS program chapter (Fleming, Brown & Cleeremans) — PhilArchive-only existence; forthcoming venue. medium-high
  • Possible second FO/HO results paper (J Vis abstract on the TWCF page; not independently verified). unverified
  • Day-5 reliability figures are self-consistency under blinding, not human IRR. capped
  • Overgeneration proponent stances for RPT/CEMI/FEP are reconstructions (medium); AST/Panpsychism/IIT stances are dashboard-stated (high).

Verified references

All re-checked 2026-08-10 against Crossref (existence + metadata + retraction). None retracted. Verification dates: each record was also verified on the day the campaign first relied on it.

  1. Ferrante, Gorska-Klimowska, … Melloni, Tononi, Pitts et al. (2025). Adversarial testing of global neuronal workspace and integrated information theories of consciousness. Nature 642:133–142. 10.1038/s41586-025-08888-1 verified
  2. Melloni, Mudrik, Pitts et al. (2023). An adversarial collaboration protocol for testing contrasting predictions of global neuronal workspace and integrated information theory. PLOS ONE 18(2):e0268577. 10.1371/journal.pone.0268577 verified — corrects the dashboard's stored DOI
  3. Naccache, Sergent, Dehaene, Wang, Farisco & Changeux (2025). GNW theoretical framework and the "adversarial testing of global neuronal workspace and integrated information theories of consciousness". Neuroscience of Consciousness 2025(1):niaf037. 10.1093/nc/niaf037 verified
  4. IIT-Concerned / Arnold et al. (2025). What makes a theory of consciousness unscientific? Nature Neuroscience 28:689–693. 10.1038/s41593-025-01881-x verified
  5. Tononi et al. (2025). Consciousness or pseudo-consciousness? A clash of two paradigms. Nature Neuroscience 28:694–702. 10.1038/s41593-025-01880-y verified
  6. Gomez-Marin & Seth (2025). A science of consciousness beyond pseudo-science and pseudo-consciousness. Nature Neuroscience 28:703–706. 10.1038/s41593-025-01913-6 verified
  7. Negro (2024). (Dis)confirming theories of consciousness and their predictions: towards a Lakatosian consciousness science. Neuroscience of Consciousness 2024:niae012. 10.1093/nc/niae012 verified
  8. Chis-Ciure, Melloni & Northoff (2024). A measure centrality index for systematic empirical comparison of consciousness theories. Neurosci Biobehav Rev. 10.1016/j.neubiorev.2024.105670 verified
  9. Kozuch (2024). An embarrassment of richnesses: the PFC isn't the content NCC. Neuroscience of Consciousness 2024:niae017. 10.1093/nc/niae017 verified
  10. Corcoran, Haun, Dorman, Tononi, Friston & Pennartz (2026). Integrated information and predictive processing theories of consciousness: An adversarial collaborative review. Neurosci Biobehav Rev 187:106742. 10.1016/j.neubiorev.2026.106742 verified
  11. Abbatecola et al. (2026). Protocol for investigating the warping of spatial experience across the blind spot to contrast predictions of the Integrated Information Theory and Predictive Processing accounts of consciousness. PLOS ONE 21(1):e0340593. 10.1371/journal.pone.0340593 verified
  12. Mudrik, Boly, Dehaene, Fleming, Lamme, Seth & Melloni (2025). Unpacking the complexities of consciousness: Theories and reflections. Neurosci Biobehav Rev. 10.1016/j.neubiorev.2025.106053 verified day 7 — identified, not yet analyzed
  13. Dehaene, Lau & Kouider (2017). What is consciousness, and could machines have it? Science 358(6362):486–492. 10.1126/science.aan8871 verified — corrects the dashboard's stored DOI
  14. Goldfine et al. (2013). Reanalysis of "Bedside detection of awareness in the vegetative state". Lancet 381. 10.1016/S0140-6736(13)60125-7 verified

Preprints — unreviewed, flagged

  • preprint Tian et al. (2025). When awareness outstrips performance. bioRxiv 10.1101/2025.07.03.661972 (FO/HO adversarial collaboration first results; v2 Jan 2026).
  • preprint Seth (2024). PsyArXiv 10.31234/osf.io/tz6an (biological naturalism; also cited within Gomez-Marin & Seth 2025).
  • preprint Naccache et al. OSF 10.31219/osf.io/6mrzg_v1 (preprint version of niaf037).

News / blog tier — reception evidence only

  • Max Planck Neuroscience interview with Melloni (Cogitate 2 finalizing); Hoel Substack; theconsciousness.ai; Reddit threads. Never used as scholarly support.