Most brain-imaging studies scan 15–30 people — too few to trust on their own. The tempting fix is to pool many small studies into one shared space. We tested that fix on two public datasets where people heard or watched the same stimulus — and got opposite answers. Together they map the one thing anyone pooling brain data needs to know: when it helps, and when it quietly makes things worse.
On a wordless visual tone-poem (Koyaanisqatsi, 93 people), alignment hurt: −0.026 accuracy, p < 0.001. Not a bug — three controls checked.
On a story-driven stimulus (Forrest Gump, heard as a 7T audio description — 19 people × 8 runs), alignment helped: +0.011, p < 0.001, 15 of 19 subjects improved.
Pooling earns its keep where a story drives each brain's idiosyncratic wiring — and costs you signal where it doesn't. Measured, not assumed.
Here's an open secret in brain imaging: most fMRI studies scan 15–30 people, and that's too few to trust. Button et al. (2013) showed that at these sample sizes, individual studies are underpowered and their headline effects unstable. Small labs each running small studies is the norm — so a natural question follows: what if we could merge them?
There's a catch. Brains differ. The patch of cortex that processes a face in your brain isn't in exactly the same spot as in mine — so naively stacking brains "voxel by voxel" (anatomical alignment) blurs the signal. The proposed fix is functional alignment: use the data itself to learn a rotation that brings each person's brain into a shared information space, then pool. The machinery — hyperalignment (Haxby et al., 2020) and the Shared Response Model (Chen et al., 2015) — is proven to work when people watch the same story: different brains follow the same plot along the same high-dimensional trajectory once rotated into register. Nastase et al. (2020) even showed heterogeneous datasets can be aggregated into one response space.
So the pitch writes itself: gather every small open movie-watching dataset, align, pool, and answer questions no 20-person study could. What the pitch skips is a boundary question: does alignment always help, or only sometimes? We built the pipeline, ran it where the literature said the answer should be easy — and tried hard to break our own result before believing it.
The idea, in one sentence: take people who all experienced the same stimulus, line their brains up two different ways, and ask which way lets us recover more of what they were experiencing. Everything is deterministic and public — the claim is about linear algebra, not model capacity. The full recipe lives in the panel below; the short version is that only one step changes between conditions, so any difference in accuracy is caused by the alignment and nothing else.
1. Z-score every voxel's timecourse per subject.
2. Split the stimulus into a train segment (first ⅔) and a held-out
test segment (last ⅓).
3. Compute a PCA shared basis (k = 100) from the other
subjects' mean train response — never the subject being scored (strict leave-one-subject-out).
4. Fit a per-subject orthogonal Procrustes rotation on train only.
5. Score on the held-out test segment with time-segment matching: given one
person's brain response, can we find the right moment of the stimulus in the group mean?
The only thing that changes between conditions is step 4: anatomical-only (everyone in a common anatomical space, no rotation) versus functionally aligned (rotation added). If alignment helps, matching accuracy should rise. Statistics: per-subject paired sign test plus a permutation test on the mean difference. Everything runs on public data with fixed seeds — the numbers below are exactly reproducible.
We ran this identical pipeline on two public OpenNeuro datasets — one where people watched a film without a story, one where people listened to a story without pictures. They differ in the way the hypothesis cares about — narrative vs. no narrative — and, unavoidably, in several other ways as well:
| Dataset | Stimulus | N used | Space | Narrative? |
|---|---|---|---|---|
| AOMIC-ID1000 ds003097 |
Koyaanisqatsi visual film · 3T |
93 of ~927 available — download subsample, see Reproduce |
fsaverage5 cortical surface | No — a visual tone-poem: no story, no dialogue, no characters |
| StudyForrest ds000113 |
Forrest Gump audio description auditory only · no visual · 7T |
19 people × 8 runs each = 152 recordings |
grpbold7Tad group template (volumetric) | Yes — a driven narrative with dialogue and plot, heard as a spoken German audio description of the film rather than watched |
One thing to hold onto before the results: nobody watched Forrest Gump here. People listened to a spoken audio description of it. So the two datasets differ in more than story-vs-no-story — and being honest about that changes how the results should be read.
The grpbold7Tad acquisition in ds000113 is the 7T audio-description phase of StudyForrest: participants listened to a spoken German audio description of Forrest Gump — Hanke et al. (2014) call it "an audio movie." There was no visual stimulus. An earlier version of this page called it a film throughout; our own citation said otherwise, and we misread it. It is a narrative, but not movie-watching — the two datasets differ in modality (visual vs. auditory), field strength (3T vs. 7T), reference space (surface vs. volumetric template), and subject pool, as well as narrative. That confound is stated in full in §07.
Cross-space comparability: the datasets live in different reference spaces because that is the form in which each is publicly distributed. The question — does functional alignment beat anatomical alignment? — is asked within each dataset against its own anatomical baseline, so the cross-space difference does not confound the comparison. Absolute accuracies are not comparable across datasets (different cortex coverage, TR counts, test-segment lengths); the within-dataset deltas are the unit of inference throughout.
On the wordless tone-poem, with 93 subjects and a leakage-free design, functional alignment did not help. It hurt — consistently, and well past any noise threshold.
Both accuracies sit far above the empirical chance level (~0.01, measured by circular time-shift on the identical pilot design), so there is real shared movie signal to find — this is not a floor effect. But the per-subject rotation reliably removed accuracy rather than adding it, and the result is stable across the basis dimension (k = 10, 20, 50, 100), so it's not a tuning artifact.
A negative result is only worth reporting if you've tried to kill it. Before believing this one we found and fixed a genuine measurement bug, then re-ran everything clean. The sign held. Three further checks are below.
Our first pilot computed the shared basis from a group mean that included the subject being scored — which would trivially cap any alignment benefit. The fix is strict leave-one-subject-out: basis and group mean exclude the scored subject. The leakage-free version shows the same negative (0.061 / 0.035). The negative is not leakage.
We built synthetic data where 40 subjects share a response but each is rotated by a known, recoverable transform — a world where alignment must win. It does, decisively: anatomical 0.024 / aligned 0.275, Δ = +0.25, 40/40 subjects. The machinery detects alignment benefit when it's present — so the real-data negative is a property of the data, not a broken method.
The negative replicates at k = 10, 20, 50 on a 40-subject subset (Δ between −0.006 and −0.017; sweep values recorded in the lab notebook). The effect doesn't flip at any tested dimensionality, so it's not a shared-space bottleneck.
On a non-narrative film, anatomical alignment already captures the shared response and the extra rotation only injects noise. That matches a specific prediction from the hyperalignment literature: alignment earns its keep on the idiosyncratic fine-grained topography that a story drives — and Koyaanisqatsi has no story. But that's an interpretation, not a result. The result needs a narrative stimulus, at full power.
This is the discriminating test. If functional alignment helps on a narrative stimulus, the Koyaanisqatsi negative is a boundary condition on the stimulus — and the two results together map where pooling works. An 8-subject, single-run pilot couldn't decide it (near-floor baseline, Δ = +0.002, p = 0.86). So we ran the whole thing: 19 subjects, all 8 runs of the audio description, each run scored separately and pooled per subject.
The sign flipped. On the narrative stimulus, functional alignment gives a small but reliable improvement: every run contributed, the effect isn't carried by one outlier subject or one lucky run (19 × 8 subject-by-run delta matrix, pooled per subject), and the permutation p-value is below 0.001.
Same code path, same seeds, same z-scoring and split rule as the Koyaanisqatsi analysis — no refitting, no silent parameter changes. The run-01 pilot's null is what an underpowered slice of this effect looks like; the full dataset resolves it. The earlier version of this page reported the pilot as "underpowered, decision pending" — this section is the decision.
Δ = +0.011 on a ~0.04 baseline is a ~27% relative gain in time-segment matching accuracy — modest in absolute terms, but this metric is a stringent lower bound on alignment benefit: it demands the exact timepoint be the top match out of ~150 candidates in a 100-dimensional space. The sign test (15/19) and tight CI are the load-bearing statistics; absolute accuracy depends on test-segment length and isn't comparable to the Koyaanisqatsi numbers (different TR counts and spaces).
Four of 19 subjects went slightly negative — alignment is a mean effect, not a universal one. Which subject properties predict the sign of the delta is an open question the per-subject delta matrix (in the results JSON) now makes addressable.
Everything above is plumbing — it tells you when pooling is safe, not what pooled data is for. The point of merging studies is discovering things no 20-person study could see. Three directions, in the order we intend to run them:
Cross-study pooling is a special case of the inter-subject alignment problem mature fMRI toolboxes already solve. Rather than one hand-rolled rotation, the pipeline should treat alignment as a variable: anatomical warping (the field's default), the linear functional rotation used here, and probabilistic shared-response models — all implemented in maintained packages with the validation literature attached — and then show the same figures computed under each. If a pooled result changes sign when the alignment method changes, it was never a finding about brains; it was a finding about preprocessing. A literature table of the alignment landscape — what each method assumes, when it fails, who validated it, and where the code lives — is on the Literature considered tab. Every citation there was verified, not recalled.
The first real payoff: with ~110 people who experienced a story, compute network-level functional connectivity in the shared space — which regions talk to which, when, and how that differs from the non-narrative cohort. This is exactly the question individual 20-person studies are too noisy to answer, and exactly what pooling exists for.
The shared basis the alignment already produces is a low-dimensional embedding of the stimulus-evoked response. Representational similarity and trajectory analyses on that embedding — how the group-wide brain state moves through the story, and where individual people depart from it — are the natural next layer up from time-segment matching.
Three findings, one map:
1. Functional alignment is not free signal. On a stimulus whose shared response is already anatomically aligned, the extra rotation reliably costs accuracy (Koyaanisqatsi: N = 93, Δ = −0.026, p < 0.001) — and the pipeline provably can detect a benefit when one exists (synthetic control: Δ = +0.25, 40/40).
2. On a narrative stimulus, the sign flips. Alignment reliably helps (Forrest Gump audio description: N = 19 × 8 runs, Δ = +0.011, 95% CI excludes zero, p < 0.001). This is the empirical core of the pooling pitch — now measured head-to-head against its own negative control rather than assumed from single-dataset studies.
3. The boundary tracks the story — with a stated confound. The same pipeline, same statistics, same public-data discipline produced opposite signs on two stimuli that differ most sharply in whether they tell one. The working model: narrative drives idiosyncratic, fine-grained topography that anatomical alignment blurs and functional alignment recovers; non-narrative visual spectacle drives responses that are already anatomically aligned, where the rotation only adds noise. Anyone planning to merge small studies should ask first which regime their stimulus lives in — the answer decides whether alignment helps or hurts.
The stimulus-boundary reading is the interpretation we find most plausible, but this pair of datasets cannot separate it from the alternatives. Going from Koyaanisqatsi to StudyForrest changes narrative and at least four other things at once: modality (visual film vs. auditory audio description), field strength (3T vs. 7T), reference space (fsaverage5 surface vs. volumetric group template), and subject pool (N = 93 vs. 19, different people). Narrative is one of several candidate explanations for the sign difference, and this design cannot tell them apart.
That does not make the boundary reading wrong — it makes it not yet isolated. Isolating it needs a comparison that holds modality, field strength and space fixed and varies only the stimulus: for example, narrative vs. temporally scrambled narrative within a single acquisition, or a narrative film and a non-narrative film in the same scanner. The full StudyForrest dataset supports the scrambled-narrative version with no new data.
Two stimuli are two points on a spectrum, not the spectrum — and the confound box above names what two points can't separate. The decisive next steps: a scrambled-narrative control within StudyForrest itself (same audio, destroyed story — isolates narrative with modality, field strength and space held fixed), a narrative and non-narrative visual film pair in one acquisition (isolates narrative from modality the other way), and the AOMIC download completed toward its full ~927. The pipeline and data-access patterns are public and deterministic — each is a config change plus compute time.
The data are public and the analysis is deterministic, so the numbers above are recoverable from scratch: the exact accessions, derivative types, subset choices and parameters follow. The scripts and result JSONs themselves live on the research machine that ran the analysis and are not yet published in this repository — the file list below is a record of what was produced, not yet an offer of download. Publishing them in-repo is outstanding work; until then the pipeline is short enough to rebuild from this description. A longer, agent-oriented account of how this experiment was actually run — including the failures — lives on the agents page.
AOMIC-ID1000 — OpenNeuro ds003097, snapshot v1.2.0. 93 subjects of ~927 available with fsaverage5 moviewatching derivatives (L+R), ~2.4GB. Film: Koyaanisqatsi excerpt (visual, 3T). The 93 are a bandwidth-limited download subsample — the subjects whose L+R pair had downloaded completely when the analysis ran. Nothing was excluded on the basis of its result, but the subsample was not drawn under pre-specified randomisation: treat it as convenience, not a random sample. A re-run at full N is open to anyone with the bandwidth and would be a stronger test than ours.
StudyForrest — OpenNeuro ds000113, snapshot 1.3.0. Group-template (grpbold7Tad) non-linearly aligned BOLD from the 7T audio-description phase (spoken German audio description of Forrest Gump; auditory only, no visual stimulus), masked to the group brain-mask intersection (493,857 voxels). Runs 1–8 (451 TRs each), 19 of 20 subjects (sub-10 has no released audio-description data).
Both pulled directly from the OpenNeuro file API. No bespoke preprocessing: the distributed derivatives are used as shipped, because the claim is about alignment on top of a common space, not about preprocessing choices.
shared basis PCA, k = 100 (swept 10/20/50/100 on a 40-subject subset)
alignment orthogonal Procrustes (per subject, fit on train only)
design strict leave-one-subject-out (basis + group mean exclude the scored subject)
split train = first 2/3 of TRs, test = last 1/3 (contiguous)
evaluation time-segment matching on the held-out test segment
statistics paired sign test + permutation test on the mean difference
(1000 draws; Forrest all-runs adds a bootstrap 95% CI on the per-subject mean delta)
seed 42
chance empirical (autocorrelation-aware) ≈ 0.01; nominal 1/nTR
aomic_loso_results.json — the Koyaanisqatsi leakage-free result (N=93, per-subject accuracies, sign, permutation p, seed, runtime).
aomic_tier1_results.json — the corrected pilot (N=93), consistent with the LOSO result.
forrest_loso_allruns_results.json — the narrative test at full power (19 subjects × 8 runs): anatomical 0.0408 / aligned 0.0517, Δ = +0.0109, 95% CI [0.0065, 0.0153], sign 15/19, permutation p < 0.001 (Monte-Carlo bound with 1000 draws; exact enumeration of the full 2^19 sign-flip distribution gives p = 0.00018, so the bound is conservative), plus the full 19 × 8 per-run delta matrix. Written by hyperalign_forrest_allruns.py.
forrest_loso_results.json — the run-01 pilot (N=8): Δ = +0.002, p = 0.86. Kept as the honest record of what an underpowered slice looked like.
Errors caught and fixed during this work, recorded rather than smoothed over.
(1) A measurement bug (operator precedence) made the first AOMIC pilot's accuracies
unreliable; found on adversarial re-read, everything re-run. (2) The first Forrest
analysis was published here as "inconclusive, underpowered" — true at the time, and now
superseded by the full 8-run result above. (3) The AOMIC dataset paper was initially cited
under a wrong DOI resolving to an unrelated paper; corrected to Snoek et al. (2021) after
a title match. (4) A concatenated-runs Forrest analysis was OOM-killed twice against the
4GB container memory cap; the per-run pooled design replaced it and is statistically
stronger anyway (a subject × run delta matrix beats a single concatenated number).
(5) The first published version of this page described the StudyForrest
grpbold7Tad data as a
film the subjects watched. It is the 7T audio description —
auditory only, no visual stimulus. The page's own citation (Hanke et al., 2014, "an audio
movie") said so; we misread it. Corrected throughout, with the modality confound it
introduces now stated in §07. (6)
dataset's complement of complete moviewatching derivatives; AOMIC ships roughly 927, and
93 was our download subsample — corrected in §08. (7) On reader feedback (Jules, 2026-08-12)
the page was restructured for a general audience: method detail moved into collapsible panels,
a broken lede sentence repaired, the 19 × 8 notation clarified, and a "What's next" section
added covering alignment-as-a-toggle and the post-alignment analyses. The references below are the verified
set.
Every reference below was verified for existence, metadata accuracy, and retraction status against Crossref (and a title match where no DOI exists), most recently on 2026-08-09. The badge is the outcome of that check, not an assertion.