Neuvara

Rankings

How brain MRI models hold up across scanners.

We run brain MRI models on the same people scanned on many different scanners, and rank them by how consistently they measure the same brain. Round 1 uses public travelling-subject data, labelled as such. A hidden test set, which no model can be tuned to, is in development.

Submit a model How it works

PreviewThese values are placeholders that show the layout. The first round of results will be published here.

Round 1 · Public set · T1-weighted · 3T · Siemens, GE, Philips

What the numbers mean

One brain, many scanners, one answer.

Each person in the test set is scanned on every scanner, so their anatomy is identical throughout. Any change in a model's output from one scanner to the next comes from the scanner, not the brain.

  • Consistency (ICC)Whether the model still tells people apart when the scanner changes. 1 means each person gets the same measurement on every scanner. A model that gives everyone the same answer scores near 0, so it cannot look consistent by ignoring the image. Models are ranked on this. Higher is better.
  • Headline scoreThe median consistency across nine key structures: hippocampus, amygdala, thalamus, caudate, putamen, pallidum, lateral ventricles, cerebral white matter and cortex.
  • Cross-scanner variationHow much one person's measured volume changes between scanners, as a percentage. Lower is better.
  • Worst scannerThe largest average offset of any one scanner, as a share of the person's average volume. Shows whether one scanner is a problem.
  • TiesEvery score has a 95% interval. When the difference between two models is within that uncertainty, they share a rank, shown as 2=.
  • ScopeBrain structure segmentation and volumetry from T1-weighted MRI. Other kinds of model get their own tables and are never ranked against these.

Getting ranked

Private first. Public only if you choose.

  1. 1

    Submit

    Send your model as a container that takes a scan and returns a label map. Open-source models we run ourselves.

  2. 2

    Private results

    We run it on the benchmark test set and send you the results before anyone else sees them.

  3. 3

    Your choice

    Appear under your name, appear anonymised, or not appear at all.

Next step

See where your model ranks.

Submit a model for the next round, or get the full audit for the detail behind the number: which scanners, which scans, and why.

Submit a model See the audit