{
 "schema": 1,
 "model": {
  "id": "96244b1f_12800",
  "name": "N16 nav model",
  "sha256": "01effc679eb46d6036ce0302673a87fe2dcfcc95a679daeeb929679e53c3647a",
  "source": {
   "kind": "huggingface",
   "ref": "3fe6f3dea4fa9613dd3582b9a3828e97d307ad9e",
   "url": "https://huggingface.co/commaai/openpilot_driving_models/blob/3fe6f3dea4fa9613dd3582b9a3828e97d307ad9e/96244b1f-6c4e-4adc-848d-eccd0b44b182/12800/big_driving_supercombo.onnx",
   "date": "2026-10-15"
  },
  "first_public": "2026-10-15",
  "on_master": null,
  "params_m": 393.67,
  "gmac": 51.9,
  "family": "residual",
  "notes": "Uploaded as \"add N16 nav model\"; never shipped. Takes a navigation map and route inputs, fed zeros here (no route). Vision base 2931d3d9 with an RL residual head."
 },
 "scored": {
  "date": "2026-10-28T00:14:18Z",
  "suite": "geval-1",
  "code": "139121bc1a13"
 },
 "baselines": [
  "lebowski_877m",
  "1365126a_12864"
 ],
 "sections": {
  "public_open_loop": {
   "status": "ok",
   "unit_of_evidence": "segment",
   "n": 39,
   "clusters": 23,
   "condition": "Measured. comma's public two-camera segments (openpilot CI data), native intrinsics and the device's own calibration, both cameras; every model on the same decoded frames. Truth is what the recording car did next: carState speed and acceleration, yaw rate from the device's cameraOdometry. Segments where the car never moves have no data. 39 of the 45 listed segments have a score, from 23 drives and 20 cars.",
   "baselines": [
    "lebowski_877m",
    "1365126a_12864"
   ],
   "metrics": [
    {
     "key": "lat_mae",
     "label": "Lateral error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1519,
     "median": 0.1341,
     "ci": [
      0.1195,
      0.1853
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.04319,
       "median": 0.03024,
       "ci": [
        0.02943,
        0.05641
       ],
       "n": 39,
       "wins": 3,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.02311,
       "median": 0.02055,
       "ci": [
        0.01317,
        0.03306
       ],
       "n": 39,
       "wins": 5,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "lat_mae_bl",
     "label": "Lateral error at the model's own best lead",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1419,
     "median": 0.1298,
     "ci": [
      0.1099,
      0.1754
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.04045,
       "median": 0.02249,
       "ci": [
        0.02482,
        0.05574
       ],
       "n": 39,
       "wins": 8,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": 0.02248,
       "median": 0.01325,
       "ci": [
        0.01213,
        0.03308
       ],
       "n": 39,
       "wins": 6,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "lat_corr",
     "label": "Lateral correlation with the car's later motion",
     "unit": "",
     "better": "higher",
     "value": 0.8277,
     "median": 0.9202,
     "ci": [
      0.7457,
      0.8971
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02992,
       "median": -0.02039,
       "ci": [
        -0.07169,
        0.005037
       ],
       "n": 39,
       "wins": 9,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      },
      "1365126a_12864": {
       "delta": -0.01899,
       "median": -0.01371,
       "ci": [
        -0.03721,
        -0.004165
       ],
       "n": 39,
       "wins": 6,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 5
      }
     }
    },
    {
     "key": "lat_gain",
     "label": "Lateral gain (1 = same size as the car's motion)",
     "unit": "",
     "better": "near1",
     "value": 0.7337,
     "median": 0.7214,
     "ci": [
      0.6437,
      0.8274
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.09198,
       "median": 0.05853,
       "ci": [
        0.01007,
        0.1668
       ],
       "n": 39,
       "wins": 11,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 8
      },
      "1365126a_12864": {
       "delta": 0.02767,
       "median": 0.03633,
       "ci": [
        -0.02422,
        0.07631
       ],
       "n": 39,
       "wins": 9,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 6
      }
     }
    },
    {
     "key": "lat_rate_rms",
     "label": "Lateral action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.4711,
     "median": 0.4217,
     "ci": [
      0.3799,
      0.5762
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.1928,
       "median": -0.1776,
       "ci": [
        -0.2352,
        -0.1455
       ],
       "n": 39,
       "wins": 37,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 23
      },
      "1365126a_12864": {
       "delta": 0.01968,
       "median": 0.02603,
       "ci": [
        -0.007571,
        0.03907
       ],
       "n": 39,
       "wins": 6,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "lat_best_lag",
     "label": "Lead of the lateral request over the car's motion",
     "unit": "s",
     "better": "lower",
     "value": 0.5386,
     "median": 0.5,
     "ci": [
      0.4289,
      0.6596
     ],
     "n": 35,
     "clusters": 22,
     "paired": {
      "lebowski_877m": {
       "delta": 0.1544,
       "median": 0.125,
       "ci": [
        0.08485,
        0.2396
       ],
       "n": 34,
       "wins": 4,
       "ties": 4,
       "clusters": 22,
       "cluster_wins": 1
      },
      "1365126a_12864": {
       "delta": 0.06857,
       "median": 0.05,
       "ci": [
        0.03444,
        0.113
       ],
       "n": 35,
       "wins": 3,
       "ties": 10,
       "clusters": 22,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "lon_mae",
     "label": "Longitudinal error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.3167,
     "median": 0.2983,
     "ci": [
      0.2742,
      0.366
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.008513,
       "median": 0.005901,
       "ci": [
        -0.008462,
        0.02783
       ],
       "n": 39,
       "wins": 14,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 8
      },
      "1365126a_12864": {
       "delta": 0.01594,
       "median": 0.01219,
       "ci": [
        0.004487,
        0.02746
       ],
       "n": 39,
       "wins": 14,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 9
      }
     }
    },
    {
     "key": "lon_corr",
     "label": "Longitudinal correlation",
     "unit": "",
     "better": "higher",
     "value": 0.7287,
     "median": 0.8474,
     "ci": [
      0.6257,
      0.8173
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.01099,
       "median": 0.008126,
       "ci": [
        -0.01493,
        0.03581
       ],
       "n": 39,
       "wins": 23,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 14
      },
      "1365126a_12864": {
       "delta": -0.009947,
       "median": -0.009404,
       "ci": [
        -0.02125,
        0.003718
       ],
       "n": 39,
       "wins": 13,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "lon_brake_mae",
     "label": "Longitudinal error while braking",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.4363,
     "median": 0.4287,
     "ci": [
      0.3729,
      0.5087
     ],
     "n": 30,
     "clusters": 21,
     "paired": {
      "lebowski_877m": {
       "delta": 0.001968,
       "median": 0.02602,
       "ci": [
        -0.02514,
        0.03511
       ],
       "n": 30,
       "wins": 12,
       "ties": 0,
       "clusters": 21,
       "cluster_wins": 9
      },
      "1365126a_12864": {
       "delta": 0.03573,
       "median": 0.04055,
       "ci": [
        0.01322,
        0.06181
       ],
       "n": 30,
       "wins": 7,
       "ties": 0,
       "clusters": 21,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "lon_rate_rms",
     "label": "Longitudinal action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.9005,
     "median": 0.8208,
     "ci": [
      0.7969,
      1.014
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": -0.4325,
       "median": -0.3706,
       "ci": [
        -0.5612,
        -0.3217
       ],
       "n": 39,
       "wins": 36,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 22
      },
      "1365126a_12864": {
       "delta": 0.01069,
       "median": 0.01384,
       "ci": [
        -0.02498,
        0.0431
       ],
       "n": 39,
       "wins": 13,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "v3_mae",
     "label": "Plan speed error at 3 s",
     "unit": "m/s",
     "better": "lower",
     "value": 0.7667,
     "median": 0.7474,
     "ci": [
      0.6565,
      0.9018
     ],
     "n": 39,
     "clusters": 23,
     "paired": {
      "lebowski_877m": {
       "delta": 0.05091,
       "median": 0.04574,
       "ci": [
        0.01517,
        0.08501
       ],
       "n": 39,
       "wins": 13,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 7
      },
      "1365126a_12864": {
       "delta": 0.006492,
       "median": 0.005619,
       "ci": [
        -0.01238,
        0.02415
       ],
       "n": 39,
       "wins": 18,
       "ties": 0,
       "clusters": 23,
       "cluster_wins": 10
      }
     }
    }
   ],
   "per_unit_file": "per_unit/96244b1f_12800/public_open_loop.csv"
  },
  "public_closed_loop": {
   "status": "ok",
   "unit_of_evidence": "segment",
   "n": 28,
   "clusters": 19,
   "condition": "Measured. comma's worldmodel-4B generates each next frame at 5 Hz from 9 history frames, 5 recorded anchor frames and the car's pose; the model drives comma's ideal car (the requested curvature and acceleration happen at once) for 7.2 s from the recorded start. Distance is to the recording car's own path. One window per segment, nearest the middle, where the car holds at least 5 m/s for the window and the 4.8 s before it; segments with no such stretch are left out. Seeds 0 and 1, averaged per segment. 28 of the 45 listed segments have a score, from 19 drives and 17 cars.",
   "baselines": [
    "lebowski_877m",
    "1365126a_12864"
   ],
   "metrics": [
    {
     "key": "xtrack_2s",
     "label": "Distance from the recorded path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.08837,
     "median": 0.05381,
     "ci": [
      0.04714,
      0.1595
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.05932,
       "median": -0.01924,
       "ci": [
        -0.1236,
        -0.01545
       ],
       "n": 28,
       "wins": 22,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 18
      },
      "1365126a_12864": {
       "delta": 0.001077,
       "median": -0.01035,
       "ci": [
        -0.01874,
        0.02818
       ],
       "n": 28,
       "wins": 18,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "xtrack_4s",
     "label": "Distance from the recorded path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.1702,
     "median": 0.09467,
     "ci": [
      0.1029,
      0.2549
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.108,
       "median": -0.003776,
       "ci": [
        -0.271,
        -0.004106
       ],
       "n": 28,
       "wins": 14,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 9
      },
      "1365126a_12864": {
       "delta": 9.161e-05,
       "median": -0.01572,
       "ci": [
        -0.03576,
        0.04611
       ],
       "n": 28,
       "wins": 16,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 11
      }
     }
    },
    {
     "key": "xtrack_7s",
     "label": "Distance from the recorded path at 7.2 s",
     "unit": "m",
     "better": "lower",
     "value": 1.129,
     "median": 0.2739,
     "ci": [
      0.3285,
      2.275
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.08475,
       "median": -0.02438,
       "ci": [
        -0.5086,
        0.891
       ],
       "n": 28,
       "wins": 15,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 12
      },
      "1365126a_12864": {
       "delta": 0.13,
       "median": -0.02175,
       "ci": [
        -0.0804,
        0.4371
       ],
       "n": 28,
       "wins": 16,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 12
      }
     }
    },
    {
     "key": "fail_1m",
     "label": "Share of episodes ever more than 1 m off the path",
     "unit": "",
     "better": "lower",
     "value": 0.2321,
     "median": 0.0,
     "ci": [
      0.08065,
      0.4062
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.0,
       "median": 0.0,
       "ci": [
        -0.05362,
        0.05357
       ],
       "n": 28,
       "wins": 1,
       "ties": 26,
       "clusters": 19,
       "cluster_wins": 1
      },
      "1365126a_12864": {
       "delta": 0.08929,
       "median": 0.0,
       "ci": [
        0.0,
        0.2083
       ],
       "n": 28,
       "wins": 0,
       "ties": 25,
       "clusters": 19,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lat_jerk",
     "label": "Requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.205,
     "median": 0.1286,
     "ci": [
      0.1422,
      0.2754
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": -0.1206,
       "median": -0.08962,
       "ci": [
        -0.1684,
        -0.07085
       ],
       "n": 28,
       "wins": 25,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 17
      },
      "1365126a_12864": {
       "delta": 0.004135,
       "median": 0.006992,
       "ci": [
        -0.02477,
        0.03123
       ],
       "n": 28,
       "wins": 10,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 7
      }
     }
    },
    {
     "key": "long_jerk",
     "label": "Requested longitudinal jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.3904,
     "median": 0.3332,
     "ci": [
      0.292,
      0.504
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.04405,
       "median": -0.004609,
       "ci": [
        -0.006412,
        0.102
       ],
       "n": 28,
       "wins": 14,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 9
      },
      "1365126a_12864": {
       "delta": 0.03409,
       "median": 0.004636,
       "ci": [
        -0.002911,
        0.07827
       ],
       "n": 28,
       "wins": 12,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 9
      }
     }
    },
    {
     "key": "speed_err",
     "label": "Speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 1.402,
     "median": 1.101,
     "ci": [
      0.9945,
      1.855
     ],
     "n": 28,
     "clusters": 19,
     "paired": {
      "lebowski_877m": {
       "delta": 0.3003,
       "median": 0.2428,
       "ci": [
        0.0483,
        0.5662
       ],
       "n": 28,
       "wins": 7,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": 0.226,
       "median": 0.2687,
       "ci": [
        0.07779,
        0.367
       ],
       "n": 28,
       "wins": 6,
       "ties": 0,
       "clusters": 19,
       "cluster_wins": 4
      }
     }
    }
   ],
   "per_unit_file": "per_unit/96244b1f_12800/public_closed_loop.csv"
  },
  "truck_open_loop": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 4,
   "clusters": 3,
   "condition": "Measured. The G55's recorded drives replayed as the truck runs the model: narrow camera only, road-plane warp to 1.45 m (plane:1.45). Truth is the truck's CAN speed and yaw rate. Window = unit. 4 windows: 3 on an earlier camera mount, 1 on the current mount; 2 at night.",
   "baselines": [
    "lebowski_877m",
    "1365126a_12864"
   ],
   "metrics": [
    {
     "key": "lat_mae",
     "label": "Lateral error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1649,
     "median": 0.1787,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.03722,
       "median": 0.03505,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.02303,
       "median": 0.0213,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lat_mae_bl",
     "label": "Lateral error at the model's own best lead",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.1498,
     "median": 0.1562,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.02334,
       "median": 0.02571,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.01707,
       "median": 0.0147,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lat_corr",
     "label": "Lateral correlation with the car's later motion",
     "unit": "",
     "better": "higher",
     "value": 0.9267,
     "median": 0.9342,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": -0.03347,
       "median": -0.02965,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": -0.01435,
       "median": -0.01405,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lat_gain",
     "label": "Lateral gain (1 = same size as the car's motion)",
     "unit": "",
     "better": "near1",
     "value": 0.8128,
     "median": 0.8119,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.1294,
       "median": 0.1329,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.01596,
       "median": 0.007413,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lat_rate_rms",
     "label": "Lateral action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.5454,
     "median": 0.5449,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": -0.4111,
       "median": -0.4045,
       "ci": null,
       "n": 4,
       "wins": 4,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": 0.0009558,
       "median": 0.01349,
       "ci": null,
       "n": 4,
       "wins": 2,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "lat_best_lag",
     "label": "Lead of the lateral request over the car's motion",
     "unit": "s",
     "better": "lower",
     "value": 0.5625,
     "median": 0.65,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.225,
       "median": 0.25,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 1,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.0875,
       "median": 0.1,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "lon_mae",
     "label": "Longitudinal error",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.3394,
     "median": 0.3285,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.02493,
       "median": 0.01921,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.007556,
       "median": 0.004399,
       "ci": null,
       "n": 4,
       "wins": 1,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "lon_corr",
     "label": "Longitudinal correlation",
     "unit": "",
     "better": "higher",
     "value": 0.5403,
     "median": 0.5971,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": -0.04207,
       "median": -0.03911,
       "ci": null,
       "n": 4,
       "wins": 0,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": -0.004645,
       "median": 0.000316,
       "ci": null,
       "n": 4,
       "wins": 2,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "lon_brake_mae",
     "label": "Longitudinal error while braking",
     "unit": "m/s²",
     "better": "lower",
     "value": 0.5558,
     "median": 0.5734,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.0403,
       "median": 0.03717,
       "ci": null,
       "n": 4,
       "wins": 1,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 1
      },
      "1365126a_12864": {
       "delta": 0.01592,
       "median": -0.005755,
       "ci": null,
       "n": 4,
       "wins": 3,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "lon_rate_rms",
     "label": "Longitudinal action rate, RMS (smoothness)",
     "unit": "m/s³",
     "better": "lower",
     "value": 1.823,
     "median": 1.482,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.07313,
       "median": -0.1216,
       "ci": null,
       "n": 4,
       "wins": 2,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 2
      },
      "1365126a_12864": {
       "delta": 0.08682,
       "median": 0.0888,
       "ci": null,
       "n": 4,
       "wins": 1,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "v3_mae",
     "label": "Plan speed error at 3 s",
     "unit": "m/s",
     "better": "lower",
     "value": 1.571,
     "median": 1.308,
     "ci": null,
     "n": 4,
     "clusters": 3,
     "paired": {
      "lebowski_877m": {
       "delta": 0.543,
       "median": 0.3971,
       "ci": null,
       "n": 4,
       "wins": 1,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 1
      },
      "1365126a_12864": {
       "delta": 0.267,
       "median": 0.2573,
       "ci": null,
       "n": 4,
       "wins": 1,
       "ties": 0,
       "clusters": 3,
       "cluster_wins": 0
      }
     }
    }
   ],
   "per_unit_file": null
  },
  "truck_closed_loop": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 18,
   "clusters": 4,
   "condition": "Measured. worldmodel-4B as in the public closed loop, but the model drives a simulation of the G55's measured steering and powertrain response instead of the ideal car. Lane-following windows from the G55's drives, day and night; distance is to the driver's recorded path; seeds 0 and 1, averaged per window. 18 windows (9 day, 9 night).",
   "baselines": [
    "lebowski_877m",
    "1365126a_12864"
   ],
   "metrics": [
    {
     "key": "xtrack_2s",
     "label": "Distance from the recorded path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.2386,
     "median": 0.196,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02964,
       "median": -0.02773,
       "ci": null,
       "n": 18,
       "wins": 10,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": 0.02771,
       "median": 0.009061,
       "ci": null,
       "n": 18,
       "wins": 8,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "xtrack_4s",
     "label": "Distance from the recorded path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.5436,
     "median": 0.4456,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.01837,
       "median": -0.006574,
       "ci": null,
       "n": 18,
       "wins": 9,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": 0.03131,
       "median": -0.02845,
       "ci": null,
       "n": 18,
       "wins": 13,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "xtrack_7s",
     "label": "Distance from the recorded path at 7.2 s",
     "unit": "m",
     "better": "lower",
     "value": 1.355,
     "median": 0.6382,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.2498,
       "median": 7.728e-05,
       "ci": null,
       "n": 18,
       "wins": 9,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 2
      },
      "1365126a_12864": {
       "delta": -0.4006,
       "median": -0.11,
       "ci": null,
       "n": 18,
       "wins": 14,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "fail_1m",
     "label": "Share of episodes ever more than 1 m off the path",
     "unit": "",
     "better": "lower",
     "value": 0.3333,
     "median": 0.0,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02778,
       "median": 0.0,
       "ci": null,
       "n": 18,
       "wins": 3,
       "ties": 13,
       "clusters": 4,
       "cluster_wins": 2
      },
      "1365126a_12864": {
       "delta": -0.1667,
       "median": 0.0,
       "ci": null,
       "n": 18,
       "wins": 6,
       "ties": 11,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "lat_jerk",
     "label": "Requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.3163,
     "median": 0.2317,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.2286,
       "median": -0.1914,
       "ci": null,
       "n": 18,
       "wins": 16,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": -0.06616,
       "median": 0.01847,
       "ci": null,
       "n": 18,
       "wins": 8,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "long_jerk",
     "label": "Requested longitudinal jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.3932,
     "median": 0.2444,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": 0.03186,
       "median": -0.04722,
       "ci": null,
       "n": 18,
       "wins": 11,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 1
      },
      "1365126a_12864": {
       "delta": 0.00934,
       "median": -0.009789,
       "ci": null,
       "n": 18,
       "wins": 11,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "speed_err",
     "label": "Speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 0.9761,
     "median": 0.7855,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.08173,
       "median": 0.07705,
       "ci": null,
       "n": 18,
       "wins": 8,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": -0.07307,
       "median": -0.01506,
       "ci": null,
       "n": 18,
       "wins": 10,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    }
   ],
   "per_unit_file": null
  },
  "truck_recovery": {
   "status": "ok",
   "unit_of_evidence": "window",
   "n": 20,
   "clusters": 5,
   "condition": "Measured. The truck closed loop with a disturbance while the model drives; only the tests listed ran. 2° off heading: an extra 2°/s yaw for the first second. 18 windows (9 day, 9 night). 0.5 m off centre: an extra yaw-rate pulse, out and back over the first 2 s, that would move an uncorrected car 0.5 m sideways with no net heading change. 18 windows (9 day, 9 night). Braking lead: windows where the car ahead braked; the gap is derived from the lead distance seen on the recording minus how far the model ran ahead of the recorded car. 2 windows (2 day, 0 night).",
   "baselines": [
    "lebowski_877m",
    "1365126a_12864"
   ],
   "metrics": [
    {
     "key": "heading_xtrack_2s",
     "label": "2° off heading: distance from the path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.3969,
     "median": 0.343,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02999,
       "median": -0.02004,
       "ci": null,
       "n": 18,
       "wins": 11,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": -0.02439,
       "median": -0.02574,
       "ci": null,
       "n": 18,
       "wins": 15,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      }
     }
    },
    {
     "key": "heading_xtrack_4s",
     "label": "2° off heading: distance from the path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.5853,
     "median": 0.3556,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": 0.04165,
       "median": 0.02628,
       "ci": null,
       "n": 18,
       "wins": 7,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 2
      },
      "1365126a_12864": {
       "delta": 0.02654,
       "median": -0.06184,
       "ci": null,
       "n": 18,
       "wins": 13,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "heading_lat_jerk",
     "label": "2° off heading: requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.4387,
     "median": 0.3122,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.3882,
       "median": -0.5008,
       "ci": null,
       "n": 18,
       "wins": 15,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": 0.03063,
       "median": 0.02718,
       "ci": null,
       "n": 18,
       "wins": 5,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 1
      }
     }
    },
    {
     "key": "offcentre_xtrack_2s",
     "label": "0.5 m off centre: distance from the path at 2 s",
     "unit": "m",
     "better": "lower",
     "value": 0.3228,
     "median": 0.2973,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.02609,
       "median": -0.01894,
       "ci": null,
       "n": 18,
       "wins": 12,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": -0.01819,
       "median": -0.04172,
       "ci": null,
       "n": 18,
       "wins": 12,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      }
     }
    },
    {
     "key": "offcentre_xtrack_4s",
     "label": "0.5 m off centre: distance from the path at 4 s",
     "unit": "m",
     "better": "lower",
     "value": 0.6605,
     "median": 0.4274,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": 0.001408,
       "median": -0.0326,
       "ci": null,
       "n": 18,
       "wins": 12,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 3
      },
      "1365126a_12864": {
       "delta": 0.1622,
       "median": 0.03841,
       "ci": null,
       "n": 18,
       "wins": 8,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "offcentre_lat_jerk",
     "label": "0.5 m off centre: requested lateral jerk, RMS",
     "unit": "m/s³",
     "better": "lower",
     "value": 0.5714,
     "median": 0.4526,
     "ci": null,
     "n": 18,
     "clusters": 4,
     "paired": {
      "lebowski_877m": {
       "delta": -0.6015,
       "median": -0.6006,
       "ci": null,
       "n": 18,
       "wins": 18,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 4
      },
      "1365126a_12864": {
       "delta": -0.1139,
       "median": 0.02313,
       "ci": null,
       "n": 18,
       "wins": 8,
       "ties": 0,
       "clusters": 4,
       "cluster_wins": 2
      }
     }
    },
    {
     "key": "brake_ahead_4s",
     "label": "Braking lead: distance ahead of the recorded car at 4 s",
     "unit": "m",
     "better": "lower",
     "value": -0.3434,
     "median": -0.3434,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": 1.45,
       "median": 1.45,
       "ci": null,
       "n": 2,
       "wins": 1,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": 0.6278,
       "median": 0.6278,
       "ci": null,
       "n": 2,
       "wins": 0,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "brake_gap_min",
     "label": "Braking lead: smallest gap to the lead (derived)",
     "unit": "m",
     "better": "higher",
     "value": 24.93,
     "median": 24.93,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": -4.351,
       "median": -4.351,
       "ci": null,
       "n": 2,
       "wins": 1,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": -2.024,
       "median": -2.024,
       "ci": null,
       "n": 2,
       "wins": 0,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      }
     }
    },
    {
     "key": "brake_speed_err",
     "label": "Braking lead: speed error against the recorded speed, RMS",
     "unit": "m/s",
     "better": "lower",
     "value": 1.489,
     "median": 1.489,
     "ci": null,
     "n": 2,
     "clusters": 1,
     "paired": {
      "lebowski_877m": {
       "delta": 0.4219,
       "median": 0.4219,
       "ci": null,
       "n": 2,
       "wins": 1,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 0
      },
      "1365126a_12864": {
       "delta": -0.2226,
       "median": -0.2226,
       "ci": null,
       "n": 2,
       "wins": 1,
       "ties": 0,
       "clusters": 1,
       "cluster_wins": 1
      }
     }
    }
   ],
   "per_unit_file": null
  }
 },
 "stills": [],
 "limits": [
  "Open-loop scores compare a model's action with what the recording driver or car did; a model that drives differently but well is scored as wrong. comma's RL residual makes the lateral action smoother, and earlier than its base's by 0.10 to 0.26 s (measured on the G55 and on 16 of comma's cars); both score as error here. Measured on the 2026-10-27 deep dive: open-loop accuracy did not predict closed-loop driving.",
  "Closed-loop numbers come from a generative world model, not a car. In a separate test on the G55's windows, worldmodel-4B followed an added yaw command through 2 s but showed 0 to 20% of an added lateral offset 3 to 4 s later; past about 4 s the recorded anchor frames also pull every model back toward the recorded path. The front-page ranking uses 2 s.",
  "Intervals are 95% percentile bootstraps that resample whole drives (the drive a public segment was cut from, the recorded drive a truck window comes from), so segments of one drive count once. Below 10 drives there is no interval: the truck sections come from fewer drives and report means and win counts only.",
  "Paired deltas are model minus baseline on the units both have (for lateral gain, the change in its distance from 1); wins count units where the model is better in the metric's direction.",
  "The lead of the lateral request is the shift, searched from 0 to 1.5 s, at which the request best correlates with the car's later lateral motion; a larger value means the request comes earlier. A value at either edge is censored and left out. Plan speed error at 3 s reads the plan head, which does not separate models built on one frozen base: it is identical on every segment for the four f22e06e5 models, and for Cinque v2 and v3.",
  "Truck open loop pools few windows (camera mounts changed between drives); read it as a check that a model works on the truck's camera, not a ranking.",
  "Truck closed loop: the steering and powertrain simulation is fitted to the G55, not validated on it in closed loop; the reference is the driver's own line, not the lane centre.",
  "Recovery tests are disturbances rendered by the world model (yaw it renders; a one-frame jump it does not, so none is used). The braking-lead test is not built yet: the drives the truck closed loop can use hold no window where a car ahead brakes.",
  "This model takes a navigation map and route; every run here feeds zeros (no route), which is not how it would be used with navigation."
 ]
}