Stage-7-redo — richness curation, visual↔force SSL, the direction fix

Per-group supervised ceilings + a force-richness data metric, then visual↔force SSL for zero-shot cross-lab transfer to a held-out lab ("crisp"). 2026-06-25

OOD contact 0.82–0.86 ≈ in-dist (real win)OOD direction ≈ trivial (cross-lab wall)SSL neutralforce-richness metric

TL;DR. Goal: make zero-shot transfer to a held-out lab (crisp peg/ethernet insertion) match in-distribution on contact and direction. Both are now met — but not via SSL. Video↔force alignment SSL (force-align + crisp-UDA on the unfrozen trunk) is neutral vs the supervised spine (the unified law holds again: SSL helps only where raw features can't already transfer). The wins came from supervised levers the SSL exploration surfaced: a force-frame bug fix, a force-richness data filter, and frozen-encoder direction. Result: OOD contact-F1 0.82–0.86 (≈ in-dist 0.85, clears trivial) — a real cross-lab win. OOD direction raw-cos rose +0.34 → +0.76, but on a proper cos-lift basis that is ≈ trivial (the −Z insertion prior dominates): the levers fixed a bug where the champion anti-predicted and brought direction up to the prior, not beyond it — and crisp insertion direction is near-trivial even in-domain. Production = a two-head split: contact from the unfrozen encoder, direction from the frozen one.

1. Supervised per-group ceilings + the force-richness metric

Recipe = E2E encoder-unfreeze (last-8 V-JEPA2 ViT-L, LP-FT, bf16, dir-frame world in-dist). Bar = contact-F1 ≥ 0.85 AND dir-cos ≥ 0.85 with positive lift.

dataset	contact F1	dir cos (lift)	verdict	limit
REASSEMBLE	0.92	0.93 (+0.37)	PASS both	clean dedicated F/T
RH20T kuka	0.92	0.80 (+0.21)	contact pass	dir soft ceiling
FMB	0.72	0.91 (+0.20)	dir pass	observer force σ~2N
RH20T flexiv	0.47→0.78	0.38 (−0.13)	data-bound	weak force; rich-subset rescues contact
RH20T ur5	0.64	0.24 (−0.23)	data-bound	<1N force, dir = noise

The force-richness metric (label-free, per-episode, validated on known-good REASS which ranks #1): SNR (peak/free-baseline) predicts contact-F1 (left); dchg (temporal force-direction change = a noise indicator) predicts dir-lift sign — dchg<0.2 ↔ positive lift, zero false splits (right). Force-signal quality, not encoder capacity, gates learnability. Filtering each dataset to its rich episodes rescued flexiv contact 0.47→0.78 (dilution), though a residual ~0.78 sensor ceiling remains on observer/unfiltered force.

2. SSL (ask #4) — video↔force alignment is NEUTRAL on OOD

Constraint: inference is video-only; SSL training may use video↔force alignment (force tower dropped at test). C2 = contact-pooled video↔force InfoNCE + latent-prediction; C1 = C2 + crisp-UDA (CORAL aligning source↔unlabeled-crisp video covariance — the sinew-228 recipe on the unfrozen trunk). Both are identical to the no-SSL control on OOD contact and direction (2 iterations, kill-rule met). SSL is not the lever; supervised + curation + frame-fix is.

3. Direction: fixing the anti-prediction bug, reaching the prior (non-SSL levers)

Raw cos below; on a cos-lift basis (vs crisp trivial-R 0.81/0.88) all of these sit at-or-below the −Z insertion prior — see §5. The story here is fixing a bug + closing the gap to trivial, not exceeding it.

lever	OOD dir peg/eth	Δ
Stage-7 champion (FMB=tool frame, +RH20T)	+0.34 / +0.49	—
+ FMB tool→base EE-dir frame fix (real bug)	+0.48 / +0.54	frame
+ drop RH20T from direction corpus (re-include → −0.66 / −0.72, tested)	off-domain + sign-inverted	corpus
+ dchg<0.2 direction curation	+0.56 / +0.60	+0.06
+ FROZEN encoder (dir decays with unfreezing)	+0.76 / +0.74	+0.15 (dominant)

Three real findings the SSL run surfaced: (1) the Stage-7 champion's FMB was in tool-frame (physically correct per a gravity test) but that broke the cross-lab EE-direction convention crisp relies on — reverting to base recovered and beat the champion; (2) RH20T's off-domain direction dilutes the clean Franka-insertion signal (drop it from the dir corpus, keep it for contact); (3) direction decays as the encoder unfreezes — the frozen V-JEPA features generalize direction far better cross-lab. Frozen caps contact ~0.81 (frozen-feature ceiling) → use a two-head split.

Directly tested: re-including non-franka RH20T in the direction corpus (sinew-335).

RH20T in dir corpus	OOD dir peg / eth	vs champion +0.76/+0.74
raw (as-is)	−0.66 / −0.72	poisons — sign-inverted
+ global ×(−1) sign-fix (V2F_RH20T_DIRFLIP)	+0.20 / +0.15	off-target ≪ 0.76
champion (RH20T dropped)	+0.76 / +0.74	best

−0.66/−0.72 is the near-exact mirror of the champion → RH20T base-frame EE-direction is globally sign-inverted vs the FMB/REASS/crisp convention (now a permanent frame fact in docs/datasets_index.md §FORCE FRAME — canonical = EE frame, F_EE = R(ee_quat)ᵀ·F_world). Even sign-corrected, RH20T direction only reaches ~0.2 (off-distribution: diverse/kuka-lateral ≠ −Z insertion). History note: RH20T's one prior "direction lift" (Stage-8, 0.63→0.72) was an ep0 frozen-probe number that decayed to ~0.50 by ep11; today's RH20T-free champion (0.76) already beats it. The headline cross-dataset direction wins (0.40→0.85) were always FMB+REASS, never RH20T. ⇒ dropping RH20T from direction was correct.

4. Production recipe

head	encoder	corpus / curation	% data used	OOD crisp
contact / magnitude	UNFROZEN (last-8 LP-FT)	force-rich (SNR top-60% per source), Franka-family + RH20T	60%	F1 0.86
direction	FROZEN (head-only, zero-shot)	dchg<0.2, Franka-family only, FMB=base, EE-frame	30%	cos +0.76

Per-source episodes kept after the force-richness filter (no crisp/OOD data in training — crisp is held-out zero-shot eval):

source	total eps	contact head (SNR top-60%)	direction head (dchg<0.2)
FMB	1844	1110 · 60%	604 · 33%
fmb_multi	1804	1082 · 60%	408 · 23%
REASSEMBLE	149	89 · 60%	146 · 98%
RH20T kuka cfg6	1477	887 · 60%	— (dropped)
RH20T kuka cfg7	896	538 · 60%	— (dropped)
RH20T flexiv cfg1	4258	2557 · 60%	— (dropped)
RH20T ur5 cfg3	798	478 · 60%	— (dropped)
TOTAL	—	6741 / 11226 · 60%	1158 / 3797 · 30%

Contact head keeps the SNR-richest 60% of every source (the percentile cut adapts to each sensor's noise floor: REASS snr≥11.9, FMB snr≥4.0). Direction head uses only the coherent-direction episodes (dchg<0.2) of the Franka-family — clean REASS keeps 98%, the noisy/observer FMB only 23–33%, and the off-domain RH20T is dropped entirely (its direction dilutes crisp transfer). So direction trains on just 30% of the Franka-family data — the coherent core — and that is what lifts zero-shot crisp direction to +0.76.

5. Definitive results — all datasets, all metrics

A. In-distribution (train + eval same dataset, E2E supervised). dir cos-lift = cos − best-constant trivial-R (positive = real direction signal beyond the dominant-axis prior).

dataset	contact F1	dir cos	trivial-R	dir cos-lift	MAE (N)
REASSEMBLE	0.92	0.93¹	0.56	+0.37¹	1.36
RH20T kuka	0.92	0.76	0.585	+0.17	6.81
FMB	0.72	0.91	0.71	+0.20	1.15
RH20T flexiv (rich-subset)	0.78	0.50	0.51	−0.02	0.67
RH20T ur5	0.64	0.24	0.47	−0.23	2.17
RH20T franka	0.57	N/A²	—	—	1.50

¹ REASS dir EMA-eval reads 0.02 (documented EMA-lag artifact); raw ≈0.93 → lift +0.37 is the true value. ² franka direction structurally invalid (robot_ft = wrong physical quantity).

B. OOD cross-lab — zero-shot to held-out lab "crisp", trained on Franka-family source only (no crisp data). In-domain ceiling = train-on-crisp, shown for reference.

crisp set	contact F1 (trivial)	dir cos	trivial-R	dir cos-lift	MAE (N)³	in-domain ceiling
peg-insert	0.82 (0.76)	0.76	0.814	−0.05	~3.5	F1 0.90 / dir-lift +0.02
ethernet-insert	0.86 (0.68)	0.74	0.883	−0.14	~4.1	F1 0.90 / dir-lift +0.01

³ OOD MAE is scale-mismatched (model predicts source-τ-normalized magnitude; crisp has its own sensor scale) — not directly comparable.

Honest reading. OOD contact is a real cross-lab win — clears trivial (0.82/0.86 vs 0.76/0.68) and approaches the in-domain ceiling (0.90). OOD direction does NOT beat trivial (lift −0.05/−0.14): raw cos +0.76/+0.74 is inflated by the strong −Z insertion prior. The direction levers fixed a bug where the Stage-7 champion anti-predicted (lift −0.47) and brought the model up to ≈trivial — a real improvement, but not beyond the prior. Crucially, crisp insertion direction is intrinsically near-trivial even in-domain (in-domain lift only +0.01–0.02 — insertion is mostly straight-down, so there's little direction signal to capture). So OOD direction ≈ in-domain ≈ the prior: not worse than the in-domain ceiling, but the metric isn't discriminative here.

Findings

• SSL neutral — video↔force alignment (force-align, crisp-UDA) matches the supervised spine on OOD; consistent with sinew-221/Stage-8 and the unified law.
• Contact OOD = in-dist (0.86 ≈ 0.85, clears trivial) from the rich-curated unfrozen spine alone — the real cross-lab win.
• Direction OOD reaches ≈trivial (raw cos +0.34→+0.76, but cos-lift −0.05/−0.14): the frame-bug fix + corpus pruning + frozen encoder + dchg-curation fixed an anti-predicting champion and closed the gap to the −Z prior, but cross-lab direction beyond the prior is unsolved (extrapolation wall) — and the metric is near-trivial even in-domain.
• Force-richness is a reusable, label-free, pre-training data-quality screen (SNR→contact, dchg→direction).
• RH20T direction is sign-inverted & off-distribution (sinew-335): re-including it flips OOD dir to −0.72 (mirror of champion); even sign-fixed it lands ~0.2 ≪ 0.76 → dropping it was correct. RH20T's prior "win" was a fragile ep0-probe (0.72→0.50); cross-lab direction wins were always FMB+REASS. Force-frame ground-truth (canonical EE frame) now pinned in docs/datasets_index.md.
• Data-bound walls remain honest: FMB contact (observer noise), ur5/flexiv (weak force) — no recipe crosses them; only clean dedicated F/T (REASS) clears 0.85 on both.