Start here
About 60 secondsThe idea, in a minute.
- The question
- Does choosing replay memories together preserve more old behavior than ranking each memory independently?
- What we found
- Direct complementary selection reduced old-behavior error by 12.4% versus random replay, while the learned selector failed to recover that advantage.
- Why it matters
- Repeatedly choosing similar memories can waste a scarce replay budget. Interactions between memories may matter more than their individual scores.
Five small synthetic models; direct selection uses extra labeled probes and much more computation, so this is an opportunity diagnostic.
A concrete example
A useful group beats four useful singletons
Recorded aggregate experiment, not a particular hand-picked correction.
- 01
Rank alone
Direct individual ranking reached protected MSE 0.02662.
- 02
Account for earlier choices
Complementary scoring reached 0.02361 with the same four slots.
- 03
Pay for the measurement
Online cost rose from 3.77 to 12.18 ms per correction.
The selection interaction is useful, but measuring it is expensive.
Archived experiment reports and measurementsRead the formal abstract
Four individually useful memories need not form the most useful group. This experiment asks whether a replay selector can do better by measuring each candidate in the context of memories already selected.
On 320 held-out corrections across five models, direct complementary selection lowered protected-behavior MSE from 0.02695 to 0.02361 versus random replay, with only 0.6% higher new-rule error. Learned individual repair instead had 3.1% higher protected error than random. The joint benefit was consistent in direction across models, but direct selection cost roughly fifteen times random online and used privileged calibration labels. The result identifies useful selection structure without delivering an efficient learned solution.
01 Research question
Does choosing replay memories together preserve more old behavior than ranking each memory independently?
02 Experiment setup
The full procedure, model settings, datasets, and evaluation choices are recorded here.
Explore the complete methodology
Executed September 17, 2026 on one Mac CPU thread. The model is a 1,033-parameter, two-hidden-layer tanh regressor with twelve contexts, ninety-six raw memories, four replay slots, 1,200 pretraining steps, and learning rate 0.02. Each correction resets the model, takes a correction-only warm-up step, then combines correction and replay gradients.
Five fresh seeds 10–14 each evaluate 64 final correction episodes. Selector training uses 192 episodes on contexts 0–7, checkpoint validation uses 32 episodes on 8–9, and policy calibration uses another 48 episodes on 8–9. Final corrections target contexts 10–11, which were present only in base prediction training. Each final episode scores 512 fresh protected and 256 fresh new-rule examples.
Random, measured-damage, learned individual repair, direct individual repair, and direct complementary repair all choose four distinct memories excluding the corrected context. Direct methods score protected-calibration loss improvement minus positive correction-support harm. They use 128 labeled calibration examples that the learned deployment selector does not receive.
For complementary selection, each next memory is tested with the selected partial set. Each slot has strength/4 throughout, preventing an accidental change in update scale. Eight replay strengths are calibrated into a mixture of at most two settings under a 1.10-times no-replay new-error allowance; the mixture is frozen before final episodes. This is greedy selection, not a proof of globally optimal subsets.
03 What happened?
Direct complementary selection reduced old-behavior error by 12.4% versus random replay, while the learned selector failed to recover that advantage.
Protected MSE ↓ · scale shown from 0 to 0.035
How to read thisComplementary selection improves the old-behavior loss while allowing a small aggregate increase in new-rule error. Its additional labels and runtime are not hidden.
The limitFive small synthetic models; direct selection uses extra labeled probes and much more computation, so this is an opportunity diagnostic.
Five model means; sixty-four held-out corrections per model. The main acquisition comparison is approximate and uses the actual final errors shown.
Full results and data table
| Selector | Protected MSE ↓ | New-rule MSE ↓ |
|---|---|---|
| Random | 0.02695 | 0.03117 |
| Learned | 0.02778 | 0.03087 |
| Direct individual | 0.02662 | 0.03076 |
| Complementary | 0.02361 | 0.03135 |
Primary old/new MSE was random 0.02695/0.03117, measured damage 0.03227/0.03052, learned repair 0.02778/0.03087, direct individual repair 0.02662/0.03076, and complementary repair 0.02361/0.03135. No replay gave 0.03961/0.02766.
Complementary selection reduced old error 12.4% versus random and 11.3% versus direct individual selection. Retention improved versus random in all five seeds, though one improved only about 0.1%. One seed had 8.2% higher new-rule error, so the aggregate acquisition tolerance is not a per-seed guarantee.
Learned repair improved retention over measured damage by 13.9%, but lost to random replay on average and won retention in only one of five models. At the declared 1.25 validation-budget sensitivity, complementary selection reduced both old error (11.3%) and new error (7.7%) versus random.
Online correction cost was 0.79 ms for random, 1.23 ms for learned repair, 3.77 ms for direct individual, and 12.18 ms for complementary selection. Including shared setup and per-method calibration over sixty-four uses, complementary selection cost 81.57 ms per use versus 10.70 ms for random. Deployment latency excludes final scoring and data generation.
04 What does this mean?
What this suggests
- Direct complementary selection reduced old-behavior error by 12.4% versus random replay, while the learned selector failed to recover that advantage.
- There is an available retention benefit from accounting for interactions among selected memories. This does not imply that complementarity alone explains every learned-selector failure: ranking quality, calibration transfer, and information access differ.
What we still don’t know
- This is independent local correction in a numerical world, not sequential continual learning or language-model training.
- Five models share one task family; calibration constraints do not transfer exactly to final contexts. Direct selection consumes additional labels unavailable to learned retrieval.
Read the full discussion
There is an available retention benefit from accounting for interactions among selected memories. This does not imply that complementarity alone explains every learned-selector failure: ranking quality, calibration transfer, and information access differ.
The direct method is best read as a teacher or diagnostic. An efficient approximation must earn both its supervision and runtime costs against a strong random baseline.
05 Limitations
Five small synthetic models; direct selection uses extra labeled probes and much more computation, so this is an opportunity diagnostic.
Read all study limitations
This is independent local correction in a numerical world, not sequential continual learning or language-model training.
Five models share one task family; calibration constraints do not transfer exactly to final contexts. Direct selection consumes additional labels unavailable to learned retrieval.
06 References & materials
Keep the complete record
Take the research with you.
The full write-up, source code, and recorded experiment materials.
Download everything together or choose individual files.