# An AI learned to predict its own changes—but chose the same updates

Action-specific self-model forecasts improve, while update control collapses to a simple learning-rate choice.

Author: Yehoyada Shtinmetz
Publication date: 2026-10-18
Version: 1.0

## The idea, in a minute

### Question

Does predicting how a model will change help it choose better learning updates?

### Finding

A learned self-model predicted changes more accurately, but both controllers chose the same half-strength update on every step.

### Why it matters

Better internal prediction is useful only if it changes a consequential decision. This experiment separates those two achievements.

### Caveat

One pretrained CIFAR-100 initialization; the useful prediction result used an exploratory higher learning rate. Production weights were not locally retrievable.

## Recorded example: The self-model did not change the decision

Recorded revised controller experiment.

### Improve the forecast

Learned residuals reduced validation prediction MSE by 25.5% at the higher update rate.

### Choose updates

Both utility heads picked half-strength on all 500 steps.

### Compare outcomes

Both reached JEPA loss 0.087218 and probe accuracy 19.42%.

Forecasting progress and decision progress need separate tests.

## Abstract

A model that can forecast its own learning might avoid harmful updates. But improved forecasting does not guarantee better control. We tested both claims separately using image representations and counterfactual update branches.

Per-block first-order forecasts plus a learned residual reduced prediction MSE by 25.5% on validation and 24.6% on held-out states at an exploratory higher update rate. Yet the analytic and learned-self utility controllers both chose 0.5-times updates at all 500 control steps. Their final JEPA losses and linear probes were indistinguishable. Prediction improved; the demonstrated decision reduced to lowering an aggressive learning rate.

## Experiment setup

Executed September 23, 2026 on DGX Spark GB10. All arms restore the same 5,000-step CIFAR-100 AdamW checkpoint, EMA teacher, and optimizer moments. The image encoder has width 192 and six blocks; its predictor has width 96 and two blocks. Eight parameter groups are controlled. The official CIFAR-100 test split remains unopened.

Eight fixed image probes produce 256 projected activation values across the encoder and predictor. A Jacobian–vector product, or JVP, approximates how activations change under a small parameter update. Version two computes a separate JVP for each parameter group, allowing any candidate gate vector to form an action-specific first-order forecast.

A small residual self-model learns the difference between that forecast and actual post-update state from 500 transitions, including 103 randomized block-gate interventions. Two utility heads then score nine candidate actions using either analytic or learned-self forecasts, trained on separate counterfactual outcomes. In control, actions are selected before drawing the fresh utility-evaluation batch.

The initial 3e-5 continuation rate made the analytic forecast almost exact and the learned residual failed its prediction gate. An exploratory 3e-4 rate increased nonlinearity and passed. Three additional audit trajectories evaluate eighty states and six held-out actions each. Control runs last 500 steps; paired branches at steps 100, 300, and 499 measure effects after one, five, and twenty further updates.

## Results

At 3e-5, validation JVP/self prediction MSE was 3.80e-11/3.97e-11: the learned residual was 4.6% worse. At 3e-4 it was 1.24e-6/9.21e-7, a 25.5% reduction; held-out error fell 24.6%. Three extra audit trajectories showed 14.2%, 18.8%, and 21.0% reductions.

At the higher rate, AdamW final JEPA loss was 0.093969 versus 0.087218 for each utility controller. Linear-probe accuracy was 19.16% versus 19.42%. Effective rank was 18.43 versus 17.81 and feature standard deviation 0.512 versus 0.480. Representation-quality measures did not uniformly improve.

Both heads selected the half-strength action on all eighty utility evaluation states and every one of 500 control steps. Exact counterfactual scoring also preferred it on 67/80 states. A fixed 0.5-times schedule would therefore enact the same decisions; the improvement over 1.0-times AdamW cannot be credited to learned self-model access.

The paired half-strength versus full-strength validation-loss advantage averaged 0.00237 after one update, 0.00063 after five, and 0.000050 after twenty. One of three twenty-step checkpoints had a negative effect. The first pilot also failed to distinguish the self-aware condition from its no-state-change ablation.

## Discussion

The study establishes a bounded prediction capability without establishing improved control. An action-dependent state forecast is more informative than a single forecast for the identity action, but the utility task must offer real selective choices.

A future test needs a calibrated base rate, stronger fixed-rate controls, longer-horizon action differences, and multiple initializations. The current result does not justify expanding controller capacity.

## Limitations

The higher learning rate was selected after seeing the lower-rate result, and all trajectories share one pretrained initialization.

Final DGX model checkpoints remain outside this publication: the original remote machine was unreachable during packaging. The archive contains code, original reports, action logs, summaries, and audit records, not substitute smoke-test weights.

## What this suggests

- A learned self-model predicted changes more accurately, but both controllers chose the same half-strength update on every step.
- The study establishes a bounded prediction capability without establishing improved control. An action-dependent state forecast is more informative than a single forecast for the identity action, but the utility task must offer real selective choices.

## What remains unknown

- The higher learning rate was selected after seeing the lower-rate result, and all trajectories share one pretrained initialization.
- Final DGX model checkpoints remain outside this publication: the original remote machine was unreachable during packaging. The archive contains code, original reports, action logs, summaries, and audit records, not substitute smoke-test weights.

## Reading the main result

500 / 500 — Half-strength choices by both controllers

The identical outcomes follow from identical action choices. A prediction gain should not be presented as a control gain.

One pretrained CIFAR-100 initialization; the useful prediction result used an exploratory higher learning rate. Production weights were not locally retrievable.

| Controller | Held-out JEPA loss ↓ | Linear-probe accuracy (%) ↑ |
| --- | ---: | ---: |
| AdamW | 0.093969 | 19.16 |
| Analytic JVP | 0.087218 | 19.42 |
| Learned self | 0.087218 | 19.42 |

Single-initialization, exploratory 3e-4 continuation. JEPA loss is in the experiment loss units; linear-probe accuracy is percent.

## Checkpoint availability

Production DGX checkpoints were not available for retrieval at publication. This download contains actual code, saved metrics, and action/audit records; it contains no trained production weights or replacement smoke-test checkpoints.

## References

- [Original reports, protocols, and saved measurements](/downloads/predicting-self-change-without-better-decisions/evidence.zip)