Hard Neural Fabrics: Structured Success, Language Failure

In the parity experiment, eight width-8 cells used a shared width-32 update kernel, persistent width-8 state, four configuration bits per cell, and three stages of hard 2-by-2 switches. Training used sequence lengths 8–32; evaluation used lengths 16, 32, 64, and 128. The final confirmation used seeds 202, 303, and 404, 500 optimizer steps, batch size 128, and 2,048 deterministic evaluation sequences per seed and length. A width-15 GRU with 1,502 parameters was the control; the fabric training graph had 1,554 parameters and froze to 1,498 floating-point parameters plus 44 configuration bits.

The final routing estimator sampled a legal hard Gumbel switch configuration on each training forward. An earlier deterministic straight-through estimator was abandoned after its initial route sent the input away from the readout and supplied almost no useful topology gradient; its seed 101 was excluded from confirmation. The WikiText transfer used byte identities, sequence length 64, batch size 16, 1.024 million training bytes per seed, and three seeds 101, 202, and 303. Its eight width-64 cells shared a width-152 kernel and were compared with a two-layer width-64 GRU. Training and inference used Apple MPS. The published local artifacts record the exact configurations and summaries.

The WikiText result record reports Python 3.12.13 and PyTorch 2.13.0 on macOS MPS. Reproduction command from the repository root: `python3 projects/trainable-neural-fabric/primitive_wikitext.py --steps 1000 --batch-size 16 --evaluation-bytes 131072 --seeds 101 202 303 --models gru fabric --device mps --output artifacts/mac/2026-09-01-hard-neural-logic-fabric-wikitext/results.json`. The checksum-pinned WikiText-2 byte corpus must be available at the path accepted by the script.

The parity confirmation ran with Python 3.12 and PyTorch on CPU. Reproduce it from the repository root with `python3 projects/trainable-neural-fabric/running_parity.py --steps 500 --batch-size 128 --evaluation-examples 2048 --seeds 202 303 404 --models gru fabric --output artifacts/mac/2026-09-01-hard-neural-logic-fabric-parity/results.json`. The report documents the focused unittest command and the excluded estimator-diagnostic seed.

The parity result is stronger than a score alone: the route intervention shows the learned topology mattered. Yet the task can be solved with one routed cell, so it is a minimal stateful sanity check rather than evidence of composed circuit synthesis. The fabric did not establish that a learned substrate can be reused to compile another task.

WikiText changes the topology problem. The current one-microstep design requires input cell 0 to connect directly to readout cell 7; persistent state is not routed through a multi-cell computation. All three runs failed to discover that connection. This is consistent with a severe credit-assignment bottleneck, but the experiment does not distinguish whether the cause is the search estimator, architecture, optimization budget, or their interaction. A fixed-route ablation that first demonstrates useful byte prediction, followed by multi-step routed state, is the most informative next gate.

These are small experiments with three seeds per task and different task-specific model sizes and budgets. Running parity is synthetic and unusually simple. WikiText is a checksum-recorded byte-level proxy, not a standard subword benchmark or large language model. The eager executor is slow and no fused kernel was evaluated. No trained fabric checkpoints are present in the selected experiment directories; the results JSON is available, but it is not a weights file. The source code and tests permit inspection and rerunning when the dataset and runtime are available, but these records alone do not supply weights for direct inference reproduction.