A task-conditioned hypernetwork transformer prototype that generates bounded per-layer deltas and uses residual adapters as the actual controllable pathway, with early evidence on synthetic digit-reasoning tasks and an explicit proxy-failure diagnostic. # MetaModel [](https://pytorch.org) [](https://www.python.org) [](LICENSE) [](https://github.com/IlumCI/Metamodel) [](https://arxiv.org) [](#phase-1-disclaimer) > ⚠️ **Disclaimer — this is Phase 1 of a multi-phase project.** The codebase represents the foundational scaffold only. All infrastructure (self-training, research agent, self-modification loop, self-critic) is built but untested under real autonomous conditions. The 340M model has not been trained. `sum_mod3` is stuck at 32.7% on the 4.92M baseline. Do not take the current results as representative of the system's eventual capability — they are a working proof-of-concept for the architecture, not a finished system. --- ## Theory ### The problem: one model, many tasks Standard models are trained for one purpose. To handle a new task, you either fine-tune (expensive, can forget old tasks) or maintain separate models (wasteful). The core idea here is different: **make a single model that reconfigures its own weights based on what it's being asked to do.** ### What hypernetworks do A hypernetwork is a neural network that outputs the weights of another network. Rather than having fixed parameters `W`, the model has a function `H(t)` that generates task-specific weights: ``` W_task = H(task_description) y = f(x, W_task) ``` This is powerful because the same base network can solve fundamentally different tasks — not by learning a shared representation, but by generating different weight configurations per task. The hypernetwork learns *how* to reconfigure, not *what* to store. ### Why weight deltas instead of full weights Generating full weight matrices from scratch is expensive and hard to train. Instead, the hypernetwork generates **deltas** (`ΔW`) that modify a frozen base model: ``` W_final = W_base + α · ΔW ``` The base model knows how to process sequences (via self-attention and FFN). The hypernetwork only needs to learn *how to nudge* those weights toward the right computation for each task. This keeps the model stable — the base stays general — while the deltas provide specialization. The scalar `α` (learned per layer) controls how strongly to apply each delta. This is the **Scope Predictor**: it decides how task-specific a given layer needs to be. ### The bottleneck trick (TaskAdapter) Naive delta injection tends to either overfit (deltas too large) or have no effect (deltas too small). The **TaskAdapter** bridges this with a low-rank bottleneck: ``` ΔW = U · V^T where U ∈ R^{H×r}, V ∈ R^{H×r}, r << H ``` The hypernetwork generates `U` and `V` (rank-r matrices), and their product produces the delta. This constrains the delta to a low-dimensional subspace — enough to meaningfully modify behavior without destabilizing the base. It's the same principle as LoRA, but generated dynamically at inference time, conditioned on the task description rather than on a static adapter. ### Closed-loop self-training (the AI-led research lab) Training the hypernet doesn't end when loss converges. The system monitors for **proxy shortcuts**: the hypernet learns a correlation that works in training but doesn't implement the actual task. For example, `sum_mod3` learned to correlate with digit magnitude (r=-0.43) instead of actual modulo-3 arithmetic. When the SelfCritic detects a proxy, it: 1. Quarantines the proxy variant 2. Triggers the TaskGenerator to propose a harder variant of the same task 3. Routes the variant through MetaAdapterTrainer for fast fine-tuning 4. Merges the result back into the main training loop The loop is autonomous: human sets strategy, the agent executes tactics, metrics drive routing decisions. ### What this achieves vs. standard fine-tuning | | Standard fine-tuning | This approach | |---|---|---| | Per-task weights | Separate model or adapter per task | Generated on-the-fly from task description | | catastrophic forgetting | Risk when fine-tuning base | Base is frozen; deltas are additive | | Zero-shot new tasks | Not possible | Hypernet reads a new task description → generates new deltas | | Self-correction | Manual review | Autonomous SelfCritic loop | | Architecture growth | Full retrain for new tasks | Register new task type, expand embedding | --- ## Quick start ```bash # Interactive REPL python3 main.py # Single prediction python3 main.py --predict 42 fib_check # Self-correction loop (retry until correct) python3 main.py --retry 1234 digit_sum # Full benchmark on all 10 tasks python3 main.py --benchmark # Train the model python3 main.py --train --epochs 50 --batch-size 64 ``` --- ## Architecture ### Data flow ```mermaid flowchart LR A["task description\n\"sum of digits\""] --> B["HyperNetworkV3i\nLSTM"] B --> C["weight deltas\nΔW per layer"] C --> D["Base Transformer\nfrozen"] A --> E["digit tokens\n1 2 3 4"] E --> D D --> F["PredictionHead\nper task"] F --> G["class prediction"] ``` ### Component interaction ```mermaid flowchart TD subgraph INPUT T["task description"]:::box D["digit input"]:::box K["task ID (0-9)"]:::box end subgraph HYPERNET["HyperNetworkV3i"] ENC["TaskEncoder\nchar-level LSTM"]:::box EMB["TaskEmbed\n128D → 256D"]:::box LSTM["4-layer LSTM\nper-layer deltas"]:::box end subgraph ADAPTER["TaskAdapter (rank r)"] DOWN["H → r down-proj"]:::box MID["SiLU + Dropout"]:::box UP["r → H up-proj"]:::box G["gate g ∈ [0,1]"]:::box end subgraph BASE["Base Transformer"] L1["LayerNorm + RoPE"]:::box ATT["Multi-head\nSelf-Attention"]:::box FFN["FFN"]:::box end T --> ENC --> EMB --> LSTM LSTM -->|ΔW_attn_q| ATT LSTM -->|ΔW_attn_k| ATT LSTM -->|ΔW_ffn| FFN LSTM -->|g| G K --> EMB D --> L1 ATT -. "delta\ninjection" .-> ADAPTER FFN -. "delta\ninjection" .-> ADAPTER ADAPTER -->|g · ΔW| ATT ADAPTER -->|g · ΔW| FFN style HYPERNET fill:#1a1a2e,stroke:#7b68ee,color:#fff style ADAPTER fill:#1a2a1a,stroke:#3cb371,color:#fff style BASE fill:#1a1a2e,stroke:#4a9eff,color:#fff style INPUT fill:#2a1a2e,stroke:#ff7b54,color:#fff classDef box fill:#2a2a4a,stroke:#aaa,color:#fff,rx:4 ``` ### Self-training loop ```mermaid flowchart LR E["Training Epoch"] --> P["PerformanceMonitor"] P --> DIFF["DifficultyEstimator"] DIFF --> CURR["CurriculumManager"] subgraph LOOP["SelfCritic loop"] SC["SelfCritic.judge()\nproxy detection"]:::critic TG["TaskGenerator\nnew variant"]:::gen MA["MetaAdapterTrainer\nfast fine-tune"]:::train end CURR -. "difficulty\nthreshold" .-> SC P -. "accuracy\nmetrics" .-> SC SC -->|quarantine proxy| MA SC -->|new harder variant| TG TG -->|variant spec| MA MA -->|merged weights| P style LOOP fill:#0a1a0a,stroke:#3cb371,color:#fff classDef critic fill:#3a1a0a,stroke:#ff7b54,color:#fff classDef gen fill:#1a2a3a,stroke:#4a9eff,color:#fff classDef train fill:#2a1a3a,stroke:#9b59b6,color:#fff ``` ### Key numbers (small model, verified) | Component | Config | Params | |---|---|---| | Base Transformer | 8 layers, 256 hidden, 4 heads, FFN=512 | ~1M | | HyperNetworkV3i | 4-layer LSTM, 256D task embedding | ~0.4M | | TaskAdapter | bottleneck rank 8, 3 injection pts/layer | ~2.5M | | PredictionHeads | 10 tasks × per-class heads | ~0.5M | | **Total (small)** | | **4.92M** | Full model (planned): ~340M — same arch at scale. --- ## Training ### Run the training loop ```bash # On-the-fly generation (no pre-processed data needed) cd training python3 train_v6.py --epochs 50 --batch-size 64 --lr 3e-4 # With pre-generated Arrow data (10-50x faster loading) python3 train_v6.py --epochs 50 --batch-size 64 --arrow-dir ../data/arrow # Generate the Arrow data first cd .. python3 vectorize_dataset.py --split all ``` ### Generate dataset ```bash python3 vectorize_dataset.py # both train + val python3 vectorize_dataset.py --split train # train only python3 vectorize_dataset.py --dry-run # show what would be generated ``` ### Training configuration Edit `TrainConfig` in `training/train_v6.py` (line ~580): | Parameter | Default | Description | |-----------|---------|-------------| | `epochs` | 50 | Training epochs | | `batch_size` | 64 | Batch size | | `lr` | 3e-4 | Learning rate | | `weight_decay` | 1e-2 | AdamW weight decay | | `num_train` | 12000 | Training samples | | `num_val` | 1200 | Validation samples | | `hidden_size` | 256 | Transformer hidden dim | | `num_layers` | 8 | Transformer layers | | `hnet_dim` | 256 | Hypernetwork LSTM size | | `adapter_rank` | 8 | TaskAdapter bottleneck rank | | `arrow_dir` | None | Use Arrow loader if set | ### Monitor training Training writes to `experiments_v6/training_history.json` each epoch: ```bash python3 -c " import json with open('experiments_v6/training_history.json') as f: h = json.load(f) e = h[-1] print(f'Epoch {e[\"epoch\"]}: val={e[\"val_acc\"]:.1%} loss={e[\"val_loss\"]:.3f}') print('Task accuracies:', json.dumps(e['task_acc'], indent=2)) " ``` --- ## Results (small model, 4.92M params) 28 epochs on RTX 3050 (~1.4h wall time): | Task | Accuracy | Status | |------|----------|--------| | ascending | 100.0% | solved | | digit_sum | 96.4% | solved | | reverse | 100.0% | solved | | fib_check | 100.0% | solved | | last_digit | 100.0% | solved | | mode_digit | 86.4% | near-solved | | count_odd | 96.4% | solved | | all_unique | 85.5% | near-solved | | min_digit | 100.0% | solved | | sum_mod3 | 32.7% | **stuck** — proxy shortcut, not modulo | **Overall:** 57.2% → 89.7% val accuracy (+32.6pp over 28 epochs) ### sum_mod3 is the key failure The hypernet learned a proxy that correlates with `digit_sum` (r=-0.43) but doesn't implement actual modulo-3. Every variant detected as a proxy by SelfCritic is correctly flagged — the infrastructure works. The issue is that with the current 4.92M model, there's insufficient capacity to learn the actual modulo operation, and the proxy provides a local minimum. **Fix path:** Scale to the 340M model, which has enough capacity to learn `sum_mod3` properly. The proxy-detection infrastructure (`self_training/self_critic.py`) will catch it if it recurs. --- ## Project structure ```mermaid flowchart RL subgraph ROOT["meta_transformer/"] subgraph CORE["core"] MAIN["main.py"]:::file RA["research_agent.py"]:::file VD["vectorize_dataset.py"]:::file DL["data_loader_arrow.py"]:::file INIT["__init__.py"]:::file end subgraph MODELS["models/"] MTV["meta_transformer_v3.py"]:::file end subgraph TRAIN["training/"] TV["train_v6.py"]:::file end subgraph ST["self_training/"] MST["meta_self_trainer.py"]:::critic SC["self_critic.py"]:::critic CM["curriculum_manager.py"]:::gen TG["task_generator.py"]:::gen DE["difficulty_estimator.py"]:::gen PM["performance_monitor.py"]:::gen MA["meta_adapter_trainer.py"]:::train RS["research_state.py"]:::gen end subgraph INF["inference/"] SML["self_modification_loop_v3.py"]:::file end subgraph DATA["data/arrow/"] TR["train/"]:::folder VL["val/"]:::folder SI["zd21_SciInstruct/"]:::folder end subgraph EXP["experiments_v6/"] BM["best_model.pt"]:::bin CK["checkpoint_latest.pt"]:::bin TH["training_history.json"]:::file TL["training_log.txt"]:::file end end MAIN --> MTV MAIN --> TV RA --> ST TV --> DL VD --> DL ST --> MTV style CORE fill:#0f1a2e,stroke:#4a9eff,color:#fff style MODELS fill:#1a0f2e,stroke:#9b59b6,color:#fff style TRAIN fill:#0f1a0f,stroke:#3cb371,color:#fff style ST fill:#1f1a0a,stroke:#f39c12,color:#fff style INF fill:#1a1a1a,stroke:#aaa,color:#fff style DATA fill:#0f0f1a,stroke:#aaa,color:#fff style EXP fill:#0f0f0f,stroke:#aaa,color:#fff classDef file fill:#1a2a4a,stroke:#4a9eff,color:#fff,rx:3 classDef folder fill:#1a2a1a,stroke:#3cb371,color:#fff,rx:3 classDef bin fill:#2a1a1a,stroke:#e74c3c,color:#fff,rx:3 classDef critic fill:#3a1a0a,stroke:#ff7b54,color:#fff,rx:3 classDef gen fill:#0a1a2a,stroke:#4a9eff,color:#fff,rx:3 classDef train fill:#1a0a1a,stroke:#9b59b6,color:#fff,rx:3 ``` Legend: `self_training/` modules are colored by role — 🟠 critic, 🔵 generator, 🟣 trainer, 🟢 core. --- ## Self-training loop The architecture supports a closed-loop self-improvement cycle: ```mermaid flowchart TD subgraph TRAIN["Training Loop"] EPO["Training Epoch"] --> PM["PerformanceMonitor"] PM --> DE["DifficultyEstimator"] DE --> CM["CurriculumManager"] CM --> SC["SelfCritic.judge()"] PM -. "accuracy\nmetrics" .-> SC end subgraph CRITIC["Proxy Detection"] SC -. "quarantine\nproxy" .-> Q["Quarantined"] SC -. "new harder\nvariant" .-> TG["TaskGenerator"] SC -. "verdict" .-> MS["MetaSelfTrainer"] end subgraph ADAPTER["Adaptation"] TG --> MA["MetaAdapterTrainer\nfast fine-tune"] MA --> MW["Merged\nWeights"] MW --> PM end MS -->|"continue / stop\n/ quarantine"| EPO style TRAIN fill:#0d1f0d,stroke:#3cb371,color:#fff style CRITIC fill:#1f0d0d,stroke:#ff7b54,color:#fff style ADAPTER fill:#0d0d1f,stroke:#4a9eff,color:#fff ``` Run the self-training harness: ```bash python3 research_agent.py --mode agent ``` Or drive it programmatically via `self_training/meta_self_trainer.py`. --- ## Related work This project builds on several lines of prior research: | Paper | arXiv | Key idea | |-------|-------|----------| | **HyperNetworks** (Ha et al., 2016) | [1609.09106](https://arxiv.org/abs/1609.09106) | Dynamic weight generation via hypernetwork — the foundational concept | | **Attention is All You Need** (Vaswani et al., 2017) | [1706.03762](https://arxiv.org/abs/1706.03762) | Transformer architecture with multi-head self-attention and positional encoding | | **FiLM: Feature-wise Linear Modulation** (Perez et al., 2017) | [1709.07871](https://arxiv.org/abs/1709.07871) | Conditioning via feature-wise affine transformation — similar to TaskAdapter gating | | **ReZero** (Bachlechner et al., 2020) | [2002.05566](https://arxiv.org/abs/2002.05566) | Gated residual connections for fast convergence and stability | | **LoRA: Low-Rank Adaptation** (Hu et al., 2021) | [2106.09685](https://arxiv.org/abs/2106.09685) | Low-rank weight decomposition for efficient fine-tuning | | **AdapterHub** (Rücklé et al., 2020) | [2007.07779](https://arxiv.org/abs/2007.07779) | Adapter-based transfer learning for NLP | | **Meta-Learning with Hypernetworks** (Wichrowska et al., 2017) | [1703.04561](https://arxiv.org/abs/1703.04561) | Learned optimizers as hypernetworks | | **Modular Meta-Learning** (Alet et al., 2018) | [1806.10140](https://arxiv.org/abs/1806.10140) | Compositional task-specific modules | | **Task-Agnostic vs Task-Specific** (Jospe et al., 2020) | [2009.01758](https://arxiv.org/abs/2009.01758) | Analysis of hypernetwork approaches to MAML | --- ## What's next - [ ] **Train the 340M model** — the architecture is ready, just needs GPU time (~6-12h on RTX 3050) - [ ] **Fix sum_mod3** at scale — 340M should have enough capacity to learn modulo-3 without proxy - [ ] **Run full self-training loop** — current self-training modules are built but haven't been exercised on real training data - [ ] **Verify proxy detection** — SelfCritic infrastructure exists but needs real difficulty-scaled variants to validate - [ ] **GSM8K evaluation** — `datasets/gsm8k/` data exists; evaluate transfer learning --- ## Background The core hypothesis: a hypernetwork that reads task descriptions and generates weight deltas can make a single model adapt to multiple tasks without fine-tuning. The architecture went through several iterations: - **v1/v2:** Frozen base + hypernet → causality-inert deltas (deltas too small to change logits meaningfully) - **v3/v4:** Joint base+hypernet training → marginal improvement - **v5:** TaskAdapter bottleneck injection → significant improvement - **v6 (current):** HyperNetworkV3i + TaskAdapter + PredictionHeads → 89.7% val accuracy The stuck `sum_mod3` task at the 4.92M scale is the primary remaining challenge. The self-training infrastructure was built specifically to detect and route around such proxy shortcuts. Source: https://github.com/IlumCI/Metamodel