Agent

Recurrent Depth Model Implementation

Creator:

About this agent

This module implements the recurrent depth transformer model as described in
'Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach'.

The model consists of three main components:

  • Prelude: Embeds input tokens into latent space
  • Core/Recurrent: Iteratively processes the latent state
  • Coda: Decodes the final latent state into output probabilities

Requirements

PackageInstallation
torchpip install torch
logurupip install loguru
transformerstransformers

Agent Code

The main implementation code for this agent

Comments & Discussion

Scroll to load comments...

Tags

PyTorch
Deep Learning
Transformer
Recurrent Neural Network
Attention Mechanism
Natural Language Processing
Model Configuration
Neural Network Layers
Text Generation
Machine Learning
Neural Network Architecture
Torch.nn
Loguru
Dataclass
Multi-Query Attention
RMSNorm
SiLU Activation
State-Space Model
Rotary Positional Embeddings

Share

Tokenization

This item is not available for tokenization.

Loading recommendations...

Yuki

Your Marketplace Companion

Agent

Hey, I'm Yuki 👋

Ask me about specific products, customer support, or anything about the Swarms Marketplace.