MambaX: Scaling Selective State Space Models for Ultra-long Sequences

Description: MambaX is a scalable model designed for processing ultra-long sequences by utilizing a combination of selective state space models, top-k selection, dynamic memory updates, and hierarchical multi-head attention. It efficiently handles sequence data by chunking, retaining salient features, and refining memory representations, with an optional decoder for generating outputs at the chunk level.