Is a Fixed-Size Hidden State Memory, or Compression?
This article distinguishes fixed-size recurrent hidden states as memory devices versus information compression. It explains the role of state through S4, Mamba, RWKV, and Resona, while treating Qdrant and Statey separately as external-memory contexts.