// TOPIC STREAM

2 posts tagged with "Attention"

Attention mechanisms and efficient alternatives.

posts: 2

Conceptual diagram representing token memory and comparison in an H3 state space layer

Research::Aug 19, 20267 min

H3: The Memory and Comparison Capabilities SSMs Missed in Language Modeling

Hungry Hungry Hippos decomposes the linguistic weaknesses of state space models into token recall and comparison, then explains how H3, FlashConv, and hybrid attention combinations address them.

AI Research Language Modeling SSM2 more topics

referenced

Sequence-model architecture where linear recurrence and local attention alternate

Research::Aug 19, 20267 min

Griffin's Compromise: Combining Gated Linear Recurrence with Local Attention

This article separates the evidence reported by the paper from the open questions of adoption, explaining how Hawk and Griffin combine recurrent-model efficiency with the Transformer's representational power.

AI Research RNN Attention1 more topics

referenced