Falcon-H1

A hybrid-architecture model in which attention heads and Mamba-2 heads run in parallel within the same block, rather than being arranged as separate layers in which one type replaces another. This parallel arrangement allows the two mechanisms to operate simultaneously on the same input representation.