Model Releases
Jamba 1.5: The New Hybrid MoE Standard for Long Context
AI21 Labs launches Jamba 1.5, combining Mamba and Transformer architectures for unprecedented speed and 256K context support in an open-weight release.
August 22, 2024
Model ReleaseJamba 1.5
Introduction
In the rapidly evolving landscape of large language models, AI21 Labs has officially released Jamba 1.5 on August 22, 2024, marking a significant milestone in hybrid architecture development. This release is not merely an incremental update but a fundamental shift towards efficiency, addressing the critical bottleneck of context window management in enterprise applications. For developers and AI engineers, the availability of open weights alongside a high-performance API means you can now deploy this model without the usual licensing constraints, fostering greater innovation in long-context workflows.
The industry has long struggled with the trade-off between model capacity and inference speed. Jamba 1.5 aims to solve this by integrating the state-space model (SSM) capabilities of Mamba with the attention mechanisms of Transformers. This hybrid approach allows the model to process vast amounts of data without the quadratic complexity associated with standard attention layers, making it the fastest long-context model available at the time of its release.
- Released Date: August 22, 2024
- Provider: AI21 Labs
- License: Open Weights Available
- Primary Focus: Long-context efficiency and speed
Key Features & Architecture
The core innovation of Jamba 1.5 lies in its unique Mamba-Transformer hybrid MoE (Mixture of Experts) architecture. Unlike pure Transformer models that scale quadratically with context length, Jamba 1.5 utilizes a sparse MoE structure where only a subset of parameters is active during inference. This design choice drastically reduces computational load while maintaining high-quality output generation.
The model boasts a massive total parameter count of 398 billion, with a highly optimized 94 billion active parameters per token. This sparse activation ensures that the model remains lightweight enough for deployment on edge devices or smaller clusters while retaining the intelligence of a much larger model. Additionally, the architecture supports a context window of 256K tokens, enabling the processing of entire books, hours of video transcripts, or massive codebases in a single pass.
- Total Parameters: 398B
- Active Parameters: 94B (per token)
- Context Window: 256K tokens
- Architecture: Mamba-Transformer Hybrid MoE
Performance & Benchmarks
Performance evaluations show that Jamba 1.5 outperforms its predecessor and direct competitors across standard reasoning and coding benchmarks. The integration of the Mamba state-space model contributes to faster inference speeds, particularly noticeable in long-context scenarios where traditional models often degrade in quality or latency. While specific latency numbers depend on hardware, the throughput improvements are significant enough to enable real-time agent workflows.