Granite 4.1 LLMs: How They’re Built

I’ve been following the Granite series for a while, and the 4.1 release from IBM’s Granite Team is genuinely interesting. Not because it’s another “bigger is better” play — these are dense, decoder-only models at 3B, 8B, and 30B parameters — but because of how they were built. The team published a detailed technical walkthrough on the Hugging Face Blog, and it’s refreshing to see a focus on data engineering and training strategy rather than just throwing compute at the problem.

The TL;DR is this: Granite 4.1 uses a multi-stage pre-training pipeline with ~15 trillion tokens, long-context extension up to 512K tokens, supervised fine-tuning on ~4.1M curated samples, and reinforcement learning via on-policy GRPO with DAPO loss (Yu et al., 2025). The 8B instruct model reportedly matches or surpasses the previous Granite 4.0-H-Small (a 32B-A9B MoE) despite being a simpler dense architecture with fewer parameters. That’s not nothing. And all models are released under Apache 2.0, which is always a good sign.

Architecture Choices

The architecture is standard but well-executed: decoder-only dense transformer with Grouped Query Attention (GQA), Rotary Position Embeddings (RoPE), SwiGLU activations, RMSNorm, and shared input/output embeddings. Nothing wild here, but that’s fine. The real magic is in the data.

| Component | 3B Dense | 8B Dense | 30B Dense |
|———–|———-|———-|———–|
| Embedding size | 2560 | 4096 | 4096 |
| Number of layers | 40 | 40 | 64 |
| Attention head size | 64 | 128 | 128 |
| Number of attention heads | 40 | 32 | 32 |
| Number of KV heads | 8 | 8 | 8 |
| MLP hidden size | 8192 | 12800 | 32768 |
| MLP activation | SwiGLU | SwiGLU | SwiGLU |
| Position embedding | RoPE | RoPE | RoPE |

All three sizes share the same training pipeline and data strategy. The differences are purely architectural dimensions.

Pre-Training: A Five-Phase Pipeline

The pre-training is where things get interesting. Granite 4.1 is trained from scratch on ~15 trillion tokens across five distinct phases. Phases 1–2 are foundational pre-training, phases 3–4 are mid-training with progressively higher-quality data annealing, and phase 5 introduces long-context training, extending the context window from 4K to 512K tokens. Each phase uses a different data mixture and learning-rate schedule, gradually shifting from broad web-scale data to more curated, domain-specific content.

Phase 1: General Pre-Training (10T tokens) — Establishes broad language understanding. Data mix: CommonCrawl ~59%, Code ~20%, Math ~7%, Technical ~10.5%, Multilingual ~2%, Domain Specific ~1.5%. Power learning rate schedule with warmup.

Phase 2: Math/Code Pre-Training (2T tokens) — Sharply increases code and math proportions. Data mix: Math ~35% (5x increase over Phase 1), Code ~30% (1.5x increase), CommonCrawl-HQ ~12%, Synthetic ~9%, Technical ~10%, Multilingual ~3%, Domain ~1%.

Phase 3: High-Quality Data Annealing (2T tokens) — Transition to mid-training with a balanced, high-quality mixture and exponential decay learning rate. Starts blending in chain-of-thought and synthetic instruction data. Data mix: CommonCrawl-HQ ~16.67%, Math ~16.67%, Code ~16.67%, Synthetic ~8.5%, Technical ~12.5%, Multilingual ~4.5%, Long Chain-of-Thought ~12.5%, Language Instructions ~7.5%, Code Instructions ~4.5%.

Phase 4: High-Quality Data Annealing — Refinement (0.5T tokens) — Linear learning rate decay to zero, focusing on the highest-quality data. Data mix: CommonCrawl-HQ ~40%, Code ~20%, Math ~20%, Long Chain-of-Thought ~6%, Code Instructions ~5%, Language Instructions ~9%.

Phase 5: Long Context Training (LCE) — Extends context window from 4K to 512K through staged extension: 32K, 128K, then 512K. The 512K stage uses 80% books + 20% code repository data (8B and 30B only). Exponential learning rate schedule starting at 1e-4, decaying to 0. After each LCE stage, they do a model merge to preserve short-context performance. RULER benchmark results for base models: granite-4.1-3b-base scored 75.0 at 32K, 66.6 at 64K, 58.0 at 128K; granite-4.1-8b-base scored 83.6, 79.1, 73.0 respectively.

Post-Training: SFT and RL

The post-training pipeline is equally rigorous. They curated ~4.1M supervised fine-tuning samples using an LLM-as-Judge framework to filter for quality. Then they applied a multi-stage reinforcement learning pipeline using on-policy GRPO with DAPO loss. This systematically strengthens performance in math, coding, instruction following, and general chat. The approach is similar to what DeepSeek and others have been doing, but the scale and data curation here feel more deliberate.

What I Think

IBM has been quietly doing solid work in the LLM space, and Granite 4.1 is a good example. The focus on data quality over quantity, the five-phase pre-training with progressive data annealing, and the careful long-context extension are all smart engineering choices. The fact that the 8B instruct model can match a 32B MoE model from the previous generation is impressive — it shows that architecture and data strategy matter more than raw parameter count.

That said, these are still small models. They won’t beat GPT-4 or Claude in general benchmarks, but that’s not the point. For enterprise use cases where you need something that runs on a single GPU, is open-source, and performs well on specific tasks like coding or math, Granite 4.1 is a strong contender. The Apache 2.0 license makes it easy to deploy and modify.

One thing I’d like to see more of is how the synthetic data was generated and validated. The paper mentions synthetic high-quality data in Phases 2 and 3, but the details are thin. Also, the RULER scores at 128K for the 3B model (58.0) suggest there’s still room for improvement in long-context performance for smaller models. But for most real-world applications, 32K or 64K is plenty.

Overall, Granite 4.1 is a well-engineered family of models that deserves attention, especially if you’re looking for open-weight alternatives to the usual suspects.