DeepSeek-R1

DeepSeek-R1 is a family of open-source large language models (LLMs) developed by the Chinese artificial intelligence company DeepSeek. Released in January 2025, the models are specifically designed to excel at complex reasoning tasks, including mathematics, coding, and logical deduction. The flagship model, DeepSeek-R1, contains 671 billion total parameters with 37 billion active parameters per token, employing a Mixture-of-Experts (MoE) architecture that activates only a subset of its parameters for each input. The model was released alongside a technical report and made available under the MIT License, allowing free distillation and commercialization.

The release of DeepSeek-R1 marked a significant moment in the development of AI reasoning models. It was the first open research to demonstrate that advanced reasoning capabilities in LLMs could be incentivized purely through reinforcement learning (RL), without relying on supervised fine-tuning (SFT) on human-annotated reasoning trajectories. This approach contrasted sharply with prevailing methods that depended heavily on extensive human-labelled data to teach models how to reason step by step.

Architecture

DeepSeek-R1 is built upon the DeepSeek-V3 Base model and utilizes a Mixture-of-Experts (MoE) framework. The architecture incorporates Multi-head Latent Attention (MLA) and a specialized DeepSeekMoE structure. It also includes built-in multi-token prediction (MTP) for speculative decoding, which improves inference efficiency. The model supports a context length of 128,000 tokens, enabling it to process and reason over long documents and complex problem statements.

The MoE design allows DeepSeek-R1 to maintain a large total parameter count while keeping computational costs manageable during inference. Only 37 billion parameters are activated for any given token, which is a fraction of the total 671 billion. This architecture enables the model to deliver high performance on reasoning-intensive tasks without the prohibitive inference costs typically associated with dense models of comparable scale.

Training Methodology

The training pipeline for DeepSeek-R1 represents a departure from conventional LLM development. The researchers first created DeepSeek-R1-Zero, a model trained via large-scale reinforcement learning directly on the base model, bypassing the supervised fine-tuning stage entirely. The reward signal was based solely on the correctness of final answers, without imposing any constraints on the reasoning process itself.

Through this pure RL training, DeepSeek-R1-Zero spontaneously developed sophisticated reasoning behaviors, including self-verification, reflection, and the exploration of alternative solution paths. The model exhibited a tendency to generate longer responses that incorporated verification steps and backtracking when encountering dead ends. This emergent behavior demonstrated that reasoning capabilities could arise from RL without explicit human instruction on how to reason.

However, DeepSeek-R1-Zero suffered from several practical issues, including poor readability, endless repetition, and language mixing, where the model would occasionally switch between English and Chinese within a single chain of thought. To address these problems, the researchers introduced DeepSeek-R1, which incorporates a small amount of cold-start data before RL training. The final pipeline involves two RL stages aimed at discovering improved reasoning patterns and aligning with human preferences, along with two SFT stages that serve as seeds for the model's reasoning and non-reasoning capabilities.

Performance and Benchmarks

DeepSeek-R1 achieves performance comparable to OpenAI's o1 model across a range of reasoning benchmarks. On the AIME 2024 mathematics competition, DeepSeek-R1 attained a pass@1 score of 79.8 percent, while OpenAI's o1-1217 scored 79.2 percent. On MATH-500, the model achieved 97.3 percent pass@1, slightly surpassing o1's 96.4 percent. For coding tasks, DeepSeek-R1 reached a Codeforces rating of 2029, outperforming o1's 1891 rating. On the GPQA Diamond benchmark, which tests graduate-level scientific reasoning, DeepSeek-R1 scored 71.5 percent pass@1 compared to o1's 75.7 percent.

Beyond the flagship model, DeepSeek released six distilled dense models based on the Qwen and Llama architectures. These smaller models, ranging from 1.5 billion to 70 billion parameters, were fine-tuned using reasoning data generated by DeepSeek-R1. The DeepSeek-R1-Distill-Qwen-32B model outperformed OpenAI's o1-mini across various benchmarks, establishing new state-of-the-art results for dense models. The 70-billion-parameter distilled model achieved an AIME 2024 pass@1 score of 70.0 percent, demonstrating that reasoning patterns from large models can be effectively transferred to smaller ones.

Open-Source Release and Distillation

DeepSeek-R1 was released under the MIT License, making it one of the most permissively licensed frontier reasoning models available. The open-source release included the full model weights for both DeepSeek-R1 and DeepSeek-R1-Zero, along with the six distilled models. The technical report was published on arXiv and later in the journal Nature, providing the research community with detailed insights into the training methodology and architecture.

The decision to open-source the model and its distilled variants had significant implications for the AI research community. Researchers could now study and build upon a reasoning model that rivaled proprietary systems, and developers could fine-tune the models for specialized applications. The MIT License explicitly permitted distillation and commercialization, removing legal barriers that often accompany the release of advanced AI models.

Impact and Reception

The release of DeepSeek-R1 generated considerable attention in both the AI research community and the broader technology industry. Its performance on par with OpenAI's o1 model, combined with its open-source nature, challenged assumptions about the necessity of massive proprietary investment for developing frontier reasoning capabilities. The model demonstrated that pure reinforcement learning could incentivize sophisticated reasoning behaviors without the need for extensive human-labelled reasoning data, a finding that has since influenced subsequent research in the field.

The distilled models released alongside DeepSeek-R1 also had a notable impact, particularly for researchers and developers with limited computational resources. The DeepSeek-R1-Distill-Qwen-32B model, for instance, achieved performance exceeding OpenAI's o1-mini on several benchmarks despite being substantially smaller and more efficient to run. This made advanced reasoning capabilities accessible to a wider range of users and applications.

The model's architecture and training methodology have been widely studied and replicated. The use of Group Relative Policy Optimization (GRPO) as the RL framework, which eliminates the need for a separate value network, has been adopted in other reasoning model projects. The concept of using pure RL to incentivize reasoning, bypassing SFT on human demonstrations, has become a prominent research direction.

References

DeepSeek API Docs. "DeepSeek-R1 Release." 

Nature. "DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning." 

SGLang Documentation. "DeepSeek-R1." 

Hugging Face. "README.md · qgallouedec/DeepSeek-R1 at main."