ByteDance's Open Source Seed-OSS-36B Model Supports 512K Token Context

TikTok is once again in the news after the White House joined the popular social media platform, but its parent company ByteDance, a major Chinese tech firm, also had a surprise announcement.

The company's AI research unit, the Seed Team, today launched Seed-OSS-36B on the AI code repository Hugging Face.

Seed-OSS-36B is a new series of open-source large language models (LLMs) built for sophisticated reasoning and developer-friendly use, featuring a longer token context than many competing U.S.-developed models—including leading ones from OpenAI and Anthropic.

The collection includes three primary variants:

• Seed-OSS-36B-Base with synthetic data
• Seed-OSS-36B-Base without synthetic data
• Seed-OSS-36B-Instruct

By releasing both synthetic and non-synthetic versions of the Seed-OSS-36B-Base model, the Seed Team aimed to balance real-world performance with research adaptability.

The synthetic-data version, trained with supplementary instruction data, achieves stronger results on established benchmarks and is intended as a higher-performing, general-purpose model.

In contrast, the non-synthetic model removes these enhancements, providing a clearer base that reduces potential bias from synthetic instruction data.

By supplying both variations, the team offers practical users improved outcomes while giving researchers an unbiased baseline for studying post-training techniques.

Meanwhile, the Seed-OSS-36B-Instruct model is post-trained using instruction data, focusing on task execution and instruction adherence instead of acting solely as a base model.

All three models are licensed under Apache-2.0, allowing free utilization, modification, and sharing by enterprise researchers and developers.

This means they can be integrated into commercial applications, whether for internal operations or customer-facing services, without ByteDance charging licensing or API fees.

This follows the mid-2025 trend of Chinese companies launching advanced open-source models, while OpenAI works to keep pace with its own recently released open-source gpt-oss duet.

The Seed Team designed Seed-OSS for global use, highlighting its adaptability in reasoning, task-oriented functions, and multilingual environments.

Established in 2023, the Seed Team has focused on creating foundational models suitable for both research and practical applications.

Design and core features

The structure of Seed-OSS-36B incorporates recognized design elements such as causal language modeling, grouped query attention, SwiGLU activation, RMSNorm, and RoPE positional encoding.

Each model contains 36 billion parameters distributed across 64 layers and supports a vocabulary of 155,000 tokens.

A signature feature is its inherent long-context capacity, supporting up to 512,000 tokens for processing lengthy documents and logical sequences without degradation.

That is twice the capacity of OpenAI's new GPT-5 family and approximates the length of about 1,600 pages of text—roughly the size of the Christian Bible.

Another standout trait is the thinking budget, which allows developers to define the amount of reasoning the model applies before generating an answer.

A similar mechanism appears in other recent open-source releases, including Nvidia's Nemotron-Nano-9B-v2, also accessible via Hugging Face.

In practical terms, this enables teams to calibrate performance based on task intricacy and deployment efficiency needs.

Budget values are advised in multiples of 512 tokens, with 0 enabling a direct response mode.

Competitive performance on third-party benchmarks

Published benchmark results place Seed-OSS-36B among the top-performing large open-source models. The Instruct version, especially, achieves cutting-edge results across several domains.

• Math and reasoning: Seed-OSS-36B-Instruct scores 91.7% on AIME24 and 65 on BeyondAIME, each representing open-source state-of-the-art (SOTA) performance.
• Coding: On LiveCodeBench v6, the Instruct model attains 67.4, another SOTA mark.
• Long-context capability: On RULER at 128K context length, it reaches 94.6, the highest open-source result reported.
• Base model performance: The synthetic-data Base variant scores 65.1 on MMLU-Pro and 81.7 on MATH—both leading results in their categories.

The non-synthetic Base model, although slightly trailing in several metrics, remains competitive on its own.

It outperforms the synthetic version on GPQA-D, supplying researchers with a cleaner, instruction-neutral baseline for testing.

For businesses evaluating open alternatives, these outcomes indicate that Seed-OSS holds strong promise for math-intensive, coding, and long-context applications while preserving flexibility for research scenarios.

Access and deployment

Beyond performance, the Seed Team emphasizes developer accessibility. The models are deployable via Hugging Face Transformers, with quantization in 4-bit and 8-bit formats to minimize memory usage.

They also integrate with vLLM for scalable serving, complete with setup examples and API server guidelines.

To further simplify adoption, the team supplies scripts for inference, prompt customization, and tool integration.

For technical leads managing small teams or operating under limited budgets, these resources help make experimenting with 36-billion-parameter models more feasible.

Licensing and considerations for enterprise decision-makers

Available under Apache-2.0, these models can be adopted without restrictive licensing—a significant advantage for teams weighing legal and operational factors.

For leaders assessing the open-source ecosystem, this release highlights three key points:

• Top-tier benchmark results in math, coding, and long-context reasoning.
• A balance between high-performance synthetic-trained models and unbiased research baselines.
• Accessibility features that reduce operational complexity for streamlined engineering units.

By combining high performance and adaptable deployment under an open license, ByteDance's Seed Team has broadened the options available to companies, researchers, and developers.