NVIDIA Open Sources Polar Framework for Zero-Barrier AI Coding Agent Evolution via Reinforcement Learning

On May 28, the NVIDIA research team open-sourced Polar, a reinforcement learning training framework. Its core innovation lies in seamlessly integrating existing mainstream code agents—such as Codex, Claude Code, and Qwen Code—into GRPO (Generalized Relative Policy Optimization) reinforcement learning training without requiring any changes to the original code.

I. Industry Pain Points: The Barrier to Agent Reinforcement Learning

As code agents evolve from simple single-step tasks to complex, long-running processes—such as warehouse-level code modifications or OS interactions—developers increasingly rely on mature execution frameworks (Harness). Yet, integrating these complex frameworks into traditional reinforcement learning infrastructure presents significant challenges:

High Integration Cost: Traditional methods require rewriting code logic into standard environment interfaces such as env.init() and env.step(), a process that is extremely tedious.

Information Loss: During refactoring, critical details—such as tool calls, multi-turn dialogue context, or sub-agent collaboration logic—are often lost, preventing the model from receiving high-quality training signals.

II. Core Solution: Using the "Boundary" as the Training Entry Point

Polar eliminates the need to rewrite the execution framework. Instead, it treats the model API boundary as the training entry point.

Black-box Processing: Polar places a transparent proxy (Gateway) between the code execution framework and the model inference server. Regardless of whether the agent uses APIs from Anthropic, OpenAI, or Google, Polar seamlessly intercepts and forwards requests.

Trace Reconstruction: While forwarding, Polar records real-time key data—such as prompts, sampled tokens, and log probabilities—and reconstructs it into the "trace" data needed by the reinforcement learning trainer.

Efficient Asynchronous Architecture: The system employs a Rollout Server for scheduling and persistence, while Gateway Nodes manage lifecycle and resource recycling. By leveraging a preheated buffer (READY buffer) and parallel task processing, it effectively eliminates long-tail tasks that could block GPU training.

III. Performance Leap: Transforming Code Agents

Experimental data shows that Polar, when combined with GRPO training, yields significant performance gains:

SWE-Bench Verified Benchmark Test: Using the same Qwen3.5-4B base model, performance varies across different code frameworks:

Codex Framework: The pass@1 score jumps from 3.8% to 26.4%—a surge of 594.74%.

Claude Code Framework: from 29.8% to 34.6%.

Pi Framework: from 34.2% to 40.4%.

Extreme Efficiency: After introducing the prefix_merging strategy, training wall-clock time is shortened by about 5.39 times compared to the traditional per-request mode, and GPU utilization rises from 20.4% to 87.7%.

Industry Commentary

The open-sourcing of NVIDIA's Polar essentially builds a "highway" for AI agents to enter reinforcement learning training. It not only enables researchers to train efficiently using massive open-source code frameworks but also lowers the GPU computing barrier through system-level optimization.

With Polar's growing popularity, developers no longer need to worry about "how to adapt models to training frameworks." In the future, the evolution of AI coding agents will become more standardized and efficient. This marks a shift in AI agent training from manual lab tuning to large-scale, systematic engineering production.

Paper URL: https://arxiv.org/pdf/2605.24220