Train a CLI Agent with GRPO using Unsloth and NeMo Gym

Why it matters

Reinforcement learning is the most expensive fix for a wrong action. Running it once shows what it needs — a verifiable reward and a fair before-and-after test — and what it can still get wrong.

What to practise

Start the LangGraph CLI resource server, load Nemotron-Nano-9B-v2 in BF16 with Unsloth and LoRA adapters, and write the reward function that posts each completion to the NeMo Gym /verify endpoint. Invalid JSON or a wrong command scores −1, otherwise the score scales with flag accuracy. Fill in num_generations, learning_rate and max_steps in the GRPOConfig, train, and read reward, frac_reward_zero_std and completions/mean_length rather than the loss. Then score the trained model on the same held-out rows as the base model and count the wrong_command, wrong_flags, invalid_output and cut_off outcomes, regressions included.

How it fits

This is the reinforcement-learning end of 2.6.3. Try a description, prompt or model change first. Read wrong_command apart from wrong_flags, as 2.6.1 separates a wrong selection from wrong arguments.