Generate, Filter, Control, Replay: A Comprehensive Survey of Rollout Strategies for LLM Reinforcement Learning

Rohan Surana, Gagan Mundada, Xunyi Jiang, Chuhan Wang, Zhenwei Tang, Difan Jiao, Zihan Huang, Yuxin Xiong, Junda Wu, Sheldon Yu, Xintong Li, Raghav Jain, Nikki Lijing Kuang, Sizhe Zhou, Bowen Jin, Zhendong Chu, Tong Yu, Ryan A. Rossi, Kuan-Hao Huang, Jingbo Shang, Jiawei Han, Julian McAuley
April, 2026
PDF
Type
Journal article
Publication
Transactions on Machine Learning Research (TMLR), 2026

This survey organizes rollout design for language-model reinforcement learning into generation, filtering, control, and replay. It connects design choices to reliability, coverage, and computational cost, with examples across reasoning and agent tasks.

Reinforcement Learning Large Language Models
Xunyi Jiang
Xunyi Jiang
PhD in CS @ 🇺🇸NYU
MS in CS @ 🇺🇸UCSD
BS in DS @ 🇨🇳SUSTech

I am interested in LLM Evaluation, Generative Engine Optimization, Human-AI collaboration, AI for creativity🎻.