Abstract
LLM-based test generation is often framed as prompt design, but its operational core is sequential budget allocation: after each execution, a system must decide whether boundary generation, fuzzing, LLM synthesis, oracle refinement, repair-context retesting, or stopping is worth the next unit of budget. We present TestGenRL , an MDP-guided framework for cost-aware LLM test generation: it records evidence, masks infeasible or unaffordable actions, rewards failure discovery, coverage, uncertainty reduction, and repair-relevant evidence against cost, and trains an action-masked Double-DQN on the logged transitions. Because it is trained offline, a vanilla learner over-estimates actions under-represented in the buffer; we train it conservatively, which in our held-out evaluation removes this over-exploration while preserving regime-dependent adaptation. On a 74-subject Python benchmark (Gemini 2.5 Flash Lite caches) with a held-out test split, the value of planning depends on the workload regime. On the full-pilot split, Greedy, a contextual bandit (Bandit), and the conservative Double-DQN all recover the cheap boundary strategy, reaching 18.2% failure-found rate and 97.3% coverage. When duplicate penalties and action-family saturation make boundary tests less useful, the adaptive planners avoid Greedy’s over-commitment: Bandit and the Double-DQN reach 66.7% failure-found rate on stress subjects and 58.3%/55.6% on action-balanced subjects, versus Greedy’s 25.0% and 16.7%, by reallocating toward fuzzing. We deliver a concrete planning interface, and a calibration lesson, for when evidence changes which action is feasible, affordable, and useful. Keywords: LLM test generation, Reinforcement learning, Markov decision processes, Budgeted testing, Fuzzing, Test repair