A paper posted to arXiv this week, cataloged as 2608.21345, asks a question most teams running LLM-based critique loops have guessed at but rarely tested directly: how should a fixed compute budget be split between generating an answer and scrutinizing it afterward. The researchers built self-refinement pipelines — generate, critique, refine — and varied the resource split between the generation and critique steps to see which allocation produced the best final output.

The setup matters because self-refinement has become a default pattern in agentic systems: a model drafts something, a second pass — sometimes the same model, sometimes a dedicated critic call — evaluates it, and a third pass folds the feedback back in. Most implementations treat generation and critique as equal-weight steps, or do not think about the split at all; they call the model twice with roughly the same token budget and sampling settings and hope the loop converges. This study treats that split as an open variable rather than an assumption.

According to arXiv, the paper's central finding is that there is no single optimal ratio between generation and critique compute — the best split shifts depending on the task. That is a useful corrective for anyone who assumed one good allocation would generalize across use cases.

What the study actually tested

The authors frame generation and critique as competing draws on the same compute pool rather than as fixed, sequential costs. Instead of asking whether self-refinement helps — which prior work has already answered in both directions — they ask a narrower and more actionable question: given a fixed total budget, what fraction should be spent producing the first draft versus critiquing it before the next pass. They test this across task types and report that the answer changes with the nature of the task: some benefit from pushing more compute into critique, others get more value from investing further in generation. The paper proposes concrete proportions rather than a single universal rule.

Why equal splits quietly waste compute

Most production reflect-and-refine loops do not make this trade-off deliberately — they inherit it by accident from how the pipeline was scaffolded. A common pattern is to call the same model with the same context length and sampling parameters for both the draft and the critique, which implicitly assumes the two steps deserve equal investment. That assumption is rarely tested against an actual quality metric. If critique is under-resourced relative to what a task needs, the refine step ends up polishing errors the critic never caught. If critique is over-resourced, teams pay for a scrutiny pass that returns diminishing marginal quality per token spent, while the draft itself would have benefited more from that budget.

Practical implications for builders

• Treat the generation and critique split as a tunable hyperparameter per task category, not a fixed default inherited from a tutorial or template.
• Run a compute-allocation sweep on your own evaluation set before locking in a pipeline — the paper's finding that optimal ratios vary by task means a ratio tuned for code review will not necessarily transfer to summarization or planning.
• Measure marginal quality gain per additional critique token separately from marginal gain per additional generation token, so you can tell which side of the loop is actually starved.
• If the critic is a separate, cheaper model, the cost calculus changes — a heavier critique pass can be worth it even where it would not be worth it using the same model for both roles.

AiiN's takeaway

For anyone building agentic systems with a reflect-and-refine loop — whether the critic is a separate call to Claude, GPT, or Gemini, or the same model prompted twice — this argues for treating compute allocation as an empirical setting tuned per task, not a structural choice made once and forgotten. In our estimation, the more immediately useful part of the paper for practitioners is not the specific ratios it reports but the method: benchmark your own critique step's marginal value before assuming that adding more of it, or less of it, is the fix for a self-refinement pipeline that is not earning its extra latency.