XSci

Primitives and Paths: Predicting What Post-Training Can Do from What the Base Model Already Has

Anqi Qu

Submitted by Anqi Qu

Published November 14, 2026 · Version v1, November 14, 2026 · DOI 10.66977/xsci.2611.0026

Machine Learning

Abstract

Does reinforcement learning teach language models new capabilities, or merely amplify ones acquired during pre-training? The literature offers evidence for both: pass@kk studies suggest RL reweights latent behaviours while some controlled composition studies demonstrate gains beyond simple amplification. We argue that this question has no answer independent of the capability state of the base model, and propose a two-dimensional state space for post-trainability. The first axis, effective full-solution support\emph{effective full-solution support}, measures whether successful trajectories can be sampled under a feasible exploration budget. The second axis, atomic primitive coverage\emph{atomic primitive coverage}, measures whether the constituent skills of a solution can be executed when the burden of composition is removed. Together, these dimensions motivate three regimes -- amplification, composition-limited, and acquisition-limited -- each predicting a distinct empirical signature and calling for a distinct intervention. We propose a prospective measurement protocol and investigate the framework using synthetic composition tasks across three small language models. We find that primitive mastery alone does not guarantee that RL can learn a composition. Checkpoints that execute every primitive reliably can fail to improve when complete solutions are never sampled during training, whereas modest composition supervision that establishes nonzero support enables substantial RL gains. The framework reframes mid-training as preparation for post-training: its role is to move models into states from which RL can operate effectively.

View PDF

Review reports

xPeerd (AI review)

November 14, 2026

AI Review (xPeerd — DAReview)

Summary

This manuscript, Primitives and Paths: Predicting What Post-Training Can Do from What the Base Model Already Has, proposes a two-dimensional framework for predicting when reinforcement learning (RL) can improve language-model reasoning: “effective full-solution support” and “atomic primitive coverage.” The paper argues that disagreement in the RL post-training literature can be reconciled by distinguishing amplification, composition-limited, and acquisition-limited regimes, then offers a measurement protocol and a synthetic experimental test across three small models. The central strengths are conceptual clarity, a plausible diagnostic reframing of RL post-trainability, and a controlled experimental design that isolates primitive availability from compositional accessibility. The paper’s main weakness is that its empirical validation is narrow relative to its conceptual ambition: the experiments use synthetic string operations, small models, and a single verifier-rich setup, while several key quantities and classification thresholds remain under-specified or non-principled. The manuscript is thoughtful and mostly coherent, and I do not see clear signs of fabrication; however, some claims are broader than the evidence currently warrants, especially when moving from toy composition tasks to general reasoning and mid-training strategy. The paper is promising, but it would benefit from stronger empirical stress tests, clearer operational criteria, and tighter delimitation of what has been shown versus what remains conjectural.

Potential Major Revisions

  1. Scope of evidence is too narrow for the breadth of the paper’s framing.

The abstract states that the framework addresses whether “reinforcement learning teach[es] language models new capabilities, or merely amplify ones acquired during pre-training” and claims that “the framework reframes mid-training as preparation for post-training” . In the conclusion, the paper further generalizes that “apparent contradictions in the RLVR literature can then be read as observations about different regions of this space” and that “outcome-based RL improved only checkpoints with some reachable successful trajectories” . These are intellectually appealing claims, but the experimental basis is a highly controlled synthetic task: “two operations on five-digit strings: F rotates the string left by one position, and G increments its first digit by three modulo ten,” with problems requiring “a sequence of 2-4 operations” and “a programmatic verifier” . The manuscript itself concedes limited generalisability: “our experiments use small language models and synthetic composition tasks with simple and known primitives and deterministic verification” .

The mismatch here is not fatal, but it is central. The paper’s framing repeatedly addresses broad reasoning post-training in mathematical, coding, and other domains, while the evidence demonstrates a much narrower proposition: in a toy compositional environment, sparse-reward RL depends strongly on whether successful trajectories are already reachable. That is useful, but the manuscript should more rigorously separate demonstrated results from extrapolated implications. I recommend rewriting the contribution statements to make the empirical claim narrower and more exact.

  1. Several central constructs remain under-operationalized despite the paper presenting itself as a diagnostic framework.

The manuscript claims to “develop a diagnostic framework for post-trainability” and asks what can be “measure[d] before post-training” to predict RL effectiveness . Yet the key regime assignment procedure depends on thresholds that the paper explicitly does not know how to choose: “Our classification also depends on thresholds τp and τc, which we do not yet offer a principled selection procedure for” . Similarly, the paper acknowledges that “failure to observe a successful trajectory at finite k does not establish that it is absent” and that held-out pass@k can “underestimate the reward accessibility encountered during RL training” .

This leaves the protocol conceptually interesting but not yet robust as a practical diagnostic instrument. The paper should either downgrade its methodological ambition or provide more concrete calibration guidance. For example, the current statement that “τp and τc… should be specified before post-training and varied in sensitivity analyses” is sensible but insufficient for reproducible deployment. As written, the framework risks becoming descriptive after the fact rather than genuinely predictive.

  1. The paper’s own results partly expose instability in the interpretation of “primitive coverage.”

The protocol defines atomic primitive coverage using the minimum isolated execution probability, “c(x) = min qjexec(x),” justified on the grounds that “a single unreliable required primitive can create an execution bottleneck” . However, the results show a notable tension: “these checkpoints retain near-zero accuracy on isolated G probes, despite successfully using G within some complete trajectories” . This is an important and interesting finding, but it also undermines the straightforward interpretation of isolated primitive probes as faithful measurements of underlying capability.

The paper acknowledges this problem under “Primitive coverage and composition,” stating that “isolated probes may fail to reveal skills available in compositional contexts” and that it does “not yet establish that measured primitive coverage adds predictive power beyond full-solution support” . That admission is appropriate, but its implications should be brought earlier and made more central. At present, the manuscript still presents primitive coverage as one of the two defining axes of the framework, even though the paper’s own evidence suggests that this axis may be fragile, probe-dependent, and possibly less predictive than advertised. This is a significant theoretical vulnerability and should be confronted directly.

  1. The causal interpretation of composition supervision enabling RL needs more careful qualification.

The paper states that “modest composition supervision that establishes nonzero support enables substantial RL gains” and that “composition-focused supervision increases effective full-solution support and enables larger subsequent RL gains” . The experimental pattern is consistent with that claim: for example, Qwen3-1.7B M2 has pre-RL pass@1/pass@64 of 0.000/0.000 and remains unchanged post-RL, whereas M2 +C (100) has 0.064/0.180 pre-RL and rises to 0.452/0.482 post-RL; M2 +C (300) starts much higher and rises to 0.988/1.000 . However, “enables” can sound stronger than the design warrants.

Composition supervision may do more than increase support. It may alter representations, output formatting priors, decomposition habits, or trajectory smoothness. The paper partly recognizes this by discussing “selection, sequence, and transition between available skills” under “compositional connectivity” , but the current exposition still leans toward a support-centric causal story. I recommend tempering the language to say the results are consistent with the hypothesis that increased reachable support is an important mediator, rather than proving that support alone is the mechanism.

  1. The theoretical quadrants may overstate separability between diagnostic categories.

Figure 1 and the surrounding text present amplification, composition-limited, acquisition-limited, and diagnostic mismatch regions . Yet the manuscript also notes that “the two dimensions are not necessarily independent,” that a complete successful solution generally implies the availability of relevant primitives, and that the space is asymmetric and continuous rather than sharply partitioned . This creates a tension between the clean quadrant metaphor and the more nuanced reality described in the prose.

The framework may still be useful as a heuristic, but the current presentation sometimes shifts between ontology and diagnostic convenience. The manuscript should sharpen whether these “regimes” are intended as real latent states, approximate explanatory clusters, or merely pragmatic reporting bins. As written, the conceptual status of the framework is somewhat ambiguous.

  1. The manuscript should engage more critically with possible alternative explanations and failure modes.

The “triage for measurement mismatch” section is one of the most valuable parts of the paper because it openly entertains “answer-first rationalisation, contamination artifacts, or reasoning that does not support the answer” . This is exactly the type of epistemic caution such a paper needs. But that caution is not fully integrated into the results section, where claims are comparatively more linear. Because the paper’s central variables are sampling-dependent and scaffold-dependent, the risk of overinterpreting probe behavior is nontrivial. The discussion should explicitly outline which empirical patterns would falsify the framework, not only which ones are consistent with it.

Potential Minor Revisions

  1. Title, abstract, and introduction could be tightened for precision.

The title, Primitives and Paths: Predicting What Post-Training Can Do from What the Base Model Already Has, is strong and memorable, but “predicting” may overpromise given the acknowledged absence of principled thresholds and limited generalisability . A slightly more cautious phrasing may better match the current evidence.

In the abstract, the phrase “each predicting a distinct empirical signature and calling for a distinct intervention” is somewhat too categorical given the admitted measurement ambiguity and continuity of the state space .

  1. Some wording should be softened where broad claims rest on synthetic evidence.

For instance, the conclusion states, “apparent contradictions in the RLVR literature can then be read as observations about different regions of this space” . “Can then be read” sounds stronger than the paper has earned empirically. Consider “may often be interpreted as” or “are consistent with the possibility that.”

Likewise, “The framework reframes mid-training as preparation for post-training” is rhetorically effective but broad; “suggests viewing some mid-training choices as preparation for post-training” would be safer.

  1. Clarify notation and operational decisions earlier.

The measurement section introduces several important quantities—pass@k, per-rollout success, false-success rate checks, atomic primitive coverage, handoff curve, and composition gap . These are useful, but the density may burden readers. A compact notation table would improve readability. Since the manuscript is methodological, small improvements in exposition would materially help.

  1. Table presentation needs interpretive support.

Table 1 is informative, but the “F / G.” column may not be immediately intuitive to all readers, and the text explanation could be more explicit. The results text says, “F / G is pre-RL execution accuracy on isolated single-operation prompts with held-out strings” , but a clearer caption or footnote would help. A visual plot of pre-RL support versus post-RL gain might also communicate the central argument more directly than the current table alone.

  1. Some references appear very recent and arXiv-heavy, which is expected in this area, but the manuscript should more clearly distinguish established findings from emerging claims.

The argument relies heavily on 2025–2026 preprints throughout the introduction and background . That is understandable for a fast-moving topic, but the prose occasionally treats those results as settled. A slightly more cautious literature review would strengthen scholarly balance.

  1. Typographic and formatting issues.

I did not detect many obvious typographical errors from the extracted text. However, there are some formatting irregularities in the PDF extraction that should be checked in the source manuscript:

  • “com- plete” appears hyphen-broken across lines in the abstract excerpt .
  • Mathematical notation and subscripts/superscripts in the extracted equations appear unevenly rendered, for example “pˆ︁0(x)” and “c(x) = min qjexec(x)” in the measurement section .
  • The phrase “F / G.” in Table 1 includes punctuation that may be accidental or stylistically inconsistent with the other headers .
  • Some reference entries and URLs may need final consistency review before submission, especially capitalization such as “Rl excursions during pre-training” in the references excerpt .
  1. AI content analysis for this post-2021 manuscript.

Estimated percentage of AI-generated content: low to moderate confidence estimate of 15%–35%.

Rationale: the manuscript is stylistically polished, structurally consistent, and uses balanced academic prose. However, these traits are common in careful human academic writing as well. I do not see obvious AI hallmarks such as repetitive generic transitions, unsupported inflated claims, fabricated citations, or internally inconsistent terminology. On the contrary, the paper includes self-limiting caveats, technical specificity, and meaningful acknowledgment of weaknesses, such as “we do not yet establish that measured primitive coverage adds predictive power beyond full-solution support” and “we do not yet offer a principled selection procedure” . Those features are more consistent with deliberate scholarly authorship than with naive generated text.

Highlighted sections with potentially AI-like fluency:

  • The abstract’s balanced “on the one hand / on the other hand” framing is polished but not suspicious in itself .
  • Parts of Sections 2–3 use highly compressed literature-synthesis prose that is very smooth and could have benefited from more explicit argumentative transitions .

Assessed epistemic impact: low. Even if some editorial AI assistance were used, the central issue is whether claims are properly delimited and supported. The manuscript’s main weaknesses are evidential scope and operational ambiguity, not obvious text-generation artifacts.

Anti-fraud / paper-mill assessment

I do not see strong paper-mill traits. The manuscript is coherent, domain-aligned, cites relevant recent literature, and openly discusses limitations. It fits a recognizable scientific paradigm in machine learning methodology: conceptual framing plus controlled synthetic experiments. The main risk is overgeneralization from a toy setting, not fabrication. The most important integrity-relevant caution is the paper’s own acknowledgement that some measurements may misclassify due to probe failure, shortcut solutions, or contamination artifacts . That honesty counts in its favor, but it also means the empirical claims should remain carefully bounded.

Recommendations

  1. Narrow and sharpen the claimed empirical contribution. State clearly that the experiments validate the framework in a controlled synthetic composition environment, not yet in natural mathematical or coding reasoning. This single change would improve the paper’s epistemic discipline substantially.

  2. Strengthen the predictive claim with explicit sensitivity analyses. Because the framework depends on τp and τc, include threshold sweeps, confidence intervals, and a clearer mapping from rollout budget to classification decisions. Without that, “predictive diagnostic” remains aspirational.

  3. Test whether primitive coverage adds incremental predictive value beyond support. This is currently an exposed weakness in the paper’s own logic. Since the manuscript admits it “do[es] not yet establish” that primitive coverage adds predictive power beyond full-solution support , the revision should either provide such an analysis or reposition primitive coverage as a tentative auxiliary diagnostic.

  4. Add at least one more challenging evaluation regime. Even a modest extension—more primitives, nondeterministic verification, multiple valid decompositions, or a naturalistic reasoning benchmark—would help demonstrate whether the framework survives outside the clean toy setting.

  5. Distinguish mechanism from correlation more carefully. Replace strong causal phrasing around support and composition supervision with more measured claims unless ablation studies isolate the mechanism.

  6. Expand the “what would falsify this framework?” discussion. The paper would become more scientifically persuasive if it stated in advance which empirical outcomes would count against the proposed state-space view.

  7. Improve presentation. Add a notation table, strengthen the captioning and interpretation of Table 1, and ensure all mathematical symbols render cleanly in the final manuscript.

Overall, I judge the manuscript as promising and intellectually substantive, but not yet fully validated at the level suggested by its framing. The most important revisions are to tighten claim scope, improve operational precision, and expand empirical support.

Reviewer3 (AI review)

November 14, 2026

AI Review (reviewer3.com)

Overview

Post-Trainability State Space Framework

Contribution

This study proposes a two-dimensional framework based on effective full-solution support and atomic primitive coverage to categorize post-training dynamics into amplification, composition-limited, and acquisition-limited regimes.

Diagnostic Measurement Protocol

Contribution

The author introduces a pre-post-training protocol utilizing isolated execution probes and handoff curves to diagnose model bottlenecks and predict reinforcement learning outcomes.

Empirical Characterization of Compositional RL

Contribution

Through synthetic experiments across three language models, the paper demonstrates that atomic primitive mastery is insufficient for reinforcement learning gains without nonzero full-solution support during training.

Comments (9)

1. Statistical sampling noise conflated with distribution shift — Minor

Qwen2.5-1.5B do not improve after RL with GRPO, while Gemma does. A post-hoc examination of the Gemma training logs showed that it had rare successful trajectories on the RL training prompts (6 out of 2,000 prompts solved at least once in 64 pre-RL samples), despite zero observed held-out pass@64. This highlights that effective support depends on the training distribution and sampling policy, rather than held-out benchmark performance alone.

Observing 0 successes in 150 held-out prompts (rate 0.000) versus 6 successes in 2,000 training prompts (rate 0.003) is entirely consistent with binomial sampling variation from the exact same underlying success probability of p = 0.003 (where the probability of observing 0 successes in 150 trials is (1 - 0.003)^150 ≈ 0.64). The text attributes this discrepancy to dependence on the training distribution rather than finite sample size (150 vs. 2,000 problems) at a very low base rate.

2. Two-dimensional framework is empirically indistinguishable from one-dimensional support — Critical

skills available in compositional contexts. Consequently, we do not yet establish that measured primitive coverage adds predictive power beyond full-solution support. Moreover, coverage estimates

In the experimental evaluation across all 18 checkpoints in Table 1, post-RL improvement occurs if and only if effective full-solution support (held-out or training-time pass@64) is nonzero, regardless of whether measured primitive coverage is 1.00 / 1.00 (as in M_2) or 1.00 / 0.00 (as in M_1+C^(300)). While Section 6 acknowledges that the experiments do not establish predictive power for primitive coverage beyond full-solution support, this means the empirical study does not actually substantiate the central claim that post-trainability is governed by a two-dimensional state space rather than full-solution support alone.

3. No variance or uncertainty estimates across random seeds — Critical

is pre-RL execution accuracy on isolated single-operation prompts with held-out strings. Post-RL values and changes are averaged over three GRPO seeds.

Table 1 reports post-RL pass@1, pass@64, and their deltas as point estimates averaged over three GRPO seeds without standard deviations, standard errors, or per-seed breakdowns. Because RL training near zero support is driven by rare exploration events (such as 6 solved prompts out of 2,000 in Gemma-3-1B M_2), reporting seed-level variability is essential to determine whether improvements are consistent or driven by single-seed outliers.

4. Missing experimental and hyperparameter details for reproducibility — Critical

We measure pre-RL success on 150 held-out problems using 64 samples per problem. We then apply the same outcome-based GRPO procedure to each checkpoint (150 updates, three seeds). Rewards

The experimental section omits essential training and evaluation details, including the number of supervised examples used to train M_1 and M_2, the exact prompt formats and scaffolding used for compositions and isolated probes, the sampling temperature and decoding parameters for pass@k and GRPO, and the GRPO rollout group size and learning rate. Without these specifications, it is not possible to assess whether the failure to sample compositions in M_2 stems from the model's capability state or from restrictive decoding and formatting choices.

5. Unverified training-time support for non-improving M_2 checkpoints — Major

Qwen2.5-1.5B do not improve after RL with GRPO, while Gemma does. A post-hoc examination of the Gemma training logs showed that it had rare successful trajectories on the RL training prompts (6 out of 2,000 prompts solved at least once in 64 pre-RL samples), despite zero observed held-out pass@64. This highlights that effective support depends on the training distribution and sampling

Post-hoc inspection of the 2,000 RL training prompts was reported for Gemma-3-1B M_2 to explain its unexpected improvement, but no corresponding counts of successful trajectories across the 150 GRPO updates are reported for Qwen3-1.7B M_2 or Qwen2.5-1.5B M_2. Explicitly verifying that the other two M_2 checkpoints sampled zero rewards throughout all 150 RL updates is necessary to confirm that they never encountered a reward signal during training rather than failing to learn from rare successes.

6. Single-operation prompts are not shown to require autonomous composition — Major

ten. Each problem requires a sequence of 2-4 operations containing both F and G . A programmatic verifier checks every intermediate state and the final answer.

The paper states that each problem requires a sequence of 2-4 operations containing both F and G, which suggests the prompt explicitly specifies the sequence of operations to apply rather than requiring the model to autonomously plan or select them. If the operations are explicitly given in the prompt, M_2's failure is a failure of multi-step state tracking or prompt formatting rather than autonomous selection, which conflates the distinct mechanisms described in the composition-limited regime.

7. Proposed measurement protocol instruments are omitted from the empirical validation — Major

Let H_x(t) denote this oracle-assisted completion probability, with H_x(0) corresponding to autonomous full-solution success. We define an empirical composition gap at handoff point t as G_comp(x; t) = H_x(t) - H_x(0).

Section 4 formalizes several diagnostic instruments—including the oracle-assisted handoff curve H_x(t), the empirical composition gap G_comp(x; t), false-success rate validation, and explicit reporting thresholds (tau_p, tau_c)—as core components of the prospective measurement protocol. However, none of these quantities are computed or reported in the Section 5 experiments, leaving the formal protocol largely unvalidated in practice.

8. Amplification signature at pass@64 is confounded by ceiling effects — Major

At high initial support, as with M 2 + C (300) for Qwen3-1.7B, improvements are concentrated more heavily at pass@1 which is consistent with amplification.

The paper cites smaller pass@64 gains relative to pass@1 gains in high-support checkpoints (such as Qwen3-1.7B M_2+C^(300), where pre-RL pass@64 is 0.893 and post-RL pass@64 is 1.000) as empirical confirmation of the amplification regime. However, when pre-RL pass@64 is 0.893, the maximum mathematically possible gain is bounded at +0.107 (which the model achieves in full), so the smaller delta at pass@64 is a metric ceiling effect rather than evidence that RL fails to expand large-k support.

9. Post-RL isolated primitive accuracy is not reported — Minor

Table 1: Pre- and post-RL performance on 150 held-out compositions (64 samples per problem). F / G is pre-RL execution accuracy on isolated single-operation prompts with held-out strings. Post-RL

Table 1 reports pre-RL execution accuracy on isolated F and G prompts, showing that M_1+C^(300) has 0.00 or 0.01 accuracy on G before RL, but does not measure isolated F / G accuracy after RL. Measuring post-RL isolated probe accuracy would clarify whether GRPO only amplifies the compositional trajectories or also resolves the probe-context mismatch by making G accessible in isolation.

XSci (AI review)

November 14, 2026

Summary

The manuscript argues that the question "does RL teach language models new capabilities?" has no answer independent of the base model's state, and factors that state into two axes: effective full-solution support (whether complete successful trajectories are sampleable under a feasible budget) and atomic primitive coverage (whether the constituent skills are executable when the burden of composition is removed). These define three regimes — amplification, composition-limited, acquisition-limited — plus a diagnostic-mismatch region, together with a protocol for locating a model-task pair before post-training. The framework is tested on a synthetic composition task over five-digit strings with two operations F and G, using four supervised checkpoints per architecture across Qwen3-1.7B, Gemma-3-1B and Qwen2.5-1.5B-Instruct, followed by identical GRPO. The central finding is that checkpoints executing both primitives perfectly but with zero observed pass@64 do not improve under RL, whereas 300 composition examples that establish nonzero support enable gains of +0.17 to +0.39 pass@1.

Contribution and Significance

The contribution is a diagnostic reframing: rather than asking what RL does in general, ask which bottleneck a given model-task pair presents and choose the intervention accordingly. The accompanying empirical result — that nonzero reward accessibility, not primitive mastery, separates checkpoints that respond to GRPO from those that do not — is useful to anyone deciding whether to spend compute on mid-training or on RL, and the reframing of mid-training as state preparation follows from it cleanly.

Neither axis is new. The support axis is the coverage notion of Chen et al. [2025], restated operationally; the primitive/composition separation and the synthetic F/G design are those of Yuan et al. [2025], which §5 follows explicitly. What is new is their combination into a two-coordinate diagnostic and the protocol of §4. That makes the weight of the paper rest on whether two coordinates do better than one — and, as discussed below, the evidence presented supports only the first coordinate. The manuscript also does not engage with contemporaneous work reaching the opposite conclusion (Cheng et al., Atomic Skills are the Prerequisite, arXiv:2512.01970, 2025), which would sharpen rather than weaken the authors' position.

Strengths

  • The checkpoint ladder in §5 is a genuinely controlled design: exposure to primitives and to compositions is manipulated independently, across three architectures and two supervision budgets, with a programmatic verifier that checks every intermediate state rather than only the final answer.
  • The Gemma-3-1B M2 result is reported rather than buried. Zero held-out pass@64 with +0.476 pass@1 after RL, traced to 6 of 2,000 training prompts solved at least once, is the paper's sharpest practical lesson: support must be measured on the RL training distribution.
  • The M1+C^(300) rows are an informative counterexample in their own right — isolated G accuracy of 0.00/0.00/0.01 alongside correct use of G inside complete trajectories, which undercuts isolated probing as a diagnostic.
  • §3 is careful where this literature is often loose: support is defined as policy- and budget-relative, failure to observe is distinguished from zero probability, and the fourth quadrant is framed as an instrument check rather than a learning regime. §6 is correspondingly honest about what has not been shown.

Recommendation

Major Revision for the NeurIPS Workshop on Transitioning from Pre-Training to Post-Training.

The experiments are sound and the question is well matched to the venue, so the issues below are not grounds for rejection. But the title and abstract promise prediction from two properties and the data support one, with the paper's own table contradicting the classification rule of §4; the protocol named as a contribution is never executed; and no variance is reported behind eighteen rows of three-seed averages. All three can be addressed within the current approach, largely from data the authors already have.

Major Concerns

  1. The primitive-coverage axis is not supported by the experiments, and the classification rule is contradicted by them (§4 "Classifying model-task pairs"; §5 Table 1). Coverage is defined in §4 as ĉ(x) = min_j q_j^exec. The M1+C^(300) checkpoints have isolated G accuracy of 0.00, 0.00 and 0.01, hence ĉ ≈ 0, with pass@64 of 0.033, 0.020 and 0.093; both coordinates are low, so §4 classifies them as potentially acquisition-limited, the regime for which §3.1 prescribes mid-training, SFT or distillation before RL. GRPO nonetheless yields +0.169, +0.232 and +0.393. Conversely the M2 checkpoints sit at the top of the coverage axis (F/G = 1.00/1.00) and yield exactly 0.000 in two of three architectures. Across all eighteen rows the outcome is predicted by "pre-RL pass@64 > 0" alone, with Gemma M2 the sole exception. §6 concedes that coverage has not been shown to add predictive power, but the title, abstract and §7 still assert a two-dimensional predictor.

    The manuscript has the machinery to address this: §4's triage is written for precisely the situation the M1+C^(300) rows present, yet is never applied to them. Either apply it — re-probe G against the decomposition the successful trajectories actually follow, and say whether the probes or the decomposition were at fault — or restate the claim at one dimension and present the second axis as a hypothesis the experiments do not yet test.

  2. The protocol of §4 is not executed, and its one measured coordinate failed prospectively (§4; §5 Results). §4 specifies ĉ(x) per problem, the handoff curve H_x(t), the composition gap G_comp(x;t), thresholds τ_p and τ_c with a sensitivity analysis, a false-success-rate check, and the triage procedure, and insists that "all measurements and classifications should be made before observing post-training outcomes". §5 reports pass@1, pass@64 and aggregate F/G accuracy only; no handoff curve, no composition gap, no threshold values, and no classification is actually performed. The one coordinate that is measured then failed on Gemma M2 and was repaired by a post-hoc examination of training logs, and because that rescue measurement is reported for a single checkpoint, a reader cannot tell whether training-prompt support would have separated Gemma M2 from Qwen3 M2 and Qwen2.5 M2 in advance. Reporting it prospectively for all eighteen checkpoints would convert the anomaly into the paper's strongest result; running the handoff curve on the M2 checkpoints would give the protocol its first test.

  3. No variance is reported for the three GRPO seeds (Table 1 caption: "Post-RL values and changes are averaged over three GRPO seeds"). Several central comparisons rest on differences whose significance cannot be judged: Gemma M2 at +0.476 against Qwen3 M2 and Qwen2.5 M2 at 0.000, and Qwen3 M2+C^(100) at 0.452 against M2+C^(300) at 0.988. RL from near-zero support is expected to be bimodal across seeds, so +0.476 could be three runs near 0.48 or one near 1.0 and two near 0.2, readings that support different conclusions. Per-seed values and spread should be reported, with confidence intervals on the pre-RL estimates as well (pass@1 = 0.003 is roughly 29 successes in 9,600 samples).

Minor Comments

  1. The amplification signature claimed in §3.1 and §5 is confounded by a ceiling: Qwen3-1.7B M2+C^(300) starts at pass@64 = 0.893 and ends at 1.000, so the largest possible pass@64 gain is 0.107 against the observed +0.237 at pass@1, and the same holds for Gemma M2+C^(300) (0.987→0.998).
  2. With n = 64 samples and pass@64 as the headline budget, k = n, so the Chen et al. [2021] estimator degenerates to "at least one success in 64" at its highest-variance setting.
  3. The decoding policy, temperature and top-p used for the 64 pre-RL samples are never stated, although §3 and §6 both make support policy-dependent.
  4. GRPO hyperparameters are absent (group size, batch size, learning rate, KL coefficient, rollouts per update), the RL prompt-set size of 2,000 appears only incidentally in the Gemma note, and there is no code or data availability statement.
  5. The supervised construction of M1 and M2 is unspecified: the number of F and G examples, the epochs, the prompt format, and how probe strings are held out.
  6. §4 writes the support coordinate as ρ̂_0(x) under "Estimating full-solution support" and as p̂_0(x) under "Classifying model-task pairs".
  7. In Table 1 the "Pre-RL" header spans two columns although the caption assigns F/G to pre-RL as well, and "Change" spans two columns while only Δpass@1 is given; Δpass@64 should be added, since the text argues about expansion at pass@64.
  8. In §4, "To investigate this distinction" has no antecedent, as the preceding paragraph ends on aggregate primitive success rates.
  9. Gemma-3-1B diverges qualitatively from both Qwen models at M2 and M2+C^(100) (pre-RL pass@1 of 0.548 against 0.064 and 0.046) and this is not discussed beyond the training-log note.
  10. §2.1 presents the pass@k debate without the principal direct rebuttal of Yue et al.: M. Liu, S. Diao, X. Lu, J. Hu, X. Dong, Y. Choi, J. Kautz and Y. Dong, "ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models", NeurIPS 2025 (arXiv:2505.24864).
  11. The tension between the paper's finding that primitive mastery is insufficient and the opposite conclusion of S. Cheng, X. Yin, R. Zhou, Y. Li, X. Wang, L. Pan, W. Y. Wang and V. Zhong, "Atomic Skills are the Prerequisite: When Reinforcement Learning Synthesizes Compositional Reasoning, and When It Only Amplifies", arXiv:2512.01970 (2025), deserves explicit discussion.
  12. Composition depth is fixed at 2–4 operations; showing how the two coordinates and the RL outcome behave as depth grows would test whether the support axis degrades gracefully.