LLM-Attack Suite: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems
Georgios Politis, Evangelos Pappas
Abstract
Split language models keep input text on a client but send intermediate representations to
a server. Attacks can reconstruct text from these messages. Comparing privacy evaluations is
difficult when attackers see different messages, know different model weights or use different
datasets and recovery measures. We present LLM-Attack Suite, a framework that records
these choices and lets researchers check, compare and extend an evaluation. Compatible
attacks reuse recorded messages, and their predictions are saved for independent scoring.
The scoring rule requires a linked undefended reference before a defence can pass. Reports
also identify attacks that remain untested. In a controlled GPT-2 experiment, adding the
returned training gradients — measured as loss-gradient probes on held-out text at the
frozen trained checkpoint — increased token recovery from 94.20% to 97.38% and exact-
document recovery from 13.71% to 37.77%. The 95% interval for the token gain lay above
the two-percentage-point threshold fixed before testing. This experiment is shared with a
companion study; here it tests how the evaluation responds to a change in attacker access.
On 28-layer Qwen3-0.6B (V = 151,936, grouped-query attention and SwiGLU), a 304-cell
controlled benchmark across depths 1,7,14,21 and eight victim seeds shows even stronger
gradient synergy: adding returned gradients at midpoint depth 14 raises token recovery from
55.20% to 69.05% (+13.85 pp) with public layers and from 72.79% to 86.87% (+14.08 pp)
with trained layers, and at depth 21 with trained layers raises token recovery from 75.48%
to 96.85% (+21.37 pp; exact documents 0.00% →44.58%). Further studies measure how
corpus choice and compression defences affect recovery, compare an adapted published attack
with its reference implementation, and test the measurement code. The framework reports
implementation checks separately from measured attack effectiveness. We recommend that
each privacy claim state the attacker, dataset and scoring rule, report its controls and identify
untested attacks.
Review reports
Version v1
xPeerd (AI review)
AI Review (xPeerd — DAReview)
Summary
This manuscript presents LLM-Attack Suite as a framework for evaluating privacy leakage in split and distributed LLM systems by explicitly recording attacker observations, model knowledge, datasets, controls, and scoring rules. The paper’s central claim is methodological rather than purely attack-driven: privacy evaluations should be reusable, comparable, and independently checkable because attack outcomes depend strongly on observation access, corpus, model split depth, and defence configuration. The abstract states that, in a controlled GPT-2 experiment, “adding gradients returned during training increased token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77%” and that saved predictions and linked undefended references support independent scoring and defence evaluation . The manuscript is generally coherent, technically focused, and unusually strong on protocol explicitness and reproducibility infrastructure. Its main strengths are the careful threat-model framing, predeclared comparisons, controls, and separation of conformance testing from effectiveness claims. Its main weaknesses are that the paper sometimes overstates generality relative to its strongest quantitative evidence, the novelty claim is more about evaluation organization than new attack capability, and some defence conclusions risk being confounded by utility degradation rather than privacy improvement .
Potential Major Revisions
- Clarify the exact scientific contribution and narrow any implied generality
The manuscript’s strongest contribution is an evaluation protocol, not a new state-of-the-art attack. This is stated clearly in places, but the framing still risks being read as broader evidence about split-LLM privacy in general than the experiments actually establish. The introduction says, “What must an evaluation record and test so that another researcher can check its conclusion, compare it fairly and extend it? LLM-Attack Suite records what the attacker can observe, what it knows and the resources it uses” . Later, Section 6 states, “Taken together, the programme establishes consequential leakage under several independent protocols” . This is reasonable, but the manuscript should more explicitly distinguish:
- evidence for the framework’s usefulness,
- evidence for specific attacks under specific contracts,
- and evidence for the broader security status of split systems.
This distinction matters because the paper itself concedes that “The strongest quantitative reconstruction comparisons in this paper concern GPT-2 124M under the stated corpora, lengths and budgets” and warns that aggregation across protocols would discard interpretive context . I think the manuscript should move this limitation forward into the abstract or end of the introduction so the scope is unambiguous from the outset.
- Strengthen the novelty argument beyond procedural packaging
The stated three-part contribution includes “A common evaluation procedure,” “Checkable privacy results,” and “Evidence for the procedure” . This is coherent, but the manuscript should more explicitly position what is substantively new relative to prior frameworks such as VFLAIR-LLM, garak, and ML Privacy Meter discussed in Related Work . At present, some of the novelty may appear to be a combination of attack harnessing, evidence retention, and scorekeeping rather than a fundamentally new methodological principle.
A sharper revision would specify which components are genuinely new and nontrivial, for example:
- linked undefended references as a gating condition for defence claims,
- explicit reporting of untested attacks,
- retained predictions enabling independent rescoring,
- and separation of conformance evidence from efficacy evidence.
Without this sharper differentiation, reviewers may see the framework as an engineering integration effort rather than a methodological advance.
- Reconcile “defence effectiveness” with “utility collapse” more rigorously
The defence analysis contains an important but potentially underdeveloped point: some apparent privacy improvements derive from crippling model utility rather than meaningful privacy-preserving transformation. The manuscript notes that mid-lite “remained at 0% recovery” but “achieves this result by largely preventing learning: mean training cross-entropy is 7.42–7.75 versus 4.39–4.44 without a defence” . Similarly, Section 5.1 reports that some perturbations barely reduce recovery, while Section 5.2 indicates other defences change the conclusion substantially .
I think the paper needs a more explicit principle for interpreting defence success under utility loss. Right now, the reader can infer the authors’ stance, but the framework would be stronger if the manuscript formalized when a defence should be treated as nonviable despite reducing leakage. Otherwise, the scoring framework may still leave room for misleading “secure because unusable” conclusions.
- Expand the discussion of statistical design and dependence structure
The main campaign is commendably prespecified: “The primary statistical test fixed in advance is the two-point gain test” and the interval is bootstrapped by resampling “trained models and documents independently” . The supplement also highlights a dependence stress case where correlated document columns were handled conservatively . This is a strength. However, because the same 256 held-out documents are reused across multiple trained models and attack seeds, the paper should explain dependence assumptions more explicitly in the main text, not only in supporting material.
In particular, the manuscript should justify:
- why the resampling unit choice is appropriate for all reported metrics,
- whether model-seed variability dominates document variability,
- and whether multiple derived comparisons from shared captures may induce interpretive dependence beyond the Bonferroni correction.
The current design looks careful, but it deserves clearer exposition because the paper’s credibility rests heavily on measurement rigor.
- Better justify the fairness of attack comparisons
Section 5.3 is transparent that attack comparison uses each attack’s “own fixed budget” rather than equal compute and that the framework avoids naming a universal winner . I think this is honest, but potentially vulnerable to reviewer criticism. A reader may ask whether the joint attack’s superiority is partly a budget artifact rather than a signal-access effect.
The manuscript would benefit from a short, explicit argument for why protocol-faithful fixed-budget comparison is the right benchmark here, and perhaps a brief sensitivity analysis or discussion of what an equal-compute comparison would and would not show. As written, the position is defensible but under-argued.
- More directly address possible paper-mill or overengineered-framework concerns
I do not see strong signs of fabrication in the usual sense; the manuscript is internally consistent, cites specific seeds, counts, intervals, retained artifacts, and validation paths in unusually concrete ways . That said, the work has a high “framework density”: many modules, taxonomies, evidence tiers, releases, and qualification matrices. For some reviewers, this can create a risk that the contribution appears diffuse or branding-heavy relative to the central scientific result.
This concern is amplified by statements such as “the registry records the available validation evidence for all 64 modules” and by extensive supplementary taxonomy tables . I think the paper should pre-empt this reaction by tightening the main narrative around a smaller set of indispensable claims and making clear which framework breadth matters directly for this paper’s argument versus what is merely release context.
Potential Minor Revisions
- Terminology and naming consistency
The paper states, “The suite was previously called LLM-Attacker” and later clarifies that historical identifiers may retain the earlier name for reproducibility . This is understandable, but the manuscript should ensure a single naming convention in the main text, with one concise reproducibility note. Repeated naming explanations can distract readers and slightly dilute the paper’s focus.
- Improve readability of dense enumerations
Several passages are accurate but overloaded, for example: “Supplementary Table S10 records each protocol’s threat model, module, control, model, corpus, budget, outcome, comparator and evidence tier” . Similar density appears in release-path descriptions and taxonomy material . I think some of these lists could be condensed or moved to appendix-forward prose, with the main text emphasizing interpretation rather than inventory.
- Reduce promotional or normative phrasing
The conclusion states, “Every privacy claim for a split model should therefore name its attacker, corpus and scoring rule, report the supporting controls and leave untested attacks labelled as untested” . The normative force is understandable, but “should therefore” may read as stronger than the evidence warrants for all future settings. A slightly more measured phrasing would preserve the recommendation while sounding less prescriptive.
- Clarify structural recovery vs text reconstruction earlier
Section 6 helpfully states, “Structural recovery and text reconstruction remain distinct outcomes” . This distinction is important and should be made earlier, ideally where the Qwen audit is first invoked, because otherwise readers may overgeneralize from one leakage mode to another.
- Tighten figure interpretation
Figure 2 is informative and the single-document example is vivid, but the manuscript should explicitly caution against overinterpreting one qualitative example. The caption already helps by describing the example as representative relative to average token recovery . A sentence in the main text reinforcing that the inference rests on aggregate resampled estimates, not anecdotal examples, would strengthen presentation discipline.
- Typographic and style issues visible in extracted text
I observed minor style irregularities in the extracted text:
- inconsistent hyphenation and spacing around percentage-point expressions such as “two- percentage-point” in the abstract excerpt ;
- occasional dense sentence chaining with semicolons and long noun stacks, especially in release and taxonomy descriptions ;
- some table-heavy sections may benefit from parallel phrasing and shorter lead sentences.
These are not major writing defects, but light copyediting would improve readability.
- AI content analysis for this post-2021 manuscript
Estimated AI-generated content: low to moderate, approximately 10–25%.
Rationale: the prose is highly regular, consistently structured, and unusually uniform in sentence rhythm, which can be compatible with either careful human technical writing or AI-assisted editing. However, the manuscript also contains many concrete experimental specifics, nuanced distinctions, and cross-referenced methodological cautions that are less typical of generic AI-generated scientific text. For example, the paper carefully separates conformance from effectiveness claims and includes highly specific validation statements such as “All 37 selected cases matched their expected outcomes, with no failures or skips” and release-level bookkeeping with stable identifiers and preserved manifests .
Potential AI-assisted zones:
- abstract and contribution framing, which are very polished and template-like ;
- conclusion and some methodology summaries, which use balanced, repetitive rhetorical structure ;
- release and taxonomy descriptions, which have a cataloguing style that could have been partly machine-assisted .
Assessed epistemic impact: limited if AI assistance was used only for drafting or editing. The paper’s contribution depends primarily on experimental design transparency and retained evidence, not on literary originality. The main risk would be not fabrication but over-smoothing of claims or excessive packaging language. I therefore see no strong AI-related epistemic concern beyond the need for tighter scope claims.
Recommendations
| Recommendation | Practical action |
|---|---|
| Sharpen contribution statement | Revise abstract and introduction so the paper is unmistakably framed as a methodological evaluation framework paper, with attack findings serving as evidence for the framework rather than universal claims about split-LLM privacy. |
| Narrow scope claims | Move the limitation now stated in Section 6—especially that the strongest quantitative evidence concerns GPT-2 124M under stated corpora, lengths, and budgets—into the introduction or abstract . |
| Formalize defence interpretation | Add an explicit criterion or discussion for when a defence reduces leakage by degrading utility so severely that it should not be considered a meaningful privacy win . |
| Strengthen comparative methodology justification | Explain more directly why fixed-budget, protocol-faithful comparison is preferable here to equal-compute ranking, and discuss what each comparison type would imply . |
| Bring statistical assumptions into the main text | Summarize bootstrap units, dependence handling, and multiplicity correction in the main paper, not only supplementarily . |
| Reduce framework sprawl in the main narrative | Keep module catalogues and release inventories in the supplement, while focusing the main text on the specific claims necessary for this manuscript’s acceptance argument . |
| Copyedit for readability | Shorten long inventory-style sentences, harmonize naming, and simplify repeated reproducibility notes without removing technical precision. |
Fraud / coherence warning
I do not find clear evidence of fabrication or paper-mill characteristics in this manuscript. On the contrary, the paper is unusually explicit about threat models, counts, seeds, controls, validation paths, and evidence retention . My main caution is not fraud but overextension: the manuscript should ensure that broad framework ambition does not outrun the narrower domain of its strongest empirical support.
Reviewer3 (AI review)
AI Review (reviewer3.com)
Comments (11)
1. Sharded module recovery range mismatch between Table S10 and S15 — Editorial
In Table S10 (page 22), the observed recovery for the sharded module is reported as "sharding 54.95–67.86%". However, in Table S15 (page 32), the detailed breakdown for the sharded module reports "Static 56.98–67.86%; staggered 47.58–65.13%; independent 12B replication 54.95–65.68%". The summary range in Table S10 combines the minimum of the independent 12B replication (54.95%) with the maximum of the static arm (67.86%), completely omitting the lower bound of the staggered arm (47.58%), even though "static and staggered arms" are explicitly listed as the evaluated conditions in that same row of Table S10.
2. Single frozen-snapshot evaluation does not capture live split-training dynamics — Major
In the main experiment the attacker reads one fixed snapshot of this traffic rather than live training traffic. We fine-tune the whole model, freeze it, split it, and pass the 256 evaluation
The authors claim that adding returned gradients during split training significantly increases text reconstruction. However, as explained in Supplementary Section S2, the attacker in this campaign never observes actual training steps on training data; instead, the whole model is fine-tuned first, frozen, and then evaluated on a single static pass of 256 held-out documents. In live split training, weights evolve continuously across batches, gradients reflect in-flight parameter updates and changing loss landscapes, and the client's layers drift away from the public pretrained initialization during the training trajectory. While the authors note that the attack reads a fixed snapshot, the headline and main-text claims frame the finding broadly as "gradients returned during training" or "during split training." The authors should temper the scope of Claim 1 in the abstract and main text to specify that the measurement reflects single-batch backward-pass gradients on a frozen checkpoint rather than live multi-step split training.
3. Order-of-magnitude calibration data asymmetry in adapted BiSR comparison — Major
It kept the main experiment’s 256 documents, ten trained models, both depths and both copies of the layers, reused the earlier results as controls, and trained fresh inverters on the public auxiliary text. Each attack used its own fixed protocol budget. At depth six with the public
When comparing the adapted BiSR attack against the joint and activation-only attacks in Section 5.3 and Section S3.4, the joint and activation-only attackers fit their linear decoders on 32 public calibration passages (1,024 tokens total) from WikiText-2, whereas BiSR's GRU inverter is trained for 20 epochs on 112 auxiliary articles (15,604 windows, roughly 500,000 tokens) from WikiText-103. While the authors note that each attack retains its own fixed protocol budget, varying both the volume of public calibration data (by nearly 500x) and the domain/source split of the calibration data (WikiText-103 vs. WikiText-2) alongside the attack algorithm means the comparison does not hold the attacker's data grant ( and ) fixed as specified in Section 2.3 ("An attack comparison holds the remaining components fixed and reports each attack's assigned budget"). The authors should explicitly note this calibration-data difference in Section 5.3 and Table 3 so readers understand that both the calibration corpus size and optimisation budgets differ across the compared attacks.
4. Main joint reconstruction routine is outside the registered module suite — Critical
This attack is a self-contained routine, distinct from the registered inversion module’s own joint variant, which explores candidates differently. The four steps match the numbered badges of Figure S3.
The authors claim that LLM-Attack Suite provides a unified experimental framework whose modules share observation contracts and evidence records, and use the primary GPT-2 joint-versus-activation campaign as the flagship demonstration of this framework. However, Supplementary Section S2 explicitly notes that the strongest reconstruction attack used in the main campaign is a "self-contained routine, distinct from the registered inversion module's own joint variant, which explores candidates differently." Furthermore, Table S14 and Table S15 (which catalogue all 64 registered modules) do not include joint_true_k32 or forward_k32 as registered modules; instead, seam-inversion is validated only on the domain-shift and white-box studies. Relying on an external, standalone script for the paper's centerpiece confirmatory experiment undermines the claim that the modular registry itself unifies and executes the core evaluations. The authors should either integrate the exact candidate-selection and beam-search routine from the main campaign into the registered seam-inversion module (or register it as its own module) and show conformance, or revise the main text to clearly distinguish between the registered module suite and the standalone experimental scripts used for the main GPT-2 campaign.
5. Candidate-ranking vs candidate-proposal effects of gradients are confounded — Major
test whether any gain depends on the gradient information. Matching the search does not match total computation; the joint attacker also ranks candidates using gradients.
The authors claim that matching the candidate count () and beam width () between the activation-only and joint attackers isolates the information contributed by the returned gradient, while also noting in passing on page 3 that "the joint attacker also ranks candidates using gradients." However, in Section 4.1 and Supplementary Section S2 (Steps 3 and 4), the beam search scoring metric is described purely as retaining the two partial texts "whose recomputed activations lie closest to what crossed the split," with no explanation of how or where gradients are used to rank candidates during beam search or whether the shuffled-gradient and random-candidate controls also apply gradient-based ranking. If the joint attacker uses gradients both to propose 16 candidates and to score/rank partial sequences during beam search, then replacing the 16 candidates in the random-candidate control while altering or omitting the ranking function confounds candidate proposal with beam-search ranking. The authors should explicitly write out the exact mathematical objective used in Step 4 (beam search) for the activation-only, joint, shuffled-gradient, and random-candidate arms so readers can verify how gradient ranking is performed and controlled.
6. Pooling positions across random seeds inflates sample size in Wilson bound — Minor
Undefended GPT-2 at depth six, public layers, inference, three seeds and 3,072 pooled positions per corpus. Excess uses the Wilson upper bound, not the recovery point estimate minus the baseline.
In Table 2 and Table S4, the authors compute the Wilson upper-bound excess and finite-sample floors by pooling 3 random seeds over the same pinned evaluation corpus to obtain inference token positions (and training positions in Table S5). Because the linear inversion decoder is deterministic given the public pretrained GPT-2 layers and the fixed calibration set, or evaluates the exact same held-out tokens across 3 seeds, pooling the 3 seeds treats repeated evaluations of the exact same 1,024 inference tokens (or 256 training tokens) as 3,072 independent Bernoulli trials. This artificially triples the sample size in the Wilson score formula , shrinking the uncertainty margin and finite-sample floor by roughly . The authors should compute the Wilson bound using the number of unique token positions per run ( for inference, for training) and average across seeds (as done in Table S6), or justify why repeated runs on identical pinned text count as independent trials.
7. Inconsistent majority-token baseline between 20/100-step and 500-step runs on same corpus — Minor
fixed amount along the direction in which activations vary most. The corpus’s majority token does not occur in the evaluated passages, giving the battery a fixed 0% guess rate.
In Section S3.2, the authors state that for the 20- and 100-step horizon battery on the pilot corpus, "The corpus's majority token does not occur in the evaluated passages, giving the battery a fixed 0% guess rate." Yet two paragraphs later, when describing the 500-step extension on the exact same declared setting and pilot corpus, they state that the inversion attack recovered "0.04-0.85% with mid-lite, below its 4.83-4.96% majority baseline. This extends the same declared setting to 500 steps". If both experiments evaluate the same pinned pilot corpus under the same declared setting, the majority-token baseline should not jump from 0% to 4.83-4.96%. Furthermore, a 0% calibration-majority baseline at 20 and 100 steps means the calibration split and evaluation split have disjoint majority tokens, making the baseline unrepresentative and allowing mid-lite (with 0% raw recovery) to hit a finite-sample Wilson floor of +1.00 pp at 20 steps. The authors should explain why the majority baseline changes between the 20/100-step battery and the 500-step extension, and ensure a consistent baseline definition across horizons.
8. Unexplained recovery ranges and missing undefended baselines for number/identifier spans — Minor
, the stretches of text that hold a name, a number or an identifier. Under secret mixup the attacker recovered 47.85–66.15% of name spans (716 spans in 249 documents), 93.94–96.50% of number spans (297 spans in 142 documents) and 90–100% of identifier spans (two spans in two documents), against 67.36–97.44% of name spans without the defence. The annotations were
The authors report defended token, name-span, number-span, and identifier-span recovery under secret mixup as ranges (e.g., "83.01-90.67% token recovery", "47.85-66.15% of name spans", "93.94-96.50% of number spans") and compare name-span recovery against an undefended range of "67.36-97.44%", without specifying which of the 12 defended attack configurations (depth 3 vs. 6, public vs. trained layers, activation-only vs. joint vs. SipIt) correspond to the endpoints of these ranges. Moreover, no undefended baseline percentages are provided in the text or tables for number spans or identifier spans, making it impossible for the reader to tell how much secret mixup actually reduced number or identifier leakage relative to an undefended model. The authors should provide a supplementary table breaking down token, document, and span recovery by specific attack configuration and depth for both undefended and defended models.
9. Identifier-span recovery relies on only two spans across 256 documents — Minor
, the stretches of text that hold a name, a number or an identifier. Under secret mixup the attacker recovered 47.85–66.15% of name spans (716 spans in 249 documents), 93.94–96.50% of number spans (297 spans in 142 documents) and 90–100% of identifier spans (two spans in two documents), against 67.36–97.44% of name spans without the defence. The annotations were
In Section S3.1, the authors report that under the secret mixup defence, the attacker recovered "90-100% of identifier spans (two spans in two documents)." A sample size of spans across the entire 256-document evaluation set is far too small to provide a reliable or meaningful estimate of identifier-span leakage. The authors should either expand the evaluation corpus/annotations to include a statistically meaningful number of identifier spans or refrain from highlighting percentage recovery rates on a two-item subset.
10. Unexplained per-token gradient extraction from a 256-document batch loss — Major
documents through it once. The activations come from the frozen client layers, and the gradients from the next-token loss averaged over all 256 documents.
The authors state that in the main experiment, the target model is frozen and the 256 held-out evaluation documents are passed through it once, with the attacker observing "gradients of the next-token loss averaged over all 256 evaluation documents." However, in split learning, the backward pass across the split returns the gradient of the loss with respect to each individual sequence's intermediate activation tensor (which is scaled by when the batch loss is averaged over 256 documents, as confirmed on page 40 where BiSR "multiplied the stored gradients by 256 to convert the loss normalisation to a per-document mean"), rather than averaging the gradients of different documents together into a single tensor. Because Step 3 of the attack relies on the cosine similarity of the negated gradient at the previous position, scalar batch normalisation () has zero effect on cosine similarity, whereas actually averaging gradients across 256 distinct documents would mix signals from 256 different texts. The phrasing "gradients of the next-token loss averaged over all 256 evaluation documents" is ambiguous and could be misread as cross-document gradient aggregation. The authors should clarify in Section 4.1 and Section S2 that each document's activation receives its own distinct backward gradient tensor scaled by the batch-mean normalisation factor, rather than gradients being averaged across documents.
11. Missing undefended and compressor-unaware baselines for white-box embedding optimisation — Minor
tokens. Across three training seeds, both methods and three widths, the attack recovers between 18.85% and 68.59% of tokens on average at every depth (Table S7). Each mean averages the three widths.
The authors evaluate a defence-aware white-box continuous embedding optimisation attack across depths 1 to 11 under PCA and bottleneck compression (64, 128, and 256 dimensions), reporting 18.85% to 68.59% mean token recovery (Table S7). However, the study does not include an undefended baseline (optimising continuous embeddings against uncompressed 768-dimensional activations at each depth) or a defence-unaware baseline on the same 32-sequence pilot corpus. Without knowing what continuous embedding optimisation recovers without compression at depths 1-11, or how much recovery varies across widths 64, 128, and 256 individually, it is unclear how much of the depth-dependent decay (from 68% at depth 2 to 19% at depth 11) is caused by the compression defence versus the inherent non-invertibility of deeper transformer layers under 128 steps of AdamW optimisation. The authors should report the uncompressed continuous-optimisation recovery across depths 1-11 and provide the disaggregated results for each compressed width (64, 128, 256) rather than only averaging across all three widths.
XSci (AI review)
Summary
The manuscript argues that privacy evaluations of split language models are not comparable because studies differ in what the attacker observes, what it knows, which corpus it attacks and how recovery is scored. It proposes LLM-Attack Suite: an evaluation tuple E = (T,V,O,K,B,C,M,G) (Eq. 1), capture-and-replay so that compatible attacks run on the same recorded activations and gradients, a scoring rule that refuses a defence a pass without a linked undefended reference, a coverage report for untested threats, and a validation matrix over 64 modules. The empirical core is a controlled GPT-2 124M campaign on 32-token WikiText-2 passages: adding returned gradients to an activation-only attacker raises token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77% at depth six with public client layers, a prespecified test passed with a 95% interval of [2.72, 3.64] percentage points. Further studies vary corpus, defence and training horizon, conformance-check a port of BiSR, and validate the measurement code on 37 cases.
Contribution and Significance
The core contribution is methodological rather than an attack: a specification of what a split-LLM privacy evaluation must record, plus machinery that makes a recorded evaluation re-scorable and extensible by a third party. What is genuinely new is the discipline applied to the evidence itself — a hypothesis and threshold fixed before running, 47 pre-registered secondary contrasts, a mandatory linked undefended control before any defence can pass, and a suite that validates the scorer rather than the attacks. The last of these, checking by fault injection and metamorphic relations that supplying ground truth to the predictor is rejected and that removing a required control blocks a pass (Section 5.4, Table S11), is not something I have seen done in this literature, and it addresses a real failure mode.
The significance is narrower than the framing suggests. The closest prior framework, VFLAIR-LLM [7], is distinguished in a single sentence of Section 7 with no capability comparison; where its defences appear, they are degraded stand-ins ("mid-lite, named after the mutual-information defence but learning nothing"; "at-lite, named after adversarial training but performing none"). The headline numbers are also not new here: Table S3's caption states that the same experiment supplies the companion paper's [17] table, and the Qwen result quoted in Section 6 comes from [16]. That the returned gradient leaks the next token follows from the structure of the cross-entropy gradient, as the authors explain in Section S2, step 3; the result quantifies a known mechanism rather than discovering one.
Strengths
- The primary test is prespecified with a fixed threshold, a declared bootstrap (100,000 replicates, seed 93281) and 47 named secondary contrasts (Sections 4.1, S3.1) — rare in this literature.
- The controls isolate the gradient's information rather than the search structure: shuffled gradients give 93.91% and random candidates 93.89%, both below the 94.20% activation-only result, while the backward-only heuristic reaches 49.98% (Section 5.1).
- The token-versus-document dissociation is a substantive lesson about metric choice: +3.17 points of token recovery corresponds to +24.06 points of exact documents (Table 1), and secret mixup leaves 83.01–90.67% token recovery while holding exact documents to at most 1.48% (Section 5.1).
- The BiSR port is held to bit-identical agreement with the archived reference across all 3,072 paired predictions before being used as a comparator (Section 5.3).
- Table S15 separates reproduced, adapted, real-model qualified, synthetic and helper modules, and the paper declines to aggregate across protocols (Section 6).
Recommendation
Major Revision for PoPETs.
The protocol, the artifact and the prespecification discipline are a contribution PoPETs should want, and the gradient result and the conformance checks are convincingly supported in the settings tested. But the scoring rule is the paper's claim to make privacy results decidable, and on the manuscript's own numbers it does not yet work as one; and the positioning against the closest existing framework and against the authors' own companion paper is not established. Both are repairable within the paper's approach, but need new analysis and experiments rather than clarification.
Major Concerns
-
The pass/fail rule is determined by sample size and ignores utility (Section 3; Tables S4, S5, S6). The rule passes a defence whose largest excess is at most one percentage point, but the manuscript reports finite-sample floors — the excess an attack scores while recovering exactly at the baseline — of 0.84–1.90 points for the n = 3,072 inference rows and 2.34–3.91 points for the n = 768 training rows (Section S3.2). At n = 768 no defence can pass whatever its protection, and the one PASS reported is explicitly "at floor": mid-lite scores +1.00 and +0.20 points (Table S6) on a battery whose guess rate is fixed at 0% and whose recovery is 0%, so the verdict encodes no measurement. The authors note the symmetric problem for the control — "a sufficiently large floor can satisfy it without high point recovery" (Section 3) — so the rule is satisfiable by choosing a small n for the reference and a large n for the candidate. Nor is there a utility arm: Section 5.2 shows mid-lite reaches its PASS by preventing learning (cross-entropy 7.42–7.75 against 4.39–4.44 undefended).
The supporting statistics are not yet justified either. The multiplier z = 2.5392 "originated in a five-percent error budget across nine known-plaintext probe attempts" (Section 3), the Wilson bound is binomial while the scored tokens are correlated (also conceded there), and the 47 secondary intervals are Bonferroni-corrected to roughly the 0.1% level from a bootstrap resampling only ten trained models (Section 4.1), where such tail quantiles are not estimable. I would be satisfied by an equivalence test against a prespecified margin in place of the screening bound; document-clustered rather than token-binomial intervals; a minimum detectable excess reported with every verdict; and a required utility arm.
-
The advance over VFLAIR-LLM [7] and over the companion paper [17] is asserted rather than shown (Sections 1, 7; Table S3 caption). Section 7 differentiates the suite from VFLAIR-LLM in one sentence listing desirable properties, with no feature-level comparison, no attempt to run a VFLAIR-LLM attack or defence in this harness, and no result the proposed contract makes possible that VFLAIR-LLM does not. The one place its defences enter, the horizon battery of Section S3.2, uses stand-ins the authors state do not implement the defences they are named after. Meanwhile the strongest empirical evidence is shared with [17] and the Qwen audit in Section 6 is from [16], leaving the reader unable to tell what this paper establishes alone. The revision should add a capability comparison against VFLAIR-LLM, garak [4] and ML Privacy Meter [13]; implement the real forms of the defences whose verdicts are reported; and state early which results are new here.
Minor Comments
- The joint-versus-activation contrast is run only on 32-token passages with GPT-2 124M, since the 64- and 128-token conditions of Table S8 use the inversion attack alone, so the central claim is untested at a realistic sequence length or model size.
- The victim is fine-tuned on 128 documents of 32 tokens, which may explain why the trained-layer condition barely differs from the public one (94.81% against 94.20% at depth six, Table 1).
- No condition tests a mismatch between the attacker's calibration text and the private corpus; each corpus in Section 4.2 is scored against its own calibration-fitted baseline and decoder.
- SipIt [14] is a self-reimplementation run at a budget that, as Section S2 states, voids its exact-recovery guarantee, and is not conformance-checked as BiSR was, yet it beats the joint attack at depth three with trained layers (98.89 against 98.72, Table 3).
- Table S15 gives 35 of 64 modules only synthetic-fixture evidence in which the expected outcome is planted by the authors, which the title's "unified red-team framework" does not convey.
- The name-span results (47.85–66.15% across 716 spans, Section 5.1) rest on annotations made "with AI assistance", with no human verification or agreement statistic.
- Section 5.2 reads "All three underfunded conditions failed the token gate"; Section S3.2 shows this should be "undefended".
- The artifact URLs disagree between the Evidence section and Section S4 (setloop-io against setloopio; LLM-Attack-Suite against LLM-AttackSuite), and the reviewed matrix covers 64 modules at tag submission-candidate-20260922-r2 while released tag v2.0 carries 65 (Table S13).
- "Data files require Hugging Face login and manual approval", which places the independent rescoring promised in Section 2.4 at the authors' discretion.
- Section 6 calls the Qwen system "deployed" two paragraphs before stating that "All experiments used author-controlled systems".
- The main text should carry the analytic reason the gradient helps, currently only in Section S2, step 3, so readers can calibrate the size of the reported gain.
- Three works are missing: Zhao, Mopuri and Bilen, "iDLG: Improved Deep Leakage from Gradients", arXiv:2001.02610, 2020, whose analytic label-recovery result is the mechanism the joint attack exploits and which Table S1 names without citing; Gupta, Huang, Zhong, Gao, Li and Chen, "Recovering Private Text in Federated Learning of Language Models", NeurIPS 35, 2022, which already pairs gradient-derived token candidates with beam search; and Li, Hong, Xie et al., "LLM-PBE: Assessing Data Privacy in Large Language Models", PVLDB 17(11):3201–3214, 2024.
Version v2
xPeerd (AI review)
AI Review (xPeerd — DAReview)
Summary
This manuscript presents LLM-Attack Suite, a framework for evaluating privacy leakage in split and distributed LLM systems by standardizing attacker access, recorded observations, controls, scoring, and evidence retention. The core contribution is methodological rather than purely algorithmic: the paper proposes a reusable evaluation procedure in which “compatible attacks reuse the same recorded messages,” candidate defences require a “linked undefended reference,” and saved predictions enable independent rescoring without rerunning the target model . The empirical sections support this framework with controlled GPT-2 and Qwen3-0.6B studies, including a notable result that adding returned gradients increases token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77% in one GPT-2 setting .
The manuscript’s strengths are its careful separation of protocol components, strong emphasis on retained evidence and rescoring, and extensive effort to distinguish implementation validation from substantive privacy findings . Its main weaknesses are scope inflation, uneven clarity about what is actually validated versus merely registered, occasional overstatement of framework generality relative to the strongest empirical evidence, and limited treatment of external validity beyond author-controlled systems . I do not see obvious paper-mill incoherence; the work fits a recognizable privacy/security evaluation paradigm. However, because many claims hinge on framework validity rather than a single attack result, the manuscript would benefit from sharper boundary-setting about which conclusions are firmly demonstrated and which remain contingent on untested attack families and environments.
Potential Major Revisions
- Clarify and narrow the manuscript’s central claim about what has been validated
A recurring issue is that the paper sometimes moves from validating parts of the evaluation machinery to broader claims about the framework’s reliability. For example, the abstract states, “We present LLM-Attack Suite, a framework that records these choices and lets researchers check, compare and extend an evaluation” and later adds that “The framework reports implementation checks separately from measured attack effectiveness” . That distinction is good, but the manuscript then broadens its claims in places such as “Taken together, the programme establishes consequential leakage under several independent protocols” and “Together, these studies establish reuse of the framework across reconstruction, robustness, obfuscation and agentic protocols” .
The problem is not that the authors lack evidence, but that the paper sometimes conflates:
- evidence that particular leakage results are real under stated conditions,
- evidence that certain modules execute correctly,
- evidence that the framework generalizes as an evaluation standard.
This distinction matters because the paper itself admits uneven evidence depth: “35 of the 64 carry synthetic-fixture qualification only” . That is a substantial fraction of the registered module set. The manuscript should explicitly separate:
- validated measurement code,
- validated attack implementations,
- empirically exercised protocol families,
- merely registered but not substantively evidenced modules.
Without this sharper partitioning, readers may overinterpret “coverage” as “validated operational capability.”
- State more explicitly the limits of external validity and deployment realism
The manuscript repeatedly and commendably notes that “All experiments used author-controlled systems” and that “Live endpoint modes require an explicitly supplied host, and the agentic planner accepts local model targets only” . Yet the broader framing often suggests implications for split/distributed LLM privacy more generally. For instance, the introduction asks, “What must an evaluation record and test so that another researcher can check its conclusion, compare it fairly and extend it?” . That is a strong methodological ambition.
The concern is that the strongest quantitative findings arise from highly controlled settings: GPT-2 124M, Qwen3-0.6B, fixed corpora, fixed split depths, and declared attack budgets . Even the paper acknowledges that “The strongest quantitative reconstruction comparisons in this paper concern GPT-2 124M and Qwen3-0.6B under the stated corpora, lengths and budgets” .
The manuscript should therefore revise its framing so that the main takeaway is not “this framework broadly evaluates split/distributed LLM privacy” but rather “this framework demonstrates a reproducible methodology whose strongest evidence currently comes from a delimited set of author-controlled protocols.” That revision would improve epistemic precision and reduce the risk of overgeneralization.
- Strengthen the justification of the decision rule and its interpretive status
Section 3 introduces a formal score rule and states, “The scoring rule asks how much text the strongest tested attack recovers above a simple guess” and that a candidate defence passes when “its largest excess is at most one percentage point” with supporting controls and reference conditions . The manuscript commendably admits that “the gate is an operational screening convention, not a distribution-free privacy guarantee” .
That admission is important, but it should be elevated and discussed more deeply in the main framing. At present, the paper gives the impression of a principled benchmark while the thresholding system is partly conventional and protocol-bound. The sentence “This multiplier originated in a five-percent error budget across nine known-plaintext probe attempts and is a fixed convention for other attacks” is especially important because it suggests historical carryover rather than a fresh derivation optimized for the broad framework now proposed.
The revision needed here is conceptual transparency: explain why this convention is appropriate for this broader evaluation framework, what types of inference it supports, and what types it does not. Otherwise, users may mistake a practical gating heuristic for a theoretically robust privacy criterion.
- Better reconcile “coverage” with untested attacks and incomplete threat closure
The manuscript repeatedly emphasizes that the framework reports untested attacks explicitly, which is a strong design choice. For example, “The coverage report marks applicable threats as covered, partial, unmeasured or out of scope” , and “A pass applies to the tested settings and attacks; the coverage report identifies what remains untested” .
However, this creates a tension with stronger interpretive language elsewhere. If the framework’s philosophy is that no privacy claim is complete without visibility into what remains untested, then the manuscript should discipline its own conclusions in the same way. This is especially necessary because the taxonomy is large: Table S16 lists 64 modules across many families, including agentic, model-IP theft, hardware/link, and application-profile threats . Yet the core empirical results center on reconstruction-focused protocols, with other areas often receiving qualification-tier or synthetic evidence rather than direct comparative privacy evaluation .
I recommend revising the claims so that “coverage” is presented as bookkeeping over a threat space, not as evidence that the threat space has been adequately tested. This is essential to preserve the manuscript’s otherwise careful methodological discipline.
- Clarify comparability across heterogeneous protocols and avoid “programme-level” aggregation language
One of the manuscript’s strengths is that it resists naïve aggregation. It explicitly states, “Aggregating different protocols into a single success rate would discard it” . Yet several sections still employ cumulative rhetoric such as “Taken together, the programme supports the following statement: LLM-Attack Suite has exposed consequential leakage under several independent protocols” .
This creates a subtle coherence problem. If the framework’s philosophy is that protocol components matter and conclusions can change when O, K, C, or B change, as defined in “E = (T, V, O, K, B, C, M, G)” , then “programme-level” summary claims should be made with greater caution. Otherwise, the manuscript appears to combine evidence narratively after warning readers not to combine it statistically or interpretively.
The revision should include a compact statement distinguishing:
- within-protocol conclusions,
- cross-protocol methodological lessons,
- non-transferable quantitative findings.
That would make the paper’s own synthesis consistent with its formal evaluation philosophy.
- Discuss reproducibility barriers created by gated evidence access
The manuscript argues that retained predictions allow “target-free rescoring” and that “all retained prediction transcripts can be verified and rescored on CPU without executing the target models” . This is a major strength. But the paper also states that the Hugging Face archive “uses gated access as a dual-use safety control for the large capture and prediction bundles” .
This may be justified ethically, but it introduces a practical reproducibility constraint that the paper currently underplays. A framework positioned as checkable and extensible should address how gated access affects:
- independent reproduction by skeptical reviewers,
- long-term availability,
- fairness of comparison by external attack developers.
The manuscript need not remove gating, but it should discuss the tradeoff directly and specify what parts of the validation remain openly checkable versus permission-dependent.
- Strengthen the manuscript’s novelty argument relative to existing frameworks
Table 4 positions the framework against VFLAIR-LLM, garak, Privacy Meter, and LLM-PBE across five evaluation capabilities, concluding that LLM-Attack Suite uniquely combines “Capture & replay,” “Mandatory gate,” “4-state matrix,” “Hash transcripts,” and “37 cases (MV1–7)” . This is a useful comparison, but novelty still needs more argumentative depth in the prose.
The introduction claims three contributions, including “A common evaluation procedure,” “Checkable privacy results,” and “Evidence for the procedure” . These are plausible contributions, but the manuscript should explain more directly why the combination is not just an engineering integration of known best practices. For instance, what is conceptually new about the mandatory undefended linkage rule? Why is the threat-model tuple itself a contribution beyond standard benchmarking metadata? Why is this framework necessary rather than a well-executed implementation wrapper around prior security-evaluation ideas?
At present the manuscript demonstrates utility more clearly than conceptual novelty. That may still be publishable, but the argument should be sharpened.
Potential Minor Revisions
- Tone down categorical phrasing where the evidence is narrower than the wording
Examples include:
- “We recommend that each privacy claim state the attacker, dataset and scoring rule, report its controls and identify untested attacks”
- “The suite also includes attacks on serving, applications and agents”
These are not incorrect, but they read broadly. Consider revising some statements to distinguish recommendation, implemented capability, and empirically demonstrated maturity.
- Reduce terminology density in the introduction
The introduction introduces several concepts in rapid succession: attacker observations, model knowledge, resources, datasets, controls, saved predictions, rescoring, and coverage. While precise, this density makes the early argument heavy. For example, “LLM-Attack Suite records what the attacker can observe, what it knows and the resources it uses” and “The framework requires a linked undefended reference before a candidate defence can pass its scoring rule” could be unpacked more gradually .
- Clarify the naming/history note earlier and more cleanly
The sentence “The suite was previously called LLM-Attacker. Existing software identifiers and frozen evidence may retain...” appears in the introduction material . Given the existence of another work cited as “LLM-Attacker” in autonomous driving, this renaming issue should be handled cleanly and prominently to avoid confusion with prior literature .
- Make the distinction between framework outputs more visually explicit
The manuscript says, “Three outputs answer different scientific questions. The scorecard evaluates the measured outcomes... The coverage report marks applicable threats... The module-validation matrix records each implementation’s qualification status...” . This is important and should perhaps be summarized in a compact visual box or a clearer in-text enumeration in the main paper, because it is central to correct interpretation.
- Explain the practical meaning of exact-document recovery earlier
The manuscript effectively notes that “A modest token gain yields substantially more complete documents” and reports exact-document recovery jumps such as “13.71% to 37.77%” . This is an important practical insight; it should be foregrounded earlier because many readers may underestimate the significance of apparently small token-level improvements.
- Tighten the prose around controls and comparator arms
Some descriptions are precise but cumbersome, e.g., “the joint attacker keeps the top 16 activation candidates and adds the top 16 tokens ranked by cosine similarity between the negated previous-position gradient...” . The technical detail is valuable, but the main paper may benefit from a simplified verbal summary before the exact mechanism.
- Typographic and stylistic issues
I did not detect many obvious typographical errors from the searchable excerpts alone, but a few stylistic matters deserve attention:
- Hyphenation and capitalization around terms such as “target-free rescoring,” “machine-readable,” and “author-controlled” should be checked for consistency across the paper .
- The manuscript alternates between compact symbolic exposition and plain-language explanation; transitions between these modes could be smoother, especially around Section 2.3 and Section 3 .
- Some long sentences in the contribution and cumulative-evidence sections may benefit from splitting for readability, especially where multiple studies are enumerated in a single sentence .
- AI-content analysis
Estimated percentage of AI-generated content: low to moderate probability of AI-assisted drafting, but not obviously machine-generated in a simplistic sense.
Rationale:
- The manuscript has unusually controlled definitional structure, repetitive enumerative phrasing, and polished, highly regular sentence construction.
- However, it also exhibits domain-specific technical specificity, coherent internal cross-referencing, and non-generic methodological distinctions that are not typical of low-effort AI-generated text.
- I do not see obvious hallucinated jargon clusters or meaningless abstraction. The paper remains scientifically legible and internally coherent across sections such as the threat tuple, scoring rule caveats, and validation properties MV1–MV7 .
Highlighted potentially AI-assisted sections:
- Highly templated contribution and cumulative-evidence summaries, such as “The contribution has three parts...” and “Taken together, the programme establishes consequential leakage...” may have been stylistically AI-assisted because of their polished genericity, though they are contentful .
- Comparative framing in Table 4 and the associated prose has a compact benchmark-marketing tone that may also reflect AI-assisted editing .
Assessed epistemic impact:
- If AI assistance was used, it does not appear to undermine the paper’s core contribution directly.
- The more important risk is rhetorical inflation: AI-assisted polishing may make claims sound more settled than the evidence warrants.
- Therefore, the impact is mainly on tone and claim calibration rather than on obvious substantive fabrication.
Recommendations
-
Reframe the manuscript explicitly as a methodology paper with bounded empirical demonstrations, not as a near-complete evaluation solution for split/distributed LLM privacy.
-
Add a concise “claim boundaries” subsection distinguishing:
- what is validated,
- what is exercised,
- what is registered only,
- what remains untested.
-
Revise the abstract and conclusion so that the strongest claims align with the strongest evidence, especially regarding generality and maturity across the 64-module taxonomy.
-
Expand the discussion of the scoring rule’s conventional elements and clarify that it is a practical benchmark gate, not a universal privacy guarantee.
-
Address the reproducibility tradeoff introduced by gated evidence access and specify the minimum independently auditable artifacts available without special access.
-
Sharpen the novelty argument by explaining why the integration of capture/replay, mandatory undefended linkage, retained predictions, and validation tiers is conceptually new rather than only operationally useful.
-
Preserve the manuscript’s strongest asset—its methodological discipline—by ensuring the rhetoric of the paper never outruns its own framework for cautious interpretation.
Reviewer3 (AI review)
AI Review (reviewer3.com)
Comments (8)
1. Coverage report states vs four-state matrix description — Editorial
outcomes under the declared rule. The coverage report marks applicable threats as covered, partial, unmeasured or out of scope. The module-validation matrix records each implementation’s qualification status as reference conformance, adaptation, real-model qualification, synthetic qualification or unmeasured capability. An execution failure is not a negative attack result, and
In Section 2.4 (page 4) and Section S6 (page 35), the coverage report is described as marking threats using four states: "covered, partial, unmeasured or out of scope" (COVERED, PARTIAL, UNMEASURED or OUT_OF_SCOPE). In Table 4 (page 13), the row "Explicit coverage report of untested threats" lists "4-state matrix" for LLM-Attack Suite. However, in the main text the term "matrix" refers to the "module-validation matrix" (Section 2.4, page 4), which has five qualification statuses ("reference conformance, adaptation, real-model qualification, synthetic qualification or unmeasured capability", or Q-REAL, Q-SYN, PARTIAL, UNMEAS, HELPER in Section S6, page 36). Calling the coverage report a "4-state matrix" conflates the 4-state coverage report with the 5-state module-validation matrix.
2. Module name mismatch for demixed joint attack — Editorial
Because the server differentiates through the mixed activation h mix = 0 . 75 h + 0 . 25 h decoy , unadjusted joint_true_k32 scores 83.24–89.26%, whereas a mixup-aware attacker that subtracts the best-matching calibration decoy activation ( 0 . 25 h cal ) recovers 93.36–97.39% of tokens with forward_k32_demixed and 96.95–98.72% (46.48– 66.13% exact documents) with joint_true_k32_demixed
In Section 5.1 (page 9) and Section S3.1 (page 21), the prose refers to the demixed joint attack configuration as joint_true_k32_demixed. However, in Supplementary Table S4 (page 22), which reports the exact breakdown of those same runs, the configuration is named joint_demixed_k32 (at both depth 3 and depth 6, public and trained).
3. Inconsistent baseline notation p_0 for Source code training — Editorial
Under z = 2 . 5392 , achieving a floor of at most 1 . 00 percentage point requires n a ≥ 639 tokens when p 0 = 0% (giving 1 . 00 pp at n a = 640 and 0 . 20 pp at n a = 3 , 200 in our horizon battery), n a ≥ 2 , 337 at p 0 = 2 . 64% (WikiText-2 inference), n a ≥ 3 , 534 at p 0 = 4 . 69% (WikiText- 2 training), n a ≥ 4 , 411 at p 0 = 6 . 25% (synthetic clinical templates), and
In Section 3 (pages 5-6), the paper lists the minimum token counts required to achieve a finite-sample Wilson floor of at most 1.00 percentage point for various corpus baselines: "na >= 2,337 at p0 = 2.64% (WikiText-2 inference), na >= 3,534 at p0 = 4.69% (WikiText-2 training), na >= 4,411 at p0 = 6.25% (synthetic clinical templates), and na >= 10,884 at p0 = 20.31% (source code)." While WikiText-2 distinguishes between its inference baseline (2.64%) and training baseline (4.69%), source code is listed only at p0 = 20.31% (its inference baseline from Table S6), omitting or conflating its distinct training baseline of 20.05% (154/768) reported in Table S7 (page 24).
4. In-sample baseline recovery conflated with calibration token frequency — Critical
For token reconstruction, that guess is the most common token in the attacker’s calibration text. The rule compares each attack’s upper recovery bound with this baseline, then takes the largest difference among attacks with valid measurements. A defence can pass only with the required
The authors define the baseline recovery rate in Equation (2) as the recovery of the most common token from the attacker's calibration text on the scored evaluation tokens, which is an out-of-sample empirical match rate that can be if that token does not appear in the evaluated passages (as occurs in the 20/100-step horizon battery) or vary across runs (as in the 500-step extension). However, in Section 3 (page 5) and Section S3.2 (page 24), when computing the minimum sample sizes required to achieve a pp Wilson floor, the authors substitute the empirical evaluation match rates (, , , , and ) directly into as if were a fixed, known binomial parameter rather than a sample proportion on the evaluated passages, and in the 500-step extension they switch from the calibration-text majority token to a "per-run majority reference of 4.83–4.96%". This inconsistency between a fixed calibration-derived guess token and an in-sample evaluated majority proportion changes the baseline subtraction across studies (for example, shifting the baseline on the same pilot corpus from in Table S8 to in the 500-step extension). The authors should explicitly distinguish in Equation (2) between the accuracy of the calibration-text mode on the evaluation set and the in-sample mode of the evaluation set itself, and use a single consistent baseline definition across the 20/100-step and 500-step horizon batteries.
5. Undefined contiguous 8-token span recovery metric on Qwen3-0.6B — Minor
At midpoint depth L = 14 , joint_true_k32 lifts candidate recall from 72 . 57% to 86 . 64% and token accuracy from 55 . 20% to 69 . 05% ( +13 . 85 pp , 95% CI [+12 . 38 , +15 . 27] pp ; +15 . 27 pp over joint_shuffled_k32 at 53 . 78% ) with public layers, and from 72 . 79% to 86 . 87% ( +14 . 08 pp, 95% CI [+12 . 82 , +15 . 41] pp; exact documents 0 . 05% → 5 . 27% , contiguous 8-token
In the Qwen3-0.6B midpoint split results, the authors report that joint_true_k32 increases "contiguous 8-token spans 19.21% -> 45.46%" with trained layers. However, unlike per-token recovery, exact-document recovery, and the pre-annotated semantic spans used in the GPT-2 study, the operational definition of contiguous 8-token span recovery (e.g., whether it is computed over all 25 overlapping sliding windows of length 8 per 32-token document or over 4 non-overlapping 8-token blocks, and why it is only reported for the trained condition and omitted from Table S5) is never specified in the text or supplementary material. The authors should define how contiguous 8-token span recovery is calculated and report it consistently across the conditions in Table S5.
6. Confounded comparison between stable and co-trained bottleneck tiers — Minor
averaged over three seeds) and the 3-width mean at each internal GPT-2 depth. Stable PCA and frozen bottlenecks attack training-time traffic with public client layers and the true compressor; the co-trained tier attacks final-state inference traffic with trained client layers and the final compressor.
In the 297-cell white-box compression study (Table S9), the authors compare fixed PCA and frozen bottleneck compressors (198 runs) against co-trained bottlenecks (99 runs). However, the two tiers simultaneously change three protocol variables: the compressor training regime (fixed/frozen vs. co-trained for 64 steps), the attacker's layer knowledge (public pretrained client layers vs. fine-tuned trained client layers), and the evaluated traffic split (training-time traffic vs. held-out final-state inference traffic). While the authors note that these are separate protocols, varying all three factors at once makes it impossible to determine whether the lower recovery at deep layers in the co-trained tier (e.g., 7.10% vs. 24.51% at depth 7, width 64) is caused by co-training the bottleneck or by evaluating on held-out inference passages with fine-tuned layers. The authors should either include a matched evaluation condition (such as attacking the frozen bottleneck on the same held-out inference passages with trained layers) or explicitly caveat that differences between the columns of Table S9 cannot be attributed solely to co-training.
7. Underspecified calibration-decoy demixing procedure under secret mixup — Minor
Because the server differentiates through the mixed activation h mix = 0 . 75 h + 0 . 25 h decoy , unadjusted joint_true_k32 scores 83.24–89.26%, whereas a mixup-aware attacker that subtracts the best-matching calibration decoy activation ( 0 . 25 h cal ) recovers 93.36–97.39% of tokens with forward_k32_demixed and 96.95–98.72% (46.48– 66.13% exact documents) with joint_true_k32_demixed
The authors claim that a mixup-aware attacker (forward_k32_demixed and joint_true_k32_demixed) that "subtracts the best-matching calibration decoy activation ()" restores token recovery to 93.36–97.39% and 96.95–98.72% under secret mixup. However, the mathematical criterion used to select the "best-matching" calibration decoy activation from the 32 public calibration passages (1,024 tokens) when the true activation is unknown is never defined in Section S2 or Section S3.1, nor is it explained why a decoy chosen from public calibration text should closely approximate a private decoy drawn from the victim's 256-row FIFO training-data pool at the exact token position. Furthermore, because the server differentiates through , subtracting from the forward activation does not alter the server's backward gradient , leaving it unclear how joint_true_k32_demixed demixes or adjusts the gradient signal. The authors should provide the exact equations for the demixing objective, clarify whether the calibration pool overlaps with the FIFO decoy pool, and explain how the backward gradient is handled in joint_true_k32_demixed.
8. Unadjusted confidence intervals reported for 15 secondary BiSR contrasts — Major
). Across 15 prespecified secondary contrasts, ten intervals lay below zero, one above and four crossed zero. The configurations
In the adapted BiSR comparison (Section 5.3 and Section S3.4), the authors evaluate 15 prespecified secondary contrasts alongside the primary contrast and base their conclusion on how many of the 15 intervals lie below, above, or cross zero ("ten intervals lay below zero, one above and four crossed zero"). Unlike the main v3 campaign where the 47 secondary contrasts are Bonferroni-corrected to control family-wise error, the manuscript does not state whether these 15 secondary bootstrap intervals are adjusted for multiple comparisons. The authors should clarify whether multiplicity correction was applied to the 15 secondary contrasts in the BiSR study and, if not, report Bonferroni- or Holm-adjusted intervals to maintain consistency with Section 4.1.
XSci (AI review)
Summary
Split language models keep text on a client but send activations to a server and, during training, receive gradients back; both messages leak. The authors present LLM-Attack Suite, a framework that records the target, attacker position, observations, knowledge, budget, corpus, metric and controls as an explicit protocol, captures split traffic once so compatible attacks replay the same messages, retains predictions for independent rescoring, and refuses a defence a pass without a linked undefended reference. It is exercised on a controlled GPT-2 124M campaign where adding returned gradients raises token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77%, and on a 304-cell Qwen3-0.6B benchmark where the gradient advantage reaches +21.37 pp at depth 21 with trained client layers. Further studies vary corpus, defence, horizon and passage length, compare an adapted port of BiSR against its reference, and validate the measurement code.
Contribution and Significance
The real contribution is a reporting and control discipline for split-LLM privacy claims, with a large body of controlled measurements showing why it is needed. The most valuable single result is not that gradients leak — prior work implies that — but that attack strength is relative to the declared budget: on GPT-2 the gradient advantage of +3.17 pp at k = 32 vanishes at k = 128 (98.10% against 98.16%, adjusted 95% CI [−0.15, +0.27] pp), whereas on Qwen3-0.6B's 151,936-token vocabulary it does not (joint at k = 32 recovers 69.05% against 57.18% for activation-only at k = 128). That contrast justifies the paper's thesis better than its headline numbers do.
Against the closest prior work the novelty is narrower than claimed. The joint attack's mechanism — gradient-derived token candidates fed into a beam search — is FILM's [9], moved to an activation seam. Relative to VFLAIR-LLM [7] and LLM-PBE [10] the delta is capture-and-replay, a mandatory undefended control and a coverage matrix, asserted in Table 4 without versions or anything run. And Section 1 and Table S3's caption state that both headline campaigns are shared with companion study [19] as "common reference benchmarks" drawing on "one evidence base", so the empirical content carrying the abstract is not new here.
Strengths
- The primary test is genuinely prespecified: threshold, bootstrap design, 100,000 replicates and seed 93281 are fixed in advance (Section 4.1), and the 47 secondary contrasts are Bonferroni-corrected.
- The negative controls behave as controls should: shuffled-gradient and random-candidate arms recover 93.91% and 93.89%, both below the 94.20% activation-only result, while the backward-only heuristic reaches 49.98% (Section 5.1).
- The authors break their own defence adaptively: secret mixup holds exact documents to at most 1.48%, but a mixup-aware attacker subtracting the best-matching calibration decoy recovers 96.95–98.72% of tokens and 46.48–66.13% of exact documents (Table S4).
- mid-lite passes the token gate at the finite-sample floor but with training cross-entropy 7.42–7.75 against 4.39–4.44 undefended, and is recorded as non-viable due to utility collapse (Table S8).
- The BiSR port matched an archived reference on 3,072/3,072 paired predictions in separate processes (Section 5.3), and the scorer itself is tested by 37 metamorphic and fault-injection cases.
Recommendation
Major Revision for PoPETs. The measurements are careful, the controls are real, and the budget-sensitivity finding matters. What is in doubt is the contribution as stated: the abstract rests on evidence shared with a companion paper, the framework's differentiation from existing suites is asserted rather than shown, the decision rule at its centre has never been shown to accept a usable defence, and the released artifact cannot reproduce the headline table through its registered interface. All four are repairable within the authors' own approach.
Major Concerns
-
What is new here, and the evidence for the framework claim (Section 1; Tables S3, S12, 4). Table S3's caption says "The same experiment supplies the companion paper's [19] table; both draw on this one evidence base", and Table S12 labels the first row "shared Paper A/B evidence" — yet both campaigns lead this paper's abstract. Please restate the abstract and contribution list around what is new, saying which numbers are reused from [18] and [19] and in what role.
The framework claim must then carry the paper, and Table 4 is its only evidence: five rows of two-word cells ("Split T/I", "Baselines", "Task grid", "Run logs", "Not reported") do not establish that VFLAIR-LLM [7], garak [4], ML Privacy Meter [15] and LLM-PBE [10] lack capture-and-replay, control-gated passes or coverage reporting. Name versions, state what was run, and ideally reproduce one shared evaluation in both VFLAIR-LLM and this suite. Please also engage Huang, Gupta, Song, Li and Arora, "Evaluating Gradient Inversion Attacks and Defenses in Federated Learning," NeurIPS 2021, vol. 34, pp. 7232–7241 — a prior evaluation-methodology paper with a released framework for this attack family, absent from the reference list.
-
The token gate is never shown to discriminate (Section 3; Tables 2, S4, S8). The gate is the framework's decision mechanism, but its multiplier z = 2.5392 "originated in a five-percent error budget across nine known-plaintext probe attempts", and the authors record that it "is not a simultaneous confidence guarantee over all 64 modules" and is "an operational screening convention, not a distribution-free privacy guarantee". Across the paper it exhibits one behaviour only: all three undefended corpora FAIL (Table 2), secret mixup and replica median leave 83.01–98.76% token recovery (Table S4), and the single pass is classified non-viable on utility. A rule with no demonstrated true positive cannot carry the framework; the revision needs either a defence that passes the gate and retains utility, or an explicit demotion of the gate to a screening heuristic with the "checkable privacy results" claim rewritten accordingly.
-
The headline gradient result is budget-conditional and the abstract omits the condition (Abstract; Section 2.2; Section 5.1). Widening the candidate list to k = 128 lifts activation-only recovery to 98.10% (50.16% exact documents) against 98.16% (51.45%) for the joint attack — +0.06 pp, adjusted 95% CI [−0.15, +0.27] pp — and the 150-cell longer-passage study repeats the convergence at 64 and 128 tokens. On GPT-2, what the gradient buys is candidate-shortlist recall that a larger forward budget also buys. The authors have the material for the correct claim, since the Qwen3-0.6B cells show the gap persisting at k = 128, but it must be stated as budget-relative in the abstract and conclusion, not only in the body.
-
The artifact does not support the checkability claim (Sections 2.4, 5.4; Supplementary S2, S4). The registered CLI module
attacker/attacks/seam_inversion.py"takes all k candidates at t ≥ 1 from the backward gradient score in its -arm joint mode", whereas every number in Table 1 comes fromattacker/reconstruction_baselines.pywith the matched k/2 + k/2 split of Equation (S3); exposing that split as a CLI flag is "recorded as open work", so a user cannot reproduce the flagship result through the framework. The evidence archive also requires "Hugging Face login and manual approval", which sits awkwardly with the claim that reviewers verify and re-score the evidence. Please register the matched joint arm and ungate enough evidence to rescore Tables 1, 2, 3 and S5.
Minor Comments
- Section 1 states that "35 of the 64 carry synthetic-fixture qualification only", but the Status column of Table S17 gives 40 Q-SYN modules — the 35 synthetic block plus vec2text-inversion, batch-label-recon, body-extraction, kv-timing and trace-inversion — against 16 Q-REAL, 5 PARTIAL and 3 HELPER.
- Section 5.2 and S3.3 give the width-128 range as "17.68–75.85%", but Table S9's largest width-128 entry is 75.13% (depth 2, frozen bottleneck), which its 68.59% mean confirms; Tables 1 and S3 likewise report 43.13% exact documents at depth three/public where Table S4 reports 43.12%.
- The text nowhere notes that gradient candidates can hurt: at Qwen L = 1, joint_true_k32 recovers 97.36% against 97.99% for forward_k32 with public layers (Table S5).
- The "analytic basis" claimed for the backward score in Section 4.1 cites iDLG [24], which concerns the final classification layer rather than an activation gradient backpropagated through server blocks, and Table S5 (backward-only 0.23% at L = 1 versus 73.67% at L = 21) shows the transfer is empirical.
- Section 4.1 states that "document-level rates are descriptive and carry no formal test", yet the abstract leads with 13.71% → 37.77%; supply intervals as Table S5 does for Qwen L = 21, or label them descriptive where quoted.
- The identifier-span range "90.00–100.00%" rests on n = 2 spans and should be dropped, and the name-span results should carry the AI-assisted-annotation caveat wherever quoted.
- Equation (1) introduces E = (T,V,O,K,B,C,M,G) but nothing is derived from it; either use it or present it as a reporting checklist.
- The active-server surface (Figure S1, marker B) rests only on synthetic fixtures in Table S17, and would benefit from Fowl, Geiping, Reich, Wen, Czaja, Goldblum and Goldstein, "Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models," ICLR 2023.
- The strongest reconstruction evidence stops at Qwen3-0.6B while the recommendations address split LLM deployments generally; one depth sweep on a 7B-class model would close that gap.
- Section S6 reports 336.1 hours of attack time for the main campaign but gives no guidance on what the gate costs on a new target.