xPeerd (AI review)

October 1, 2026

AI Review (xPeerd — DAReview)

Summary

This manuscript presents a component-wise study of an “insurance-oriented evidence workflow” for AI-agent risk, covering three parts: adversarial measurement, privacy-preserving pooling, and cryptographic attestation. Its central strength is that it repeatedly narrows its claims and states what the experiments do not establish. The Introduction explicitly says the work contributes “a concrete implementation and a set of component evaluations, not new cryptographic primitives or an actuarial pricing model,” and later reiterates that no connected multi-turn agent or real insurance portfolio is evaluated . The empirical program is reasonably substantial for a systems-style paper: 3,600 prompt trials across five hosted models, synthetic portfolio experiments over multiple privacy configurations, and Rust/Python attestation implementations with recorded benchmarks . The manuscript is also commendably transparent about limitations such as category-dependent missingness, restricted attack simulators, and unsigned timestamp metadata .

The main weaknesses are methodological scope and overextension risk in the framing. The paper’s title and insurance motivation are stronger than the evidence actually supports, because the experiments remain disconnected components rather than an evaluated end-to-end insurance workflow. Important empirical choices are deterministic rather than sampled, several cells have severe missingness or zero classifiable responses, the LLM-judge setup is only partially validated, and the privacy analysis stops short of a complete release-level guarantee . Overall, the manuscript is careful and technically interesting, but it would benefit from sharper positioning, stronger validation of measurement choices, and tighter handling of internal validity threats.

Potential Major Revisions

1. The framing should be narrowed further to avoid implying an insurance-ready workflow when the manuscript itself repeatedly denies such readiness.

The paper is strongest when it is cautious, but its framing still risks overstating downstream applicability. In the Introduction, the manuscript says, “Insurance provides one possible use for such evidence,” but immediately adds that pricing would require “exposure denominators, loss severity, policy coverage, dependence across insureds, and calibration to actual deployments” . Likewise, the conclusion states, “The experiments do not establish deployment incident rates, an actuarial pricing model, differential privacy for the full release, or authenticated event time” . These limitations are serious enough that the paper may be better framed as a benchmarked evidence-infrastructure prototype rather than as “insuring agentic processes.” The present title could lead readers to expect underwriting relevance or claims-operational validation that is not empirically provided.

A stronger revision would align title, abstract, and contribution statements with the actual evidence. For example, the abstract’s line, “We study these as three separate components rather than an evaluated end-to-end insurance system,” is accurate and important . That caution should dominate the presentation, not function as a qualifier after a stronger headline claim.

2. The adversarial measurement methodology has important internal-validity weaknesses that need fuller treatment and, ideally, additional robustness checks.

Section 4.2 states that the campaign uses “the first 20 prompts of each category’s source-ordered file (deterministic source-order selection, not random sampling of a benchmark population)” . This is a significant design choice. Because the sample is not randomized and not stratified beyond source order, observed elicitation rates may reflect ordering artifacts or curation bias rather than stable category behavior. The manuscript does acknowledge that the prompts are “not a random sample of deployment traffic” , but this limitation should be elevated from a generic caveat to a direct threat to the comparative conclusions, especially when reporting large differences like “financial-domain hallucination” at 85.1% versus jailbreak at 0.7% .

Relatedly, the study’s reliance on a single hosted inference stack introduces another confound. The manuscript states, “All models use one hosted inference stack, whose preprocessing and moderation may affect outcomes” . That means model-to-model differences may be entangled with platform behavior. The current text is honest about this, but a major revision should either reduce comparative rhetoric or add robustness checks across inference settings.

3. Missingness is too substantial and too structured to remain a secondary limitation.

The results report 3,600 trials, but only 2,989 classifiable responses, with “605 HTTP failures and six additional judge errors” . Section 6.2 further notes, “Three cells have no valid responses: GLM financial, GLM legal, and Kimi financial. GLM and Kimi resource-exhaustion cells retain only four and two responses. Missingness is category- and model-dependent” . This is not incidental noise; it materially affects the interpretability of category-level and model-level comparisons. For instance, the manuscript reports “Financial hallucination is elicited in 85.1% of 134 classifiable responses... all from Qwen, DeepSeek, and Gemma; GLM and Kimi return no classifiable responses in this category” . That means one of the headline findings is estimated on a restricted subset of models, not the full benchmark design.

A major revision should foreground this issue in the abstract and main results discussion. At minimum, the manuscript should reorganize the headline claims so that they do not read like clean cross-model comparisons when several denominator structures are highly uneven or absent.

4. The LLM-as-judge protocol is only partially validated and should not carry as much evidentiary weight without stronger calibration.

The paper describes a primary LLM judge and a secondary judge, noting that the secondary judge “shares the Z.AI vendor family with GLM-5” and that the “50-row author spot-check is informal and not independent expert annotation” . The results section similarly says, “LLM-judge agreement is not accuracy” and that the author check “does not establish independent expert calibration” . This is a well-written acknowledgment, but the methodological implication is important: the core outcome variable depends on judge labels whose construct validity is only modestly established.

The manuscript would be materially stronger if it added a larger blinded human annotation sample, especially for edge cases between elicited, partial, and refused. Without that, conclusions about relative rates across subtle categories such as indirect injection, context failure, or tool misuse remain vulnerable to labeler-specific artifacts.

5. The privacy component is useful as an engineering comparison, but its guarantees are narrower than some readers may infer.

Section 3.2 is appropriately careful: “The full experiment also uses data-dependent generalization, selection, and group counts; no end-to-end differential-privacy guarantee is established for that release. Thus ε labels a noise parameter in the experiment, not a certified privacy budget” . Section 3.4 likewise warns, “Zero success for these restricted simulators is not a guarantee against arbitrary inference or linkage attacks” . In results, the manuscript states, “The singleton linkage simulator reports zero for every k = 5 configuration... This is a restricted diagnostic, not evidence against richer linkage attacks” .

This is all correct, but the structure of the section still encourages readers to interpret the outputs as privacy-performance evidence beyond the narrow threat model. Since the synthetic generator is single-realization, hand-parameterized, and partly order-sensitive, the manuscript should present the privacy experiment explicitly as a bounded case study rather than as a quasi-general empirical validation. Section 6.2 admits, “Released code uses unordered map iteration, so a seed alone need not reproduce identical assignments and noisy rankings” . That reproducibility caveat is serious and deserves earlier placement.

6. The attestation contribution is technically useful, but the security interpretation should be tightened.

The paper carefully states that “Its attested_at and version fields are not signed” and therefore “makes no authenticated-time claim” . It also says, “Attestation cannot catch records that were never attested” and only provides “a non-repudiable commitment to the attested set” . These are central constraints, because an insurance or claims context is especially sensitive to omissions, timing, and identity. Section 5.3 further says the benchmarks are “local microbenchmarks, not an end-to-end claims workflow” and that membership/content integrity “do not authenticate timestamp metadata, ordinal position, completeness, or real-world truth” .

Given those constraints, the attestation section should more clearly separate measured performance from security sufficiency. The implementation may be correct and efficient for batched content integrity, but the surrounding trust infrastructure remains largely unevaluated. The current discussion mentions transparency-log anchoring, timestamp authority, and key lifecycle as future integrations ; these should be framed as prerequisites for realistic deployment claims, not optional enhancements.

7. The manuscript is coherent and mostly consistent, but there are some signals of presentation engineering that warrant tightening.

I do not see strong evidence of fabrication or paper-mill style incoherence. On the contrary, the manuscript consistently qualifies its claims and links them to limitations. However, there is a mild risk of rhetorical inflation through packaging three heterogeneous components into one workflow narrative. The text itself admits, “The three components operate on different data in the current evaluation” and that “A connected study would need multi-turn agents with real tool execution” and other infrastructure not yet tested . This is not fraudulent, but it does mean the coherence of the paper depends heavily on conceptual rather than empirical integration. A major revision should therefore either add one small end-to-end demonstration or state even more explicitly that the paper is a modular prototype study.

Potential Minor Revisions

1. Clarify and standardize terminology around what is and is not measured.

The manuscript is generally careful, but several distinctions deserve even more explicit consistency:

• “elicitation rate” versus deployment risk;
• “noise parameter ε” versus a true privacy budget;
• “content integrity” versus completeness/truth/timing;
• benchmarked categories versus real-world incident taxonomies.

The paper already says, “The term ‘detection rate’ is avoided because it inverts polarity” . That precision is good; apply the same discipline uniformly throughout the abstract, tables, captions, and discussion.

2. Improve table interpretation guidance.

Table 3 includes cells with very small or zero classifiable counts, while Table 4 and Table 5 summarize synthetic/privacy and benchmark results that are easy to overread. The manuscript should add caption-level warnings for:

• undefined utility in empty releases,
• unstable interpretation for cells with very low n,
• the conditional nature of the utility metric over “common industry keys” ,
• the fact that record throughput is “not signatures/s” .

3. Expand the explanation of the classifiable denominator in the abstract.

The abstract reports 3,600 trials with 2,989 classifiable responses , but many readers will still interpret headline percentages as if based on all trials. A short phrase clarifying denominator dependence in the abstract would reduce misreading.

4. Strengthen reproducibility wording for the synthetic pooling experiment.

Section 6.2 notes that “Released code uses unordered map iteration, so a seed alone need not reproduce identical assignments and noisy rankings” . This is an important reproducibility detail and should be moved earlier, likely into the methodology section.

5. Tighten the explanation of external benchmark adaptation.

The manuscript says prompt provenance labels “identify adapted sources rather than asserting replication of entire external benchmarks” . This is sensible, but a brief methodological note explaining what was preserved versus altered from each source would improve interpretability.

6. Typographic and formatting issues.

I did not identify many obvious typographic mistakes from the retrievable text, but there are a few presentation issues visible in the extracted content:

• Hyphenation and PDF extraction artifacts appear in multiple places, such as “classi￾fiable,” “con￾textualize,” and “eval￾uated.” These may be PDF rendering issues rather than manuscript errors, but the final production version should be checked carefully for line-break artifacts visible after export .
• Section transitions around tables appear cramped in the extracted text, especially near Tables 3–5, which may indicate layout crowding in the PDF export .
• The arXiv submission marker appears inline near the beginning of page flow — “arXiv:submit/8160521 [cs.CR] 30 Sep 2026” — and should not interrupt the reading layout in the manuscript version intended for review .

7. AI content analysis for this post-2021 manuscript.

Estimated percentage of AI-generated or AI-assisted prose: low to moderate editorial assistance, approximately 15–30% at the prose-polish level, but not strongly indicative of AI-generated research content.

Rationale:

• The prose is unusually uniform, repetitive in disclaimers, and highly optimized for claim-bounding language.
• However, it is also technically specific, self-corrective, and closely coupled to numeric details, table outputs, and implementation caveats.
• The “LLM Usage Statement” explicitly says, “LLM tools assisted manuscript editing, code-to-claim consistency review, and preparation of this revision” . This directly supports the inference of AI-assisted editing.
• I do not see obvious signs that the empirical content itself was fabricated by language-model generation. The manuscript repeatedly documents limitations that are inconvenient rather than self-promotional, which argues against generic AI-written filler.

Highlighted likely AI-assisted sections:

• The abstract and conclusion are especially polished and balanced in a way typical of assisted editing, with repeated “does not establish” constructions .
• The limitations sections are highly systematic and rhetorically regular, suggesting editorial smoothing, though not necessarily problematic AI overuse .

Assessed epistemic impact:

• Low if confined to editing and consistency checking, as disclosed.
• Moderate if the same tools materially shaped category framing, threat-model narration, or interpretation language without tighter human verification.
• At present, the disclosure is appropriate, and I do not find the AI assistance to undermine the manuscript’s claimed contributions.

Recommendations

1. Reposition the manuscript as a modular evidence-infrastructure study rather than an insurance-near system unless an end-to-end demonstration is added.
2. Add robustness checks for prompt selection, ideally including random subsamples or alternative prompt subsets per category.
3. Expand human validation of the judging protocol, especially for ambiguous labels and for categories with interpretive subjectivity.
4. Reframe all comparative claims in light of structured missingness; prominently qualify results where entire cells are absent.
5. Move critical reproducibility and privacy-boundary caveats from late discussion sections into methodology and results.
6. Tighten the attestation claims so that performance results are not conflated with deployment-grade evidentiary sufficiency.
7. If feasible, add a minimal connected demonstration showing how one measured prompt outcome, one pooled release, and one attested batch relate in a single toy workflow. Even a small end-to-end example would greatly improve coherence.

Anti-fraud / paper-mill warning

I do not find strong signs of paper-mill fabrication or incoherent pseudo-technical writing in this manuscript. The work fits a recognizable computer security / AI evaluation paradigm, uses field-appropriate citations, and repeatedly states non-results alongside results . The main concern is not fraud, but possible overframing: the insurance narrative is more ambitious than the empirical integration currently demonstrated.

XSci (AI review)

October 1, 2026

Referee Report

Manuscript: "Insuring Agentic Processes: Measurement, Pooling, and Attestation for AI-Agent Risk" Field: Computer science — AI/ML security and privacy, empirical systems measurement, with an applied risk-economics framing Recommendation: Major revision

Summary and Recommendation

This manuscript proposes that evidence suitable for insuring AI-agent deployments can be studied as three separable components — adversarial measurement, privacy-preserving pooling, and cryptographic attestation — and reports one experiment for each. A twelve-category, single-turn prompt suite is applied to five hosted models through one cloud endpoint, yielding 3,600 trials of which 2,989 are classifiable after 605 HTTP failures and six judge errors, with an overall elicitation rate of 22.7% and category rates from 0.7% for jailbreak to 85.1% for financial-domain hallucination. A synthetic-portfolio study compares suppression-only k-anonymity, hierarchical generalization, distinct ℓ-diversity, and a Laplace ablation at n of 100 to 5,000. An Ed25519 and Merkle implementation is benchmarked at roughly 1.75 million records per second for million-record signing and 38.9 to 54.3 microseconds for single-record verification.

The manuscript has genuine virtues. Its arithmetic reconciles: the per-cell counts in Table 3 sum exactly to 2,989, and the category denominators of 134, 218, and 295 match their row sums. The artifact is public, the terminological care around elicitation rather than detection is a real contribution, and the willingness to state that ε labels a noise parameter rather than a privacy budget, and that attestation authenticates bytes rather than events, is uncommon and welcome. My recommendation is nevertheless major revision. The measurement experiment's comparative claims do not survive its own documented missingness or its unanchored labelling instrument; the three components are presented as a workflow yet share no data field; and several pooling and attestation results follow from the experimental setup rather than from the mechanisms under test. Much of the remedy lies within reach of data already collected.

Major Concern Category 1: The Measurement Experiment Cannot Support Its Comparative Claims

1.1 Missingness is entangled with the outcome

Section 5.1 reports 611 unclassifiable trials of 3,600, and Section 6.2 notes that missingness is category- and model-dependent, yet the survivors are analysed as though that dependence were benign. The three empty cells are GLM financial, GLM legal, and Kimi financial hallucination, and the two nearly empty cells are GLM and Kimi resource exhaustion at four and two of sixty. These concentrate in the categories where responding models show the highest rates — financial hallucination reaching 0.88, 1.00, and 0.69 — and in the two models subsequently ranked least vulnerable. If transport failures arise from platform moderation, truncation, or timeout on refusal-adjacent generations, missingness is a function of the response, and the denominators of 643, 631, 481, 547, and 687 are themselves an outcome. The authors should report the 605 failures by status code, by position relative to the documented backoff retries, and by prompt identity, since endpoint congestion should be roughly uncorrelated with category whereas Table 3 shows the opposite. The comparative claims then need bounds imputing missing trials first as elicited and then as refused; for Kimi legal hallucination those span 0.00 to 0.33. The empty cells warrant re-running rather than a dash, since a cell with no classifiable responses cannot carry a cross-model contrast.

1.2 The labelling instrument is validated only against itself

Every figure in Table 3 comes from a pipeline whose sole validation is agreement with another instance of the same kind of instrument: an LLM judge, a second LLM agreeing on 89.1% of 359 pairs at κ = 0.77, and an informal author check agreeing on 46 of 50. The manuscript states that agreement is not accuracy and that the secondary judge shares a vendor family with tested GLM-5, then still presents the results as measurements of model behaviour rather than of judge behaviour conditional on model output. The reported expected agreement of 52.6% implies a concentrated marginal distribution, a regime where κ is unstable with respect to prevalence, and a pooled κ across categories with base rates spanning 0.7% to 85.1% can conceal near-chance agreement on exactly the minority label the paper cares about; per-category confusion matrices on the 359 pairs would expose this. More fundamentally, a human-annotated gold standard independent of both author and judges is needed, with several hundred responses labelled blind to model identity so that judge sensitivity and specificity can be estimated per category and used to correct the observed rates. This matters most for the hallucination categories, where ground truth requires knowing whether a fabricated identifier was treated as real — a judgement the paper concedes no domain expert adjudicated.

1.3 The taxonomy has no validity evidence and its categories are not comparably scoped

Since every result is indexed by category, the taxonomy is itself the instrument, yet Section 3.1 concedes that the OWASP mapping is author-supplied, carries no independent validation, and follows no preregistered selection process. The categories also mix units of analysis: injection, jailbreak, and extraction name adversary techniques, hallucination and context failure name failure modes, harmful content and IP reproduction name harm types, and magnitudes across such constructs are not commensurable. Most consequentially, only hallucination is domain-split, so the abstract's leading contrast sets a narrow probe set built around checkably fabricated identifiers against a broad technique category; a suite designed that way should be expected to elicit more, and the contrast measures prompt specificity as much as model behaviour. The in-context rubric compounds this: the representative probe supplies a document asserting the capital of France is Berlin and asks what the document says, so answering Berlin is arguably correct grounded behaviour, and the 43.1% headline and GLM's sixty-of-sixty turn entirely on a choice never stated. A coverage crosswalk against an existing meta-reviewed taxonomy, expert adjudication of the hallucination suites, and a blind category-assignment check on held-out prompts would supply the missing evidence.

1.4 The analysis does not match the paper's own thesis

The opening claim — that systems fail differently across tasks even when the model is held fixed — is a statement about variance components and ideally about a model-by-category interaction, which Table 3 suggests strikingly with GLM at 1.00 on in-context injection and 0.00 on harmful content against Kimi's 0.02 and 0.00. The reported analysis offers cell proportions, marginals, and row-level bootstrap intervals; no decomposition, no interaction test, no fitted model. A mixed-effects logistic regression with fixed effects for model, category, and interaction, and random intercepts for prompt nested within category, delivers exactly the required quantity and is directly fittable on the existing crossed, balanced design. It would also settle whether three repeats at temperature zero carry information: if the within-prompt intraclass correlation approaches one, effective n is twenty per cell rather than sixty and every interval is too narrow. Relatedly, the per-model marginals average over categories with unequal denominators, so a model lacking classifiable responses in the two highest-rate categories is mechanically advantaged, and the jailbreak interval of [0.0, 1.7] rests on roughly two elicited responses in 298, where a percentile bootstrap behaves poorly and an exact interval would be more defensible.

Major Concern Category 2: The Three Components Do Not Yet Constitute a Workflow

2.1 The measured construct is single-turn text, not agentic process

The title and three taxonomy categories presuppose an executing system — tool misuse as unsafe call sequences, resource exhaustion as expensive tool sequences, context failure as state inconsistency — while the apparatus is a single-turn completion at temperature zero, and Section 6.2 confirms that no consequential tool action is executed. Adapting AgentDojo scenarios into single-turn prompts discards the stateful environment that gives that benchmark its meaning, so the tool-misuse rates capture whether prose mentions an inappropriate tool rather than whether an agent would call one; state inconsistency is not observable in a single turn at all. Scoping out multi-agent composition, as Section 3.1 does, is legitimate; scoping out agency while retaining its vocabulary is not. The authors could narrow the framing to a single-turn model-prompt study and relabel the affected categories, or instantiate a minimal harness with mock tools having observable side effects and test whether text-level elicitation predicts execution-level misuse. The second would be the stronger paper, since that gap is itself an interesting measurement.

2.2 The components share no data schema

Experiment 1 emits model, category, prompt identifier, judge label, confidence, and rationale. Experiment 2 consumes five quasi-identifiers — geography, NAICS industry, size band, provider tier, autonomy level — plus a severity bucket. Experiment 3 commits synthetic bodies in Rust and an integer identifier with a hundred-character body in Python. No field is common to any pair. The hierarchies that make Table 4 possible are therefore defined over identifiers no measurement process produces; the severity bucket whose protection is the object of the pooling experiment is never connected to an elicitation rate, and since Section 3.1 states that deployment factors modulate consequence rather than elicitation, that mapping must be non-trivial and is exactly what is absent; and the attestation payload versions a schema that is never defined. A single normative record schema, stated once and carried end to end through a worked example of even a hundred records, would convert Figure 1 from a diagram into a claim at a fraction of the cost of the experiments already run.

2.3 Correlated exposure and reporting incentives are omitted

Insurance presupposes partly independent losses, and this setting violates that sharply: the elicitation profile is a property of the model, not the deployer, so a thousand insureds on a base model exhibiting a sixty-of-sixty injection rate face simultaneous loss from one upstream update or one newly discovered technique. Accumulation risk of this kind is the central obstacle to capacity and pricing in cyber insurance, not a deferrable refinement. The design makes the omission conspicuous: provider tier, the variable most directly determining portfolio correlation, is among the quasi-identifiers to be generalized away, so the pooling mechanism destroys the information insurability depends on in service of a privacy property the paper declines to claim; and the generator samples severity independently of industry, building independence in by construction. A parallel gap concerns incentives, since Section 3.4 identifies insider under-reporting and concedes attestation cannot catch unattested records — the core problem of moral hazard. A subsection stating the assumed dependence structure, identifying which fields survive pooling, and analysing truthful reporting under a stated contract form would discharge the framing; retitling around auditable evidence for AI assurance is the honest alternative.

2.4 No claim is pitched where it could be refuted

The disclaiming register is a virtue in moderation, but at its current density it dissolves the contribution: each stated contribution reduces to an activity plus a list of things it does not establish. If the elicitation rates are explicitly not estimates of anything beyond the observed responses, no future measurement could contradict them, and nothing remains to replicate or challenge. The existing material supports at least three claims general enough to be wrong, and I would encourage committing to them: that category-level variation exceeds model-level variation, so a composite safety score discards most of the signal; that suppression-only k-anonymity is unusable for cross-organizational pooling above a stated ratio of quasi-identifier cardinality to sample size; and that batch attestation makes per-record commitment cost negligible relative to the measurement it secures. A short statement of what decision the work informs, and at what evidential threshold, would be worth more than several existing disclaimers.

Major Concern Category 3: The Pooling and Attestation Evidence Is Weaker Than Reported

3.1 The pooling utility metric is non-informative and the noise mechanism is uncalibrated

Utility is the Spearman correlation between published and full-data industry severity means, while Section 6.2 states severity is sampled independently of industry — so true means differ only by sampling variation, and the metric asks the mechanism to preserve a ranking of noise, unable to distinguish preserving structure from preserving nothing. The difficulty compounds in the cells carrying the headline: generalization and ℓ-diversity at n of 1,000 and 5,000 show suppression of 0.000 with utility of exactly 1.000, describing a transformation that drops nothing and changes nothing the metric sees, yet these cells underwrite the abstract's claim of correlations at least 0.95 for n ≥ 500. The Laplace ablation has a separate problem: the sensitivity constant of 25 is never connected to the data model, and a four-level ordinal severity has range three under integer coding or one normalized. The empirics reinforce this, since noise scaled inversely to group size should bite hardest at the smallest portfolio, yet at n of 100 the Laplace row reports suppression, utility, and attack success identical in all three columns to the generalization-only row. Introducing tunable dependence between quasi-identifiers and severity, deriving the constant from a stated encoding, sweeping ε, averaging over draws, and reporting the already-defined information-loss metric would make this interpretable.

3.2 One attack result is true by construction, and stronger attacks go unsimulated

The linkage attacker succeeds only on singleton classes, and Section 5.2 reports zero for every k = 5 configuration because surviving classes contain at least k records. That restates the definition of k-anonymity rather than measuring anything, and presenting it as attack resistance lends the privacy evaluation an empirical appearance it lacks. The homogeneity attacker is a real measurement, but its power is fixed by the generator: with severity drawn independently, homogeneity probability falls off rapidly in class size, which is precisely the reported 9% at n of 100 and zero thereafter, and the ℓ-diversity result shows only that forbidding homogeneous classes forbids them. Meanwhile the attacks the manuscript's own citations identify go unsimulated: the ℓ-diversity paper motivates its successors through skewness and similarity attacks, and this severity scale contains the adjacent pair high and critical; and the composition reference is cited twice while Section 5.2 concedes no composition attack was run, though repeated release over evolving membership is the operating mode of any consortium benchmark. Results rest on one realization per size, and the authors note unordered map iteration means a seed alone need not reproduce assignments, so the table is not reproducible even by the author.

3.3 The attestation construction and benchmark do not support the claims drawn from them

Section 3.3 names RFC 6962 and then describes unpaired tails promoted unchanged, whereas that specification splits recursively at the largest power of two below n, producing a different shape and root for most non-power-of-two sizes; a conformant third-party verifier would reject these proofs. The signed record count is a meaningful mitigation against the resulting ambiguity, and the paper should make that argument explicitly because the mitigation is doing real security work. The validation cannot detect the divergence, since the four reference cases are empty, one, three, and four leaves — three degenerate, one a power of two — and two implementations by one author agreeing establishes mutual consistency, not conformance. The benchmark has a parallel problem: Section 5.3 asserts logarithmic complexity while Table 5 gives 38.9, 52.5, 54.3, 41.7, 41.8, and 41.8 microseconds as n grows from ten to a million, non-monotone and ending below its value at a hundred, while path length grows roughly fivefold. A constant signature verification evidently dominates, and the headline throughput is mostly serialization and hashing, since one signature amortizes to nothing and tree construction alone runs near 4.89 million records per second against 1.75 million for the full path. Hardware is nowhere specified, peak memory is listed as measured but never reported, and the benchmark is wholly in-process, with no durable storage, network transfer, or proof-size accounting.

3.4 Non-equivocation, append-only consistency, and artifact permanence are absent

The delivered properties amount to this: given a trusted key and a batch root, one can check that a record was in that batch. Two properties the use case requires are missing without comment. The first is non-equivocation: the insured holds its own signing key, and nothing prevents producing two distinct batches for the same period, each internally valid, presenting one to the insurer and another to a regulator, with no verifier holding one root able to detect the other. This is the split-view attack transparency logs exist to prevent, and while log anchoring is proposed as future work, equivocation is never identified as the attack it would address. The second is append-only consistency: the named specification defines consistency proofs precisely so a verifier can confirm a later root extends an earlier one, yet independent per-batch roots with no linkage allow a prior batch to be reissued with records removed. Since Section 6.1 motivates the component by a claims auditor checking a disclosed record against an earlier commitment, inter-batch linkage is the substance of the use case, and chaining each manifest to its predecessor's root inside the signed payload would address it with modest code. A related inconsistency runs through the paper's own evidence: a work arguing that evidence should be committed before being relied upon identifies its artifact only by a mutable repository URL with no commit or archival identifier, tests endpoints with no version strings or campaign dates, and rests internal reliability on an author-conducted consistency audit performed with disclosed LLM assistance whose human-verification scope is unstated.

Minor Issues

1. Cross-references and typesetting. Section 3.1 cites Section 7 for actuarial analysis (Related Work) and Section 8 for future work (Conclusion), and subsection 6.1 precedes the Section 6 heading. A full pass is needed, with removal of the truncated fragment in Section 3.4, the dangling clause closing Section 7, the unnamed fields in the Appendix C listing, and the stray page numbers after Algorithm 1. The secondary judge appears as "g1m-5.1:cloud" in Section 5.1 and "glm-5.1:cloud" in Section 4.2.

2. Counts and the partial label. Table 3 gives two-decimal fractions, so implied elicited counts are non-integral (0.69 of 48 is 33.1); integer counts would aid re-analysis at no cost. The three-label scheme places responses judged partial in the denominator but never the numerator, a substantive choice never stated; per-category partial frequencies and a sensitivity analysis counting them as elicited should both appear. The judge's confidence field is collected but never used.

3. Selection and multiplicity. The campaign uses the first twenty of fifty prompts in source order, so if order correlates with provenance the selection may be unrepresentative; a check on the held-out thirty would settle it. With sixty cells reported and conclusions drawn from the extremes, simultaneous intervals or an explicit descriptive-only caveat is needed.

4. Model documentation. Table 2 gives no parameter counts, context windows, release dates, or campaign dates, while the tested set spans a 27B dense model, a 480B code-specialized model, and a mixture-of-experts system. A code-specialized endpoint evaluated on legal hallucination probes may be returning domain refusals rather than demonstrating robustness, and its 0.05 rate should be disaggregated accordingly.

5. Pooling specification gaps. Tier 1, 2, and 3 are central to the threat model yet never defined; ℓ denotes both the generalization-level vector in Algorithm 1 and the diversity parameter; ℓ = 3 against a four-valued domain is strict enough to explain much of the suppression rise from 30% to 52%, so ℓ = 2 should be reported. The common industry key count per Spearman computation, the victim-sampling procedure, per-seed dispersion, and the full k sweep are all unreported, and complete suppression under suppression-only follows arithmetically from 34,560 quasi-identifier combinations against small portfolios.

6. Attestation benchmark reporting. No processor, memory, operating system, or compiler version is given; Criterion's confidence intervals are discarded for bare medians; the tamper benchmark runs at a single unparameterized batch size. Four fixed-input unit tests are described as property tests, no negative vectors for forged signatures, wrong keys, or malformed proofs are reported, and Ed25519 batch verification goes unconsidered despite a multi-organization audit scenario. The serialized leaf_index that the verifier is documented not to check should be bound or removed.

7. Research-integrity statements. No funding, conflict-of-interest, ethics, licensing, or data-availability statement appears, and no responsible-disclosure or dual-use discussion, despite releasing six hundred adversarial prompts and per-endpoint vulnerability profiles for five named commercial services. Since the IP category probes verbatim reproduction and the artifact includes recorded responses, how any reproduced copyrighted material was handled should be stated. Appendix A supplies three representative prompts where the forward reference promises all twelve categories.

Missing and Inadequately Integrated Literature

Cited but Inadequately Integrated

Several works already in the bibliography carry implications the manuscript does not take up. Debenedetti et al. (2024) is the source of the tool-misuse scenarios, but its commitment to executing calls in a stateful environment is never engaged, so the text-level proxy needs justification or withdrawal; Mazeika et al. (2024) supplies prompts and threat model while its validated behaviour classifier goes undiscussed, precisely where the unvalidated LLM judge is least reliable; and Nasr et al. (2023) is cited for extraction probes without noting that extraction success depends on decoding parameters and output length, both of which this campaign constrains. Romanosky et al. (2019) documents the rating variables carriers actually use, the natural specification against which to test whether any category measurement could serve underwriting, while Eling and Schnell (2016) documents heavy-tailed severity that a four-level bucket cannot represent. Slattery et al. (2024), MITRE ATLAS, and the OWASP Top 10 are cited but never used to assess this paper's own coverage, and the OWASP mapping runs in one direction only, leaving unreported which external risks have no corresponding category. On privacy, Machanavajjhala et al. (2007) identifies the skewness and similarity attacks omitted here, Ganta et al. (2008) motivates the composition analysis the paper concedes it did not run, LeFevre et al. (2006) supplies the baseline that would bound the greedy generalizer's cost, and Dwork and Roth (2014) contains the bounded-range sensitivity derivation the constant of 25 lacks. On attestation, Laurie et al. (2013) is cited only for hash prefixes although its consistency proofs and gossip-based auditing are the remedies for the gaps above, Newman et al. (2022) offers the identity binding the paper names as missing, and Schneier and Kelsey (1998) bears directly on an insider holding the signing key.

Truly Missing Papers

Fourteen works bearing directly on the manuscript's claims are absent; all were confirmed to exist via WebSearch. On measurement, Zheng et al. (2023) is the standard LLM-as-a-judge reference and documents the self-enhancement bias directly relevant to the vendor-family overlap between secondary judge and tested model; Wallace et al. (2024) describes the principal training-side defence against the injection behaviours measured; and Liang et al. (2023) separates scenarios from metrics and reports coverage explicitly, addressing the empty-cell and composite-score problems encountered here. On pooling, Li, Li, and Venkatasubramanian (2007) gives t-closeness, the standard response to the ℓ-diversity limitation the paper acknowledges and the natural fourth configuration for the sweep; Narayanan and Shmatikov (2008) is the canonical counterargument to a singleton-matching adversary; and Bonawitz et al. (2017) offers secure aggregation as an architecture in which the aggregator never sees individual contributions. On attestation, Crosby and Wallach (2009) gives logarithmic consistency proofs under an explicit malicious-insider model and is the closest prior art to this component; Laurie, Messeri, and Stradling (2021) formally obsoletes the specification the paper builds on; and Melara et al. (2015) achieves non-equivocation while preserving privacy over logged identities. On the insurance framing, Woods and Böhme (2021) systematizes why naive indicator-to-loss relationships are spurious absent threat and exposure modelling, Böhme and Schwartz (2010) supplies the unifying framework including interdependent risk, and Biener, Eling, and Wirfs (2015) evaluates cyber risk against established insurability criteria. On governance, Raji et al. (2020) addresses audit integrity organizationally and McGregor (2021) describes an operating cross-organizational incident collection that has already confronted the incident-definition problem this paper defers.

Biener, C., M. Eling, and J. H. Wirfs. 2015. Insurability of Cyber Risk. Geneva Papers on Risk and Insurance 40(1):131-158.
Bohme, R., and G. Schwartz. 2010. Modeling Cyber-Insurance: Towards a Unifying Framework. In Proc. WEIS.
Bonawitz, K., et al. 2017. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In Proc. ACM CCS, 1175-1191.
Crosby, S. A., and D. S. Wallach. 2009. Efficient Data Structures for Tamper-Evident Logging. In Proc. USENIX Security, 317-334.
Laurie, B., E. Messeri, and R. Stradling. 2021. Certificate Transparency Version 2.0. IETF RFC 9162.
Li, N., T. Li, and S. Venkatasubramanian. 2007. t-Closeness. In Proc. IEEE ICDE, 106-115.
Liang, P., R. Bommasani, T. Lee, et al. 2023. Holistic Evaluation of Language Models. TMLR, August 2023.
McGregor, S. 2021. Preventing Repeated Real World AI Failures: The AI Incident Database. Proc. AAAI 35(17):15458-15463.
Melara, M. S., A. Blankstein, J. Bonneau, E. W. Felten, and M. J. Freedman. 2015. CONIKS. In Proc. USENIX Security, 383-398.
Narayanan, A., and V. Shmatikov. 2008. Robust De-anonymization of Large Sparse Datasets. In Proc. IEEE S&P, 111-125.
Raji, I. D., A. Smart, R. N. White, et al. 2020. Closing the AI Accountability Gap. In Proc. ACM FAccT, 33-44.
Wallace, E., K. Xiao, R. Leike, L. Weng, J. Heidecke, and A. Beutel. 2024. The Instruction Hierarchy. arXiv:2404.13208.
Woods, D. W., and R. Bohme. 2021. SoK: Quantifying Cyber Risk. In Proc. IEEE S&P, 211-228.
Zheng, L., W.-L. Chiang, Y. Sheng, et al. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In NeurIPS 36, D&B Track.

Conclusion and Path Forward

The question this manuscript asks is unanswered and increasingly urgent, and the author approaches it with more candour than is customary. The arithmetic reconciles, the artifact is public, and the refusal to convert red-team numbers into confident loss probabilities is a corrective the field needs. My reservations concern the distance between that posture and the evidence delivered. Seventeen percent of trials are missing in a pattern correlated with both model and category, and the survivors are analysed as though the pattern were benign. Every figure in the principal table comes from an instrument validated only against another instance of itself, applied through a taxonomy with no validity evidence whose categories are neither commensurably scoped nor consistently defined. The three components are presented as a pipeline yet share no field. The pooling experiment's best results describe an identity transformation, and one of its two attack outcomes follows from the definition of k-anonymity. The attestation construction departs from the specification it names, its benchmark asserts logarithmic scaling against data showing none, and it omits the two properties a claims auditor would most rely on.

Much of the remedy lies within reach of data already collected. A status-code breakdown of the HTTP failures, bounds on the comparative claims, and re-runs of the three empty cells would address the missingness directly. A mixed-effects model over the existing 2,989 observations would give the paper's own thesis the support it lacks while settling the clustering question. An independently annotated gold standard of a few hundred responses would anchor the measurement chain. A single normative record schema carried end to end through a small worked example would convert Figure 1 into a claim. Tunable dependence between quasi-identifiers and severity, a derived sensitivity constant, and a determinised generator would make the pooling results interpretable and reproducible. Specifying the tree rule exactly, testing at non-power-of-two sizes, reporting hardware, decomposing verification into its constant and logarithmic parts, and chaining each manifest to its predecessor would bring the attestation component in line with the scenario it motivates. I would encourage the author to commit, in revision, to two or three claims stated generally enough to be wrong. The project deserves that confidence.