XSci

Calibrated Generative AI as Meta-Reviewer: A Systemic Functional Linguistics Discourse Analysis of Reviews of Peer Reviews

Gabriela C. Zapata, Bill Cope, Mary Kalantzis, Duane Searsmith

Submitted by Gabriela C. Zapata

Published October 26, 2026 · Version v2, October 27, 2026 · DOI 10.66977/xsci.2610.002s

Linguistics, Higher Education, Educational Technology

Abstract

This study investigates the use of generative AI to support formative assessment through machine-generated reviews of peer reviews (meta-feedback) in graduate online courses in a public university in the United States. Drawing on Systemic Functional Linguistics and Appraisal Theory, we analyzed 120 meta-reviews to explore how generative AI feedback constructs meaning across ideational, interpersonal, and textual dimensions. The findings suggest that generative AI can approximate key rhetorical and relational features of effective human feedback, offering directive clarity while also maintaining a supportive stance. The reviews analyzed demonstrated a balance of praise and constructive critique, alignment with rubric expectations, and structured staging that foregrounded student agency. By modeling these qualities, AI meta-feedback has the potential to scaffold feedback literacy and enhance leaners’ engagement with peer review. We conclude that calibrated AI, when embedded in thoughtfully designed workflows, can contribute to hybrid human–AI feedback ecosystems, while also requiring critical attention to its limitations and contextual adaptation in educational practice.

View PDF

Review reports

Reviewer3 (AI review)

October 26, 2026

AI Review (reviewer3.com)

Comments (15)

1. Examples of "modulated interrogatives" are not interrogatives and match "offers" — Minor

AI as a cooperative partner. Particularly notable was also the employment of modulated interrogatives, which softened commands into collaborative proposals while also functioning as textual markers. Structures such as “ Happy to elaborate further if you’d like to continue the conversation! ” or “ Please feel free to reach out if you’d like a more detailed breakdown 19 or examples to refine your review further ” often signaled transitions toward concluding remarks, creating a natural closure to segments while maintaining a collegial stance.

In the "Textual Metafunction" section (pages 18–19), the text distinguishes between "offers" and "modulated interrogatives," yet uses the exact same grammatical structures to illustrate both categories, and neither example of a "modulated interrogative" is actually an interrogative. First, the text classifies "Let me know if you’d like a deeper explanation or examples—I’m happy to help further!" as an offer. Immediately afterward, it introduces "modulated interrogatives" and illustrates them with "Happy to elaborate further if you’d like to continue the conversation!" and "Please feel free to reach out if you’d like a more detailed breakdown or examples to refine your review further." Both of these examples are declarative/imperative statements with conditional clauses (not interrogatives) and have the same structure and function as the sentence just classified as an offer.

2. Table 2 mislabels circumstantial elements as "Processes" — Editorial

Table 2 Percentage of circumstantials in the corpus Processes Percentage of circumstantials Location 40% Manner/Means 32% Cause/Purpose 28%

On page 12, the coding scheme defines ideational features as consisting of two distinct components: "Process types (material, relational, mental, verbal) and circumstantial elements (location, manner/means, cause/purpose)." Table 1 (page 15) reports the percentage of process types under the column header "Processes." However, Table 2 (page 15), titled "Percentage of circumstantials in the corpus" and listing "Location," "Manner/Means," and "Cause/Purpose," erroneously reuses the column header "Processes" instead of "Circumstantials" or "Circumstantial elements."

3. Inconsistent citation for calibrated AI study: Tzirides et al. (2024c) vs. (2024b) — Editorial

  1. These changes positively influenced our participants’ perceptions of the AI reviewer, and they increasingly began to value its feedback for its precision, relevance, and actionability (Tzirides et al., 2024c; Zapata et al., 2024b). These findings suggested that

In the Introduction (page 3), the study demonstrating improved participant perceptions following RAG calibration is cited as "(Tzirides et al., 2024c; Zapata et al., 2024b)." However, in all subsequent sections discussing this exact finding—including "Educational Context" (page 10), "Discussion" (pages 21 and 22), and "Limitations" (page 25)—as well as in the References list (page 32), the study is cited as "Tzirides et al. (2024b)," and no "2024c" entry exists.

4. Missing total clause counts and undefined 'transitivity proxies' in Table 1 — Minor

Processes Percentage of transitivity proxies Material 51% Relational 21% Mental 16% Verbal 12%

Table 1 reports percentages of 'transitivity proxies' (51%, 21%, 16%, 12%) without reporting the total number of clauses or tokens (N) in the corpus or defining what constitutes a 'transitivity proxy' as distinct from a clause-level process. Reporting raw counts alongside percentages and specifying the exact lexical items used as proxies would make the quantitative summary transparent.

5. Conflation of prompt compliance with emergent discourse and interactive affordances — Critical

calibrated critique, scaled by graduation resources. Moreover, the AI’s dialogic openness, evinced by its use of hedges, modulated interrogatives, and explicit offers to elaborate, parallels the feedback conversations described in dialogically oriented work on metadiscourse 22 and engagement (e.g., Ädel, 2018).

Several features interpreted as evidence that the AI approximates effective human feedback and invites collaborative dialogue are either directly dictated by the metaprompts (e.g., 'Please address the reviewer by their name', 'An excellent review will be gently critical', 'Supportive, written in a collegial and seriously minded tone', 'Congratulatory') or assume interactive platform capabilities—such as closing offers to 'continue the conversation' or 'workshop one of your comments together'—without clarifying whether the CyberReviewer interface actually allows students to reply to the meta-reviewer.

6. Coding single clauses for multiple process types violates SFL transitivity — Major

Each clause in the 13 dataset was coded for one or more process types, with circumstantials tagged to indicate how the AI anchored its suggestions in context.

In Systemic Functional Linguistics, each ranking clause selects a single process type realized by its verbal group, yet the methods state that each clause was coded for 'one or more process types' and appear to rely on auto-coded lexical proxies. Clarifying whether clauses were segmented into ranking clauses or whether embedded/projected clauses were conflated with main clauses is necessary to interpret the transitivity distributions.

7. Unquantified prevalence of cohesive devices and Mirador's clinching moves — Minor

In terms of rhetorical organization, the reviews closely aligned with Mirador’s (2000) clinching pattern, moving from summary, to strengths, to weaknesses, and concluding with overall judgments. For example, reviews began with affirmation (“

While the methods specify that paragraph-level units across all 120 reviews were coded for marked themes, cohesive devices, and Mirador's (2000) six-move clinching pattern, no frequency counts or compliance rates are reported to show how many of the 120 reviews (or 360 criterion segments) actually contained all six moves or how often marked themes occurred.

8. Conflation of manner/means and cause/purpose circumstantials in examples — Minor

” Additionally, circumstantials of manner/means clarified how improvements might be achieved, while cause/purpose explained why certain changes would strengthen the review, as in “ This is a good start, but you could go further by identifying how those theories might reframe the argument

The text introduces a single excerpt—'This is a good start, but you could go further by identifying how those theories might reframe the argument'—to illustrate both manner/means and cause/purpose circumstantials, even though 'by identifying...' realizes manner: means only and contains no circumstantial element of cause or purpose. Distinct examples and clear operational criteria for distinguishing manner/means from cause/purpose are needed to validate the reported breakdown in Table 2.

9. Absence of frequency distributions for Engagement and Graduation subsystems — Minor

Beyond attitude, the reviews demonstrated dialogic openness through engagement resources 17 or the consideration of other viewpoints/analytic lenses to invite further interpretation, as seen in comments such as the following:

Although the methods state that all 120 reviews were systematically coded for Attitude, Engagement (monoglossic vs. heteroglossic), and Graduation, quantitative distributions are provided only for Attitude (Table 3), leaving claims about the systematic prevalence of heteroglossic engagement and graduation markers supported only by a few qualitative excerpts.

10. Misclassification of verbal, mental, and relational clauses as material processes — Minor

and knowledge construction. Comments such as “ You identify specific areas that need improvement such as…” and “ You point out that more support is needed to… ” framed feedback as doable steps in the writing process. Complementing this action orientation,

The illustrative excerpts used to exemplify material processes—'You identify specific areas that need improvement such as…' and 'You point out that more support is needed to…'—are in standard SFL transitivity analysis relational/mental ('identify') and verbal ('point out') processes rather than material processes of 'doing' or 'happening,' and 'Your review demonstrates a good understanding of…' is an inanimate abstract subject with a relational/symbolic verb rather than a mental process with a conscious Senser. If these common feedback verbs were systematically coded as material or mental processes, the 51% material process figure and the resulting claim that the AI predominantly casts students as active material agents are substantially inflated.

11. Missing raw counts and mislabeled column header in Table 2 — Minor

Processes Percentage of circumstantials Location 40% Manner/Means 32% Cause/Purpose 28%

Table 2 labels the circumstantial categories as 'Processes' in the left column header and provides only relative percentages (40%, 32%, 28%) without reporting the total number of circumstantial elements identified across the corpus or how many clauses lacked circumstantials altogether.

12. Declarative conditional offers misclassified as 'modulated interrogatives' — Minor

AI as a cooperative partner. Particularly notable was also the employment of modulated interrogatives, which softened commands into collaborative proposals while also functioning as textual markers. Structures such as “ Happy to elaborate further if you’d like to continue the conversation! ” or “ Please feel free to reach out if you’d like a more detailed breakdown 19 or examples to refine your review further ” often signaled transitions toward concluding remarks, creating a natural closure to segments while maintaining a collegial stance.

The structures cited as examples of 'modulated interrogatives'—'Happy to elaborate further if you’d like to continue the conversation!' and 'Please feel free to reach out if you’d like a more detailed breakdown or examples to refine your review further'—are grammatically declarative clauses and imperative clauses with conditional ('if') adjuncts, not interrogatives. Similarly, reporting that 'you' as topical Theme was 'nearly ubiquitous in all reviews (98%)' conflates review-level presence (at least one clause starting with 'you' in a multi-paragraph review) with clause-level ubiquity across the text.

13. Conflation of Judgment and Appreciation and misclassification of politeness formulae — Minor

constructive guidance. As summarized in Table 3, positive judgment and appreciation were nearly universal, appearing in comments such as “ Your review demonstrates a strong effort to guide the writer with thoughtful and aligned suggestions

In Martin and White's (2005) Appraisal framework, Judgment evaluates human behavior/character whereas Appreciation evaluates texts and artifacts, yet Table 3 collapses Positive Judgment and Positive Appreciation into a single row while separating Judgment– and Appreciation–, and the example given for positive Judgment/Appreciation ('Your review demonstrates a strong effort...') evaluates the text ('Your review') alongside student effort. Furthermore, coding formulaic epistolary openings like 'Thank you for your thoughtful and diligent review' as Affect+ (and 'thoughtful and diligent review' as affect rather than judgment/appreciation) inflates the prevalence of emotional affect in the corpus.

14. Asymmetric aggregation and imprecise reporting ('Rare') in Table 3 — Minor

Category

Table 3 switches the unit of aggregation from corpus-wide token proportions (used in Tables 1 and 2) to the 'Percentage of reviews' containing at least one instance of a category, and reports 'Affect– (negative affect)' simply as 'Rare' rather than providing the exact percentage and count out of the 120 reviews.

15. Omission of coder details, inter-rater reliability, and researcher positionality — Critical

To ensure systematic coverage, all 120 reviews were coded in full. MAXQDA’s lexical search and auto - coding functions were used to identify recurring markers (e.g., second - person pronouns, modal verbs, evaluative adjectives), which were then validated manually. Visualization tools (e.g., code frequency tables) supported the quantitative

The manuscript reports that lexical auto-coding in MAXQDA was 'validated manually' across the 120 meta-reviews, but it omits how many researchers performed the coding, how boundary ambiguities in SFL and Appraisal categories were resolved, whether inter-coder agreement was calculated, and how the authors' dual roles as developers of the CyberScholar platform and analysts of its output were managed. Providing these procedural details is essential to establish the trustworthiness and replicability of the discourse analysis and its reported quantitative distributions.

Author response

We have considered the 15 comments in the AI-generated report against the revised manuscript and the SFL and Appraisal frameworks guiding the study. The report does not document specialist SFL review, so its terminological and methodological objections have been assessed against our expertise in these frameworks. In what follows, we offer a description of how the recommendations have been addressed and incorporated in version 2 of the paper:
1. Classification of “modulated interrogatives”: We agree that the cited expressions are not interrogative clauses. The revised Textual Metafunction section now identifies them as offers realized through declarative and imperative structures containing conditional clauses. For example, “Happy to elaborate further if you’d like to continue the conversation!” expresses willingness to provide assistance, whereas “Please feel free to reach out…” is an imperative invitation. We have retained the interpretation that these expressions can also perform textual functions, since their recurrent placement near the ends of feedback segments helps mark closure. This distinction between grammatical realization, speech function, and textual organization is consistent with the SFL approach used in our study.
2. Column heading for circumstantial elements: We have corrected this editorial error. In the revised manuscript, the table reporting location, manner/means, and cause/purpose is Table 3, and its left-hand column is labeled “Circumstantial elements.” These categories are no longer presented as processes. The numbering changed because the Results section now includes an initial table reporting the AI’s rubric ratings (pp. 14–16).
3. Tzirides et al. (2024c) citation: We have added it to the reference list.
4. Process-type percentages, clause counts, and “transitivity proxies”: As explained in the revised Data Collection and Analysis section, the process type was identified at the clause level, and MAXQDA lexical search and auto-coding were used to locate recurring markers for subsequent manual validation by the paper’s authors. We have removed “transitivity proxies” from the table and now report “Percentage of process types in the corpus” (Table 2, pp. 15–16). The table presents the proportional distribution of the four process types, not a count of search terms. We acknowledge that adding an absolute clause denominator would provide further quantitative detail; that figure is not reported in the current manuscript, and we do not want to substitute a lexical-token count for a clause count.
5. Meta-prompt specifications, discourse realization, and interaction: We agree that meta-prompts are an essential part of the context in which these reviews were produced. Our analysis does not treat the system’s supportive and constructive stance as independent of those instructions. Appendix A makes the meta-prompts and sample outputs available, and the revised Discussion explicitly explains that the instructions requested supportive, balanced, and constructive feedback. Our contribution is to examine how these expectations were realized linguistically through transitivity, Appraisal, Theme, and rhetorical organization. We do not claim that such discourse arose without prompting (Discussion, pp. 25–26). We have also clarified that the corpus includes only the initial AI-generated evaluations, excluding later conversational turns that might have taken place between the AI and students. The AI's offers to continue the conversation are analyzed as linguistic resources expressing an interpersonal orientation and marking textual closure; their occurrence is not treated as evidence of student uptake or of an actual dialogue in the data analyzed.
6. Unit of process coding in SFL transitivity: We have addressed this recommendation in the revised methods section. Lexical search and auto-coding in MAXQDA helped identify candidate linguistic markers, but the analysis was not defined as a tally of isolated verbs irrespective of their grammatical function. The revised discussion also distinguishes the process expressed in a clause from interpretations of understanding or agency that may be realized through other grammatical constructions.
7. Cohesion, marked Themes, and Mirador’s clinching pattern: Our discussion of cohesion, thematic progression, and Mirador’s (2000) clinching pattern is an interpretation of recurring rhetorical organization, not a claim that every review contains all six moves in exactly the same form. The revised Textual Metafunction section states that the reviews “exhibited features associated with” the clinching pattern and describes the sequence illustrated by the excerpts. We have not represented this qualitative observation as a six-move compliance rate. The analysis of marked Themes and cohesive ties also explains how these resources function in the excerpts. We do not claim that a specific prevalence for each connective or Theme type. The 120-review corpus was analyzed in full, but we do not believe a frequency table is necessary for every discourse-semantic observation made in a mixed quantitative and qualitative analysis.
8. Manner/means versus cause/purpose: We have addressed this recommendation and have revised the Ideational Metafunction section. Our revisions now make the functional distinction explicit: manner/means specifies how an action may be accomplished, whereas cause/purpose specifies the intended outcome of the recommendation. We have retained the original circumstantial categories and percentages in Table 3.
9. Distribution of Engagement and Graduation resources: We have retained the distinction between quantitative summaries and qualitative interpretation in our analysis. Table 4 reports review-level coverage for the Attitude categories, whereas the Engagement and Graduation sections examine the linguistic work performed by particular resources in context.
10. Illustrative process classifications: The reviewer correctly identifies the need to distinguish grammatical process type from the broader interpretation of what a clause represents. We have revised the Ideational Metafunction section accordingly.
11. Table 3 totals and circumstantial reporting: The revised Table 3 clearly distinguishes circumstantial elements from process types. The table reports percentages of the circumstantial elements identified. We recognize that absolute counts and a clause-level denominator would add detail, but neither is supplied in the current version; we therefore avoid implying that the table establishes how many clauses have no circumstantial element.
12. Conditional offers and prevalence of second-person Themes: The revised Textual Metafunction section no longer refers to conditional declaratives or imperatives as “modulated interrogatives.” It explains their status as offers and examines how their placement can contribute to textual closure (pp. 20–21). We have also clarified the unit underlying the 98% figure for second-person “you”: the text now states that “you” as topical Theme appeared in 98% of reviews (p. 21). This is a review-level presence measure. It is not presented as a claim that 98% of the clauses in the corpus began with “you.” The discussion of repeated topical Themes concerns their cohesive function across the reviews in which they occur.
13. Judgment, Appreciation, and politeness formulas: We agree that Judgment and Appreciation must remain analytically distinct. The revised Theoretical Framework defines these resources separately, and the Interpersonal Metafunction section specifies that Judgment evaluates reviewers’ demonstrated capacities and practices, whereas Appreciation concerns qualities of the review as a text or product (pp. 11, 17–18). Table 4 retains a compact combined positive row for the reported review-level prevalence, but that presentation does not mean the two Appraisal categories are theoretically identical.
14. Review-level percentages and “Rare” negative Affect: Tables 2 and 3 report relative distributions of ideational features, whereas Table 4 is expressly headed “Percentage of reviews” and concerns whether the relevant Attitude categories appeared in the reviews. These are different levels of aggregation, and the table headings now make that difference explicit (pp. 15–17).
15. Coders, interpretive decisions, reliability, and researcher roles: The revised Data Collection and Analysis section clarifies that the MAXQDA-assisted identification of recurrent markers was “validated manually by the paper’s authors” (p. 13). It also specifies the units of analysis for ideational, interpersonal, and textual features, describes the use of qualitative memoing to record interpretive observations, and explains the complementary roles of quantitative distributions and close discourse analysis (pp. 12–13). We agree that decisions about coding boundaries and researcher involvement should be transparent, particularly in SFL and Appraisal analysis. We do not, however, claim an inter-rater reliability statistic or an adjudication procedure not documented in this manuscript. The article makes the research team’s prior involvement with CyberScholar explicit through its account of the preceding studies and the system’s development (Introduction and Educational Context, pp. 3, 9–10). Our interpretation rests on the stated linguistic categories and the analysis of examples in context; no claim of independent validation is made.

xPeerd (AI review)

October 26, 2026

AI Review (xPeerd — DAReview)

Summary

This manuscript examines AI-generated “reviews of peer reviews” in graduate education, using Systemic Functional Linguistics and Appraisal Theory to analyze a corpus of 120 GenAI meta-reviews generated within the CyberScholar platform. The paper’s central claim is that calibrated GenAI can approximate important rhetorical and relational qualities of effective human feedback, especially by foregrounding student agency, offering context-anchored suggestions, and organizing comments coherently. The manuscript is strongest when it clearly defines the educational context, the AI workflow, and the analytic framework, and when it acknowledges limits around empathy, generalizability, and contextual dependence. The study is also reasonably transparent about its coding categories and reports interpretable descriptive distributions such as “Material 51%,” “Relational 21%,” “Mental 16%,” and “Verbal 12%” in the transitivity analysis.

Its main weaknesses are not in writing clarity but in evidentiary scope and inferential reach. Several claims about educational value, actionability, and feedback literacy appear stronger than what discourse analysis alone can directly support. The manuscript itself concedes that it “does not directly capture students’ perceptions of or responses to the AI feedback,” which is a significant limitation given the practical and pedagogical conclusions drawn later. Overall, this is a coherent and potentially publishable manuscript, but it would benefit from tighter claim calibration, stronger methodological transparency, and more explicit separation between linguistic description and pedagogical impact claims.

Potential Major Revisions

  1. Align the strength of the conclusions with the actual evidence base.

The manuscript states that “the AI can externalize tacit instructor expectations and give reviewers a reusable template for future feedback episodes” and that GenAI meta-feedback can “serve as a powerful scaffold for feedback literacy.” However, the evidence presented is discourse-analytic, not behavioral, longitudinal, or intervention-based. This creates a recurrent overreach from description of linguistic patterns to claims about learner uptake, scaffolding, and development.

The paper does acknowledge in Limitations that the study “does not directly capture students’ perceptions of or responses to the AI feedback” and that future work should “triangulate linguistic findings with learner reflections.” That caveat is important, but it currently does not sufficiently constrain the claims in the Discussion and Pedagogical Implications. I would revise these sections so that claims about literacy development, uptake, or educational effectiveness are framed as plausible implications rather than demonstrated outcomes.

  1. Clarify the boundary between novelty and continuation of prior programmatic work.

The manuscript claims that “The present paper extends this line of investigation by focusing on GenAI-generated reviews of peer reviews, an area not yet widely investigated.” This may be a fair positioning statement, but the paper also repeatedly frames itself as part of an ongoing internal research program on the same CyberScholar/CyberReview ecosystem, the same instructional setting, and similar calibration logic. For instance, the introduction notes “In our most recent work (Zapata et al., 2025b), we sought to further understand...” before presenting the current paper as an extension.

The manuscript should more sharply articulate what is genuinely new here beyond a change in object of analysis from AI feedback on student work to AI feedback on peer reviews. Is the novelty empirical, theoretical, methodological, or infrastructural? At present, the contribution is credible but somewhat incremental. A revised introduction should specify exactly what gap is being closed and why discourse analysis of meta-feedback changes the field’s understanding in a way prior work did not.

  1. Increase methodological transparency and reproducibility of the coding process.

The Data Collection and Analysis section gives the coding categories and mentions MAXQDA functions, full-corpus coding, and manual validation. This is useful, but not yet sufficiently reproducible for a rigorous qualitative/discourse-analytic study. The manuscript should clarify:

  • who performed the coding,
  • whether coding was conducted by one or multiple analysts,
  • whether inter-coder agreement or adjudication procedures were used,
  • how clause segmentation decisions were made,
  • how ambiguous cases were handled,
  • how exemplar excerpts were selected for reporting.

Right now, statements such as “all 120 reviews were coded in full” and “validated manually” are too high-level to allow readers to assess robustness. Since the paper places substantial weight on proportional distributions and patterned discourse findings, the coding pipeline needs fuller auditability.

  1. Address possible circularity in the evaluative design.

The manuscript describes a calibrated AI system that was guided by “metaprompts designed to calibrate the AI’s evaluative stance and ensure alignment with pedagogical and disciplinary expectations,” and then analyzes outputs to conclude that the system is rubric-aligned, coherent, supportive, and pedagogically meaningful. There is a risk of partial circularity here: if the system is explicitly prompted to produce rubric-aligned, supportive discourse, finding those properties in the output is unsurprising.

This does not invalidate the study, but it should be addressed directly. The paper would improve if it clarified that the contribution is not discovery that calibration shapes outputs, but specification of how those calibrated properties are linguistically realized. That distinction is implied in places, especially where the manuscript says it aims to illuminate “the specific linguistic levers by which calibration improves perceived quality.” Make that the central methodological contribution more explicitly, and moderate any language suggesting stronger causal conclusions than the design supports.

  1. Strengthen the evidence chain from descriptive counts to interpretive claims.

The paper reports aggregate figures such as “Material 51%,” “Relational 21%,” “Mental 16%,” and “Verbal 12%,” and similarly distributes circumstantials by location, manner/means, and cause/purpose. These tables are helpful, but the inferential link between these percentages and broader claims about pedagogical quality could be better justified. For example, the manuscript claims these features help make feedback “actionable,” “usable,” and “legible as pedagogy.” Those are reasonable interpretations, but the paper should explain why the reported linguistic distributions are sufficient indicators of those constructs.

A stronger revision would include a more explicit analytic logic: what counts as evidence for actionability, supportiveness, or coherence, and why? Without this, some interpretive steps feel intuitive rather than demonstrated.

  1. Clarify sampling and representativeness.

The manuscript states that the dataset comprised “the entire corpus of 120 reviews of peer reviews carried out by the AI Assistant during summer 2025 in the participating graduate courses.” That is clear as a local sample description, but several questions remain unanswered: How many courses? How many students? How many reviews per student? Were all outputs retained, or were there exclusions? Were prompts stable across all cases? Did rubric criteria vary across assignments, or were they invariant?

Because the paper draws conclusions about the discourse behavior of a calibrated system, readers need to know whether these 120 outputs reflect one stable prompting regime or a heterogeneous set of prompt/task conditions. More procedural detail is needed.

  1. Make the anti-overclaiming boundary more explicit regarding human equivalence.

The abstract says GenAI can “approximate key rhetorical and relational features of effective human feedback,” and the Discussion reiterates that a calibrated GenAI reviewer can “approximate key discursive qualities of effective human feedback.” This is a defensible rhetorical claim if limited to textual features. However, because the manuscript also states that AI lacks “shared histories and the situated empathy that peers and instructors bring,” the wording should stay consistently bounded to discourse-level approximation, not functional equivalence in pedagogical practice.

I recommend revising repeated “approximate” claims so that they explicitly refer to identifiable linguistic and rhetorical features, not the broader educational function of human feedback.

Potential Minor Revisions

  1. Grammar, diction, and wording issues
  • In the abstract, “enhance leaners’ engagement with peer review” should be corrected to “learners’ engagement.” This appears to be a typographical error on page 1.
  • In the introduction, “the value of GenAI feedback lied not only in its accuracy” should be “lay not only in its accuracy.” This tense choice appears on page 3.
  • In the Results section, “evincing that the AI both validated learners’ analytic engagement” reads slightly awkwardly; “indicating” or “suggesting” may be stylistically smoother for this genre. Page 14.
  • In the Discussion, “These outcomes extend and help explain shifts we reported” is understandable but slightly compressed; consider separating “extend” and “help explain” into a more precise causal formulation. Pages 20–21.
  • The phrase “how calibrated GenAI sounds as pedagogy” is rhetorically interesting but colloquial relative to the rest of the paper; a more formal alternative may improve tonal consistency. Page 8.
  1. Formatting and presentation issues
  • Tables are presented as plain text in the extracted file, and it is not always fully clear whether journal formatting requirements are met for titles, column alignment, and consistency of labels. For example, Table 1 uses “Percentage of transitivity proxies,” while Table 2 uses “Percentage of circumstantials.” These labels are intelligible, but the terminology should be standardized and justified. Pages 14–15.
  • The manuscript would benefit from a concise schematic figure or workflow diagram showing the relation among student work, peer review, AI meta-review, rubric criteria, calibration inputs, and analytic outputs. The prose description is clear, but the process is layered and would be easier to follow visually.
  • Appendix A is mentioned as containing “an example of these reviews, including the metaprompts given to the AI,” but the manuscript’s central argument depends heavily on calibration details. Consider making key prompt components more visible in the main text, not only the appendix.
  1. Citation and literature-positioning refinements
  • The paper is well referenced, but because much of the justification relies on the authors’ prior studies, the manuscript risks appearing somewhat self-referential. Consider broadening engagement with adjacent external literature on AI feedback discourse, educational linguistics, and algorithmic evaluation in peer assessment.
  • Where strong claims are made about “feedback literacy,” “actionability,” and “support,” direct linkage to external conceptual definitions would make the argument more portable beyond the CyberScholar research program.
  1. AI content analysis

Estimated percentage of AI-generated content: low.

My assessment is that the manuscript does not display obvious signs of wholesale AI-generated authorship. The prose is conceptually coherent, terminologically stable, field-aware, and internally structured in a manner typical of experienced academic writing. The argument develops cumulatively, and the limitations section is appropriately specific rather than generic. That said, some recurrent phrasings are highly polished and formulaic, and there are localized stretches of rhetorically smoothed prose that could have been AI-assisted at the sentence level, particularly in the Discussion and Pedagogical Implications. Examples include phrases such as “the AI can externalize tacit instructor expectations and give reviewers a reusable template for future feedback episodes” and “we move beyond generic claims about ‘better prompts’ toward replicable, theory-grounded design principles.”

Estimated AI-generated content range: 10–20%.

Highlighted AI-susceptible sections:

  • Discussion pages 20–23, where several paragraphs use uniformly balanced, polished argumentative cadence.
  • Pedagogical Implications pages 23–24, where claims are rhetorically smooth and generalizable in a way often associated with AI-assisted drafting.

Assessed epistemic impact: Even if minor AI assistance was used for stylistic drafting, I do not currently see strong evidence that it materially compromises the manuscript’s scholarly contribution. The epistemic risk lies less in possible AI authorship and more in claim inflation relative to the study design.

Fraud / paper-mill warning

I do not see clear paper-mill traits such as incoherent concept mixing, fabricated technical terminology, impossible methods, or citation-pattern breakdown. The manuscript fits a recognizable educational research paradigm and is internally coherent. The main concern is inferential overextension, not fabrication. Still, because the paper evaluates outputs from a system developed and iteratively refined within the authors’ own research program, stronger transparency about prompts, coding procedures, and analytic decisions is important to reduce the appearance of confirmatory framing.

Recommendations

  1. Reframe the headline contribution more narrowly and more rigorously: Position the paper as a discourse-analytic account of how calibration is realized linguistically in AI meta-feedback, rather than as evidence that AI improves feedback literacy per se.

  2. Revise all educational impact claims so they distinguish among:

  • observed discourse properties,
  • inferred pedagogical affordances,
  • untested downstream learning outcomes.
  1. Expand the methods section with concrete reproducibility details:
  • coder identities/roles,
  • training process,
  • agreement or adjudication procedure,
  • segmentation rules,
  • coding examples,
  • criteria for excerpt selection.
  1. Add a short subsection explicitly addressing design circularity: Acknowledge that a calibrated system was expected to produce rubric-aligned and supportive discourse, and clarify that the manuscript’s true contribution is analytical specification, not surprise at the outcome.

  2. Tighten the novelty claim: State exactly what is new about meta-review discourse, and differentiate this paper from prior studies in the same program with more precision.

  3. Improve evidentiary discipline in the Discussion: Where the manuscript currently states or implies learner impact, replace with formulations such as “may support,” “appears to afford,” or “is consistent with the possibility that.”

  4. Correct the minor language issues and standardize table terminology and presentation.

Overall recommendation

Promising manuscript with a solid conceptual foundation and a coherent analytic frame, but it requires major revision before publication-level confidence is warranted. The central problem is not clarity or topic relevance; it is the need for better calibration between what the data show and what the manuscript claims.

Author response

We thank the reviewer for recognizing the coherence of the SFL and Appraisal framework and the value of examining GenAI-generated reviews of peer reviews as a distinct form of feedback. The responses below refer to the revised manuscript (version 2 of the pre-print). We have addressed several questions about the evidentiary scope and linguistic classification of the findings while retaining the study’s purpose as an analysis of discourse, instead of an investigation of student learning outcomes or an experimental comparison of AI configurations.
Major concern 1. Aligning conclusions with the evidence base: We agree that a discourse analysis of the initial AI-generated meta-reviews cannot demonstrate changes in students’ feedback literacy, subsequent uptake, or learning outcomes. The revised Discussion, Pedagogical Implications, Limitations, and Conclusion now distinguish the linguistic resources identified in the 120 reviews from the educational possibilities those resources might support.
Major concern 2. Novelty and relationship to prior studies: The present work is a continuation of our research program, and we do not claim that it introduces a new SFL framework, a different institutional setting, or a separate assessment platform. Its contribution concerns the object and function of the discourse analyzed. In our earlier work, the AI reviewed students’ academic projects; here, it evaluates students in their role as reviewers, responding to the feedback they provided to their peers. This shift is significant because the language under examination concerns the quality of students’ evaluative practices instead of the academic content of their projects. The revised Discussion makes this distinction explicit. The study also examines how process types, Appraisal choices, Theme, cohesion, and rhetorical organization are realized in this meta-feedback context. Its novelty is therefore mostly empirical and analytical: it extends an established linguistic approach to an additional level of formative assessment and explains what that analysis reveals about the discourse of AI reviews of peer reviews.
Major concern 3. Transparency of the coding procedure: The revised Data Collection and Analysis section identifies the corpus as 120 initial AI meta-reviews, comprising 360 criterion-based evaluation components, and specifies that the process type was identified at the clause level; interpersonal resources were examined at phrase or clause level, and textual resources over larger stretches of discourse. It also states that MAXQDA lexical searches and auto-coding were used to identify recurring markers, that these were manually validated by the paper’s authors, and that qualitative memoing supported the interpretation of examples. These clarifications are relevant to the reviewer’s questions about the unit of analysis and the relationship between automated retrieval and grammatical interpretation.
Major concern 4. The relationship between meta-prompt instructions and findings: The reviewer raises an important distinction between identifying a supportive, rubric-aligned tone and explaining how that tone is realized linguistically. We do not regard the presence of requested characteristics in the AI output as unexpected, nor do we treat prompt compliance itself as evidence of educational effectiveness. Appendix A makes the instructions visible, including the requests to address students collegially, offer constructive guidance, and assess their reviews against specified criteria. The revised Discussion now states explicitly that our purpose is to examine how such expectations were expressed through transitivity, circumstantial elements, Appraisal resources, thematic choices, and rhetorical organization. The analysis thus concerns the linguistic realization of a designed evaluative stance, not a causal demonstration that calibration independently produced those features. As the revised manuscript acknowledges, isolating the effect of calibration would require comparison with another system configuration, which was not the objective of this work.
Major concern 5. From descriptive distributions to linguistic interpretation: The percentages in the Results are intended to describe the distribution or review-level presence of linguistic categories, not to serve as stand-alone measures of pedagogical quality. We agree that a high percentage of material processes, for example, does not establish that students would find the feedback actionable. For this reason, the revised Results develops the qualitative interpretation alongside the distributions. Material processes are considered in terms of how students’ reviewing activities are represented; circumstantials are examined for their functions in locating a suggested change, specifying how it could be made, or expressing its purpose. Supportiveness is analyzed through the combination of evaluative Attitude, mitigation, and offers, while coherence is examined through thematic organization, referential ties, conjunction, and rhetorical sequencing. These are linguistically grounded interpretations of what the text does, not measurements of students’ comprehension, motivation, or eventual revisions. The revised Pedagogical Implications and Limitations sections maintain this distinction by presenting possible pedagogical uses as implications requiring further study.
Major concern 6. Corpus boundaries, criteria, and representativeness: The manuscript now defines the dataset more precisely as the complete corpus of 120 AI-generated reviews of peer reviews produced during the summer 2025 deployment in the participating graduate courses. Each exported record contained three criterion-based components—Constructive Advice in the Review, Weaknesses of the Review, and Strengths of the Review—yielding 360 evaluations, each with a rating and discursive comment. The revised Methods also makes clear that subsequent conversational turns were excluded, so the analysis concerns the initial output of the AI Assistant and not any later exchange with a student. These details establish the scope of the data and the common rubric components represented in the corpus. The manuscript does not report a breakdown by number of students, course sections, or records per student, or document whether every prompting parameter remained unchanged throughout deployment. Our conclusions concern the observed discourse of this configured system in its institutional context, and the revised Limitations expressly cautions against assuming generalizability to other disciplines or settings.
Major concern 7. Linguistic resemblance and human feedback: We agree with the need to distinguish discursive resemblance from functional equivalence with human feedback. Our claim about approximation concerns identifiable rhetorical and relational features of written comments described in the literature on instructor and peer feedback. The study did not include a corpus of human meta-reviews, and its findings do not establish that the AI performed like a human reviewer in an empirical comparison or provided equivalent relational support. The revised Discussion and Limitations explicitly state that the AI does not share students’ classroom histories or situated interpersonal experiences and that a supportive linguistic stance should not be equated with the relational depth of human interaction. We therefore retain the distinction between reproducing discourse features associated with effective human feedback and reproducing the broader educational relationship in which human feedback occurs.
Minor comment 1. Grammar, diction, and wording: We have addressed all these recommendations.
Minor comment 2. Tables, workflow presentation, and the metaprompts: The revised manuscript has standardized the relevant table terminology. The Educational Context and Data Collection and Analysis sections currently describe the relation between peer review, the AI Assistant’s rubric-based meta-review, and the SFL analysis, while Appendix A reproduces the meta-prompts and examples of the resulting ratings and comments. Because the meta-prompts themselves are primary contextual materials for interpreting the output, we have retained them in the appendix rather than reducing them to a schematic description. A figure could provide an additional visual orientation, but it is not necessary to establish the sequence or units of analysis already explained in the text.
Minor comment 3. Literature and conceptual positioning: The study builds on our preceding investigations because these establish the development of the CyberScholar feedback workflow and the research question addressed here. They are not the sole basis for the linguistic or pedagogical framing. The manuscript also discusses research on formative feedback, rhetorical organization, hedging, metadiscourse, and peer assessment, and the SFL and Appraisal scholarship that supplies the analytical categories. The Literature Review additionally discusses recent work on AI-supported peer reviewing. We agree that further work could connect our findings with additional conceptual accounts of feedback literacy and algorithmic evaluation. However, the present paper does not claim to test a theory of learner uptake or to measure an intervention’s effectiveness; it uses established linguistic concepts to describe how the AI meta-reviews enact evaluation and organize guidance. The revised Discussion and Conclusion now make that theoretical and evidentiary scope clearer.
Minor comment 4. Speculative assessment of AI-assisted authorship: We note the reviewer’s observation that the manuscript is conceptually coherent and does not present clear evidence of fabrication or paper-mill practices. The report also assigns a percentage range to possible AI-generated prose on the basis of stylistic impressions. No evidence is provided on how such a percentage was determined. We affirm that the authors were solely responsible for research design, data collection and analysis, and manuscript development.

XSci (AI review)

October 26, 2026

Summary

The manuscript examines a new stage in a university's AI-supported assessment workflow: generative AI reviews of student peer reviews ("meta-feedback"). Using Systemic Functional Linguistics and Appraisal Theory, the authors code the complete corpus of 120 meta-reviews produced by the CyberScholar AI Assistant in graduate Education courses in summer 2025, across ideational, interpersonal and textual categories. They report that material processes dominate transitivity (51%), that circumstances of location, manner/means and cause/purpose anchor suggestions, that positive judgment and appreciation appear in 92% of reviews alongside calibrated critique in 79%, and that vocatives and second-person topical Themes appear in 98%. The authors conclude that calibrated GenAI approximates the rhetorical and relational qualities of effective human feedback and can scaffold students' feedback literacy.

Contribution and Significance

The contribution is a descriptive linguistic profile of a form of machine feedback that almost no one has examined: AI commentary on student peer reviews, analysed as discourse rather than as output quality. The field has largely asked whether AI feedback is accurate or helpful; the authors ask what it sounds like, and operationalize that question into checkable categories. Given how quickly meta-feedback features are appearing in assessment platforms, a vocabulary for describing what such systems say to students is useful to designers and instructors.

What is genuinely new here is the object, not the apparatus. The framework, coding scheme, system, institutional setting and headline conclusion are carried over from the authors' own Zapata et al. (2025b); what changes is that reviews of student work become reviews of student reviews. The manuscript also understates relevant prior work when it claims, at the close of the Literature Review, that "none of these studies has explored how the language used by GenAI can enact evaluative stance, open/close dialogic space, scale force/precision". That is fair for the three tool papers cited (Guo et al., 2025; Liu et al., 2024; Su et al., 2025), but not for the wider literature: Yao, G., & Liu, Z. (2025), "Exploring artificial intelligence appraisal: Appraisal patterns in GPT-generated and human-authored book reviews", Applied Linguistics, amaf064, applies the same Appraisal subsystems to GPT-generated evaluative texts against a human comparison corpus, and reports limited interpersonal Engagement in AI writing — a finding the authors should engage, since it cuts against their dialogic-openness claim.

Strengths

  • The study analyses the complete population of 120 meta-reviews generated in the deployment rather than a sample (Data Collection and Analysis), which makes the proportional claims defensible as a description of this system.
  • Appendix A is unusually valuable: it publishes the metaprompts for all three criteria together with the AI's full outputs and ratings (3, 2, 4), letting readers see exactly what was asked and what was produced.
  • The observation that interpersonal resources do textual work is a real analytic finding, not a restatement of the design: vocatives, offers ("Let me know if you'd like a deeper explanation or examples") and modulated interrogatives are shown to mark transitions and closure (Textual Metafunction section; Discussion).
  • The Limitations section is honest about the two most important boundaries — the absence of learner uptake data and the system's inability to draw on shared classroom histories.

Recommendation

Major Revision for Assessment & Evaluation in Higher Education.

The object of study and the corpus are worth publishing, and the problems are repairable within the authors' own approach. But the two claims the title makes — that the system is calibrated and that it approximates human feedback — each name a comparison the design does not make, and the reporting of the analysis is currently too thin to assess. A revision that reports the analysis fully, reads the findings against the metaprompts, and scales the claims back to what a single-condition descriptive corpus supports would be a solid contribution here.

Major Concerns

  1. The design supports neither "calibrated" nor "approximates human feedback" (title; Discussion; Appendix A). The study has one corpus and one condition. No non-calibrated output is analysed here, so the claim that "calibration (via metaprompts and RAG) functions as a discourse-level constraint" is imported from Zapata et al. (2025b) rather than evidenced; and no human meta-reviews are coded, so "approximation" is established against prose descriptions in Hyland and Hyland (2001), Mirador (2000) and Pearson (2022), not against data analysed with the same scheme.

    The sharper difficulty is that the main findings are what the metaprompt requests. Appendix A instructs the system to "address the reviewer by their name" (vocatives: 98%), to be "Supportive, written in a collegial and seriously minded tone" and "Congratulatory" (positive judgment and appreciation: 92%), to be "gently critical" and "Balanced" (calibrated critique: 79%), and to "Suggest ways the review could be more helpful" (material processes: 51%; manner/means circumstances: 32%). As it stands the study documents instruction-following and interprets it as discursive achievement. The authors should either add a comparison set — an uncalibrated or minimally prompted baseline, or a set of human meta-reviews — or reframe the paper explicitly as an analysis of how prompt specifications are realized in discourse, including where the output exceeds or falls short of what was asked.

  2. The analysis is not reported in enough detail to assess or reproduce (Data Collection and Analysis; Tables 1–3). Table 1 reports percentages of "transitivity proxies", a term used once and never defined, while the method states that "Each clause in the dataset was coded"; no clause count, segment count or denominator is given for Tables 1 or 2. There is no reliability check of any kind — no second coder, no agreement statistic — although Appraisal coding is interpretively unstable and the MAXQDA auto-coding was "validated manually" by an unstated number of analysts, with the search strings unreported. The system is also unspecified: the model and version behind the summer 2025 AI Assistant are never stated (GPT-4 is mentioned only for earlier work), nor what the RAG pipeline retrieves over, nor how many students and courses the 120 meta-reviews come from. Percentages reported to the point require these bases.

  3. Several claims overstate the reported distributions (Table 3; Interpersonal and Textual Metafunction sections). Table 3 gives Judgment− at 79% and Appreciation− at 58%, meaning roughly one review in five contains no negative judgment and more than two in five no negative appreciation; yet the abstract reports "a balance of praise and constructive critique" and the Results say the reviews "consistently" combined praise with calibrated critique. The 92% figure is called "nearly universal" while 98% elsewhere is "nearly ubiquitous". Table 3 also enters "Rare" in a column headed "Percentage of reviews", and the text then reasons from it. The rhetorical-structure claim has the same problem: the reviews are said to align "closely" with Mirador's (2000) clinching pattern, but no frequency is given, and the pattern is described with six moves in the Literature Review, four in the Results and five in the Discussion. Reporting the distribution of structures actually observed, and matching the prose to the numbers, would resolve this.

Minor Comments

  1. The 0–4 ratings are part of every meta-review and are shown in Appendix A, but their distribution is never reported or related to the discourse features.
  2. Whether the AI's claims about each peer review were accurate is not examined, although "rubric-aligned" is asserted throughout.
  3. Results are pooled across the three criteria even though Appendix A shows the stance varying sharply between "Weaknesses" (rating 2) and "Strengths" (rating 4); a per-criterion breakdown would strengthen the analysis.
  4. Engagement and Graduation are listed as coded categories and discussed qualitatively but appear in no table.
  5. The uniformity of the output (98% vocatives, near-identical openers across 120 reviews) is read only as pedagogical consistency; the risk that students discount visibly templated feedback deserves a sentence.
  6. "Feedback literacy" carries the paper's pedagogical argument but Carless, D., & Boud, D. (2018), "The development of student feedback literacy: enabling uptake of feedback", Assessment & Evaluation in Higher Education, 43(8), 1315–1325, is not cited.
  7. The gap claim closing the Literature Review should be narrowed to AI-supported peer review tools, given the Appraisal work on GPT-generated evaluative discourse noted above.
  8. The description of Su et al. (2025) appears inaccurate: Reviewriter (note spelling) generates German-language instructions from a fine-tuned German GPT-2 and was evaluated with fourteen students for technology acceptance, not with computer science undergraduates writing feedback with and without the system.
  9. No ethics approval, consent or data-handling statement is given, although the corpus derives from identifiable coursework.
  10. Tzirides et al. (2024c) is cited in the Introduction but is absent from the reference list, which contains only 2023, 2024a and 2024b.
  11. Table 2's header reads "Processes" where the rows are circumstance types, and Table 3 merges positive judgment with appreciation while splitting their negative counterparts, preventing comparison of Judgment+ with Judgment−.
  12. Copy-editing: Ädel's name appears as "Ådel", "Ädel", "Adel" and "Àdel"; "Holmeier" in text versus "Holmeir" in the reference list; "Zapata, 2024b, 2025b" omits "et al."; and "leaners" and "the value of GenAI feedback lied" should be corrected.

Author response

We thank the reviewer for recognizing the contribution of examining AI-generated reviews of peer reviews as discourse and for identifying issues of scope and reporting that merit careful attention. The responses below refer to the final revised manuscript (uploaded as version 2). Our study analyzes the initial corpus of 120 AI meta-reviews; it does not include human comparison data, subsequent student interactions, or measured learning outcomes.
Major concern 1: We appreciate the reviewer’s recognition that the distinctive contribution of the study lies in its object of analysis: GenAI-generated reviews of students’ peer reviews. We agree that the present design does not compare calibrated and uncalibrated versions of the AI Assistant or directly compare AI meta-reviews with human meta-reviews. We do not claim that it does. “Calibrated” describes the configuration of the system whose discourse was analyzed, including the rubric-based meta-prompts and retrieval-augmented workflow described in the Educational Context and illustrated in Appendix A. Our earlier research investigated calibration in other feedback contexts, whereas the present study examines the linguistic realization of meta-feedback within this configured system. The revised Discussion makes this boundary explicit, stating specifically what our contribution is. We also agree that many recurrent features reflect explicit instructions. This does not invalidate a discourse analysis: the relevant question is how those instructions are realized in linguistic choices in the resulting texts, including the intersection of ideational, interpersonal, and textual meanings. We have retained Appendix A precisely to make the relationship between instructions and outputs available for examination. The possibility of further conversational engagement is treated as an offer represented in the discourse; the study does not establish whether students took up those offers or whether a particular interface enabled such exchanges.
Major concern 2. Analytical reporting, coding transparency, and system description: We agree that the quantitative distributions require a clear account of their units of analysis. The revised Methods now specifies that the process type was identified at the clause level; the earlier expression “transitivity proxies” has been removed, and Table 2 reports percentages of process types. The manuscript also states that the full corpus comprised 120 exported meta-reviews, each containing three rubric-based components, yielding 360 criterion-based evaluation components. It distinguishes clause-level process coding, phrase- or clause-level Appraisal coding, and textual coding of longer stretches, and explains that MAXQDA searches and auto-coding assisted identification of markers that were manually validated by the paper’s authors.
Major concern 3. Prevalence claims, Appraisal categories, and rhetorical sequencing: We appreciate the distinction between the presence of an evaluative resource in the corpus and its presence in every review. The revised Table 4 explicitly reports the percentage of reviews containing the listed Attitude categories. We interpret the combination of praise and critique as a recurrent pattern at the corpus level, not a claim that all 120 reviews exhibit identical combinations of evaluative resources. The manuscript also now clarifies that the 98% figure for vocatives and second-person topical Themes concerns their presence across reviews, and not the proportion of clauses realizing those features. Regarding rhetorical organization, the Literature Review explains Mirador’s six-move clinching pattern, whereas the Results describe features associated with that pattern, not universal conformity to every move. The claims in the revised Discussion and Conclusion have instead been qualified to emphasize recurrent discourse features and the limits of the descriptive analysis.
Minor comment 1. Distribution of the 0–4 ratings: This concern has been addressed in the revised Results. A new subsection, “Rubric Ratings in the AI Evaluations,” includes Table 1, which presents the ratings for all three criteria across the 120 meta-reviews. We also clarify that these are judgments assigned by the AI Assistant, not independent assessments of the quality of students’ reviews. The new analysis situates the linguistic findings within the distribution of the system’s own evaluations without treating its scores as externally validated measures.
Minor comment 2. Accuracy of the AI’s judgments and rubric alignment: We agree that the accuracy of individual judgments cannot be established from a corpus comprising only the AI’s initial meta-reviews. The revised Educational Context explicitly states that we did not compare those judgments with the underlying student reviews or projects. Accordingly, our use of “rubric-aligned” refers to the criteria and evaluative expectations that structured the system’s output, not to independently verified correctness. The present study only examines how the AI articulated evaluations and suggestions within the rubric-governed workflow.
Minor comment 3. Pooling across the three criteria: The three criteria are distinct, and we agree that a criterion-specific comparison of linguistic features could be informative. The revised manuscript now reports ratings separately Table 1, while the linguistic analyses characterize the complete set of meta-review texts. This aggregation corresponds to our research objective of describing the ideational, interpersonal, and textual resources used by the AI Assistant across its meta-feedback. The differing ratings in the example presented in Appendix A do not, by themselves, establish a systematic difference in linguistic stance across criteria. Such a claim would require criterion-level coding and comparison, which the present results do not report.
Minor comment 4. Engagement and Graduation reporting: The revised Interpersonal Metafunction section retains a qualitative analysis of Engagement and Graduation, distinguishing how conditional constructions, qualification, modal expressions, and scalar modifiers function in context. Table 4 summarizes the review-level coverage of Attitude categories; it is not presented as a complete frequency inventory of every Appraisal subsystem. We agree that separate quantitative distributions for Engagement and Graduation would add a different dimension to the study, but these figures have not been reported, and we do not assert a numerical prevalence for those resources beyond what was documented.
Minor comment 5. Uniformity, templating, and possible student responses: We agree that recurrent forms of address and recognizable openings can be interpreted as signs of standardized output as well as of a consistent rhetorical structure. The revised Discussion now links such regularities directly to the system’s meta-prompt specifications instead of treating them as emergent interpersonal qualities. Their textual and interpersonal functions remain appropriate objects of analysis: for example, vocatives can frame openings and closings regardless of whether the instruction to address students by name was explicit. Whether students experienced this uniformity as supportive, repetitive, or less credible cannot be determined from the AI-generated texts. The Limitations section explicitly notes that students’ perceptions and uptake were not examined, so we do not claim an observed response to the templated character of the feedback.
Minor comment 6. Feedback literacy and Carless and Boud (2018): We appreciate the suggestion to engage with Carless and Boud’s work on feedback literacy. The manuscript currently develops its pedagogical discussion through research on formative feedback, peer reviewing, evaluative practice, and student responses to feedback, including Nicol et al. (2014) and Van Zundert et al. (2010). We agree that a direct reference to scholarship defining feedback literacy would help sharpen this aspect of the conceptual framing. That citation has not been incorporated into the supplied revised manuscript, and we do not claim otherwise. Importantly, the Pedagogical Implications and Conclusion now distinguish the discursive modeling of feedback practices, which our analysis demonstrates, from students’ subsequent development of feedback literacy, which remains a possibility requiring further investigation.
Minor comment 7. Scope of the literature gap and AI Appraisal research: Our statement that “none of these studies” analyzes the linguistic realization of GenAI evaluation refers to the AI-supported peer-review studies discussed immediately beforehand—Su et al. (2025), Guo et al. (2025), and Liu et al. (2024)—and not to all research on AI-generated evaluative discourse. We agree that this scope should be explicit. The reviewer also draws attention to a study by Yao and Liu concerning Appraisal in AI-generated book reviews. That work, as described in the report, concerns a different genre and comparative object from AI feedback on students’ peer reviews, and thus, we do not deem its inclusion relevant for this study. Also, we do not claim that the present study is the first application of Appraisal Theory to AI-generated evaluative language. Instead, its contribution is the analysis of meta-feedback within this particular educational assessment practice.
Minor comment 10. Tzirides et al. (2024c) in the references: This omission has been corrected.
Minor comment 11. Table headings and Judgment/Appreciation aggregation: The circumstantial table has been corrected.
Minor comment 12. Copy-editing and names: We have incorporated all the reviewer's recommendations. Also, this study received ethics approval and informed consent was obtained from all participants. If this study were published in a journal, we would provide detailed ethics information.