Assessing a Mathematical Model of Syllable Production via CTW Alignment with EMA Data
Frédéric Berthommier
Abstract
This study evaluates a mathematical model of syllable production using an EMA dataset from a recently published study, which proved highly compatible with the model's architecture. The data set consists of regularly structured French phrases of equal duration and reduced phonetic and syllabic complexity. A dedicated procedure was used to transform EMA recordings into Maeda parameters. The Model-generated trajectories were then realigned with these transformed data using Canonical Time Warping (CTW). Statistical validation was conducted via a permutation test, comparing alignment quality against a surrogate condition with phonetically incongruent model outputs. The results show significantly better alignment for phonetically congruent pairings, revealing a strong structural correspondence between the model and the articulatory data. These findings provide experimental support for the representational validity of the model.
Review reports
xPeerd (AI review)
AI Review (xPeerd — DAReview)
Summary
This manuscript, titled Assessing a Mathematical Model of Syllable Production via CTW Alignment with EMA Data, presents a focused and technically coherent attempt to validate a deterministic articulatory model against EMA-derived articulatory trajectories using Canonical Time Warping (CTW). The central strength is the explicit and interpretable modeling framework: the paper clearly states that it combines “(i) explicit parametric generation, (ii) asymmetric parametric temporal alignment, and (iii) statistical validation of parametric and formant correspondence” in order to test whether the model captures observed syllabic organization . The reported results are internally consistent: congruent alignments outperform incongruent ones across all six articulatory parameters and across formants F1–F3, with corrected significance tests reported .
The main weaknesses concern the limited empirical basis and the validation logic. The dataset is restricted to “nine repetitions of seven different phrases recorded with EMA by a single speaker” , and the phonetic inventory is reduced to four vowels and two consonants, while some database consonants are treated as “phonetically congruent” substitutes rather than directly modeled . In addition, the preprocessing and renormalization steps partially bias the empirical trajectories toward the model’s statistical range, which weakens the independence of the validation signal . Overall, the work is promising, methodologically interesting, and reproducible in spirit, but the current evidence is better interpreted as a constrained proof-of-concept than as strong validation of representational validity.
Potential Major Revisions
- The empirical scope is too narrow to support the strength of the validation claim.
The manuscript concludes that “These findings provide experimental support for the representational validity of the model” in the Abstract and later states in the Conclusion that “This experiment demonstrates that articulatory parameters can be generated by a mathematical model and aligned with natural articulatory data” . I find this claim stronger than the evidence presently warrants. In Section 2.1, the empirical basis is explicitly limited: “The empirical dataset consists of nine repetitions of seven different phrases recorded with EMA by a single speaker” . That is a highly constrained design with one speaker, fixed-duration phrases, and regular syllabic structure. This supports a carefully bounded demonstration, but not broad validation of representational adequacy.
The paper should narrow its central claim or add evidence showing that the approach generalizes across speakers, broader phonetic inventories, less regular prosody, and more naturalistic utterances. As written, the argument risks conflating successful alignment under a controlled, structurally favorable design with genuine model validity.
- The validation pipeline is not fully independent from the model being validated.
A substantial concern lies in the preprocessing step in Section 2.1. The manuscript states: “the normalized empirical data, denoted CS, are then rescaled using the mean and standard deviation computed from the corresponding synthetic trajectories produced by the syllabic planning model” and gives the transformation “CSrenorm = CSnorm · σmodel + µmodel” . The manuscript itself acknowledges that “This operation biases the empirical data toward the statistical range of the model” .
That admission is important. When the empirical data are transformed using model-derived moments prior to dynamic alignment and evaluation, the resulting comparison is no longer cleanly external. Even if this is done for practical reasons, it reduces the independence of the validation procedure. The problem is not merely technical; it bears directly on the epistemic status of the paper’s main claim. The revised manuscript should quantify how much the reported condition differences survive without this renormalization, or under alternative normalization schemes that are not model-conditioned.
- The manuscript relies on an alignment procedure that may absorb mismatches rather than test genuine predictive adequacy.
The paper emphasizes that CTW is applied “asymmetrically to warp the model-generated Maeda trajectories onto the temporal grid of the EMA-derived Maeda parameters” and further notes that this “may introduce substantial local temporal distortions” while preserving overall syllabic structure . Later, the Results section states that “CTW globally compensates for most structural discrepancies in condition B” and that it even resolves discrepancies by “deleting the syllable /di/ in condition B through many-to-one temporal mapping” .
This is methodologically interesting, but it also raises a core interpretive issue: if the alignment procedure can absorb substantial structure mismatch, then improved post-alignment correspondence does not necessarily imply that the generative model predicted the observed articulatory dynamics well. Rather, it may imply that CTW is powerful enough to recover similarity under a flexible mapping. The revision should explicitly separate what is attributable to the model from what is attributable to the alignment procedure. For example, stronger baselines, ablation tests, or constraints on path flexibility would help.
- The paper’s phonetic substitutions and reduced inventory weaken the interpretability of “phonetic congruence.”
Section 2.2 states that “The phonetic inventory is limited to four vowels /e, a, i, u/ and two consonants /d, g/, which are considered phonetically congruent with /s, t, k/ present in the database” . This substitution is central to the experiment, yet its consequences are not fully unpacked. The manuscript’s Condition A / Condition B comparison depends on a distinction between phonetic congruence and incongruence, but some of the so-called congruent material is already only approximately related to the recorded phonetic content.
That choice may be reasonable within the model’s representational constraints, but it complicates interpretation. The paper should discuss whether the observed gains reflect true phonetic matching, broad syllabic similarity, or artifacts of inventory compression. A more explicit justification of the substitution logic is needed, especially because the manuscript later argues that “phonetic congruence drives significantly better articulatory and acoustic correspondence” .
- The model’s physiological claims are ambiguous and should be more carefully bounded.
The introduction commendably distinguishes levels of articulatory transparency. However, it also states that “the purely mathematical formulation of our model appears to contradict this level of physiological fidelity” with respect to Level 4 transparency . This is an honest and useful limitation. But the manuscript then draws conclusions that may still be read as stronger biomechanical support than is justified, especially because Maeda parameters are described as approximating “biomechanically meaningful control variables” .
The revision should make clearer that the model is being validated at a representational or structural level, not at a physiological-control level. Without such clarification, readers may overinterpret the findings as evidence for motor-planning realism rather than for restricted articulatory alignability.
- Statistical reporting is directionally persuasive but still incomplete for a validation paper.
The manuscript reports paired t-tests with Holm correction and gives effect sizes in the form of mean differences and t-values for parameters and formants . That is helpful, but not yet sufficient for a strong validation study. Confidence intervals, permutation-test details, exact p-values, assumptions checks, and perhaps cross-validated or held-out analyses would improve rigor. Because the study’s argument depends on a contrast between congruent and incongruent pairings, the paper should show the robustness of that contrast under alternative statistical summaries and potentially under speaker- or phrase-level resampling if possible.
- Anti-fraud / coherence assessment
I do not see obvious paper-mill markers such as meaningless jargon strings, citation padding without thematic relevance, or grossly incoherent argumentation. The paper is conceptually coherent, the methods are traceable, and data/code availability is explicitly stated . However, there is a softer integrity risk: the central conclusion may overstate what the experiment shows relative to the narrow design and partly model-conditioned preprocessing. This is not evidence of fraud, but it is a credibility risk if left unqualified.
Potential Minor Revisions
- Terminology consistency
The manuscript alternates between “Dorsum” and “Drsm.” In the equation block, the parameter is defined as “Drsm = TBx + TBy + TDx + TDy” , whereas the Results section refers to “Dorsum” among the six articulatory parameters . The terminology should be standardized.
- Copyediting and typographic details
The manuscript contains several visible hyphenation and spacing artifacts in extracted text, which may originate from typesetting rather than author error, but they should still be checked in the source file. Examples include split words such as “realign- ment,” “sys- tematic,” “pa- rameters,” and “con- gruent” as visible in the extracted passages . If these appear in the rendered PDF only due to line breaks, no correction is needed; if they persist in source or export metadata, they should be cleaned.
- Clarify naming consistency for variables and conditions
The manuscript uses “CS,” “CSnorm,” and “CSrenorm” in close succession in Section 2.1 . The notation is understandable but somewhat dense for readers. A brief notational table or a more explicit verbal summary would improve readability.
- Clarify whether “copy synthesis” is standard or manuscript-specific usage
The text says, “This process is referred to as copy synthesis in the following” . Because “copy synthesis” can be used differently in speech literature, the manuscript should specify whether this is conventional terminology or a local label for the EMA-to-Maeda-to-formant reconstruction pipeline.
- Section-level exposition could better separate rationale from implementation
The Introduction and Methodological details sections are intellectually rich but compact. In particular, the move from transparency levels to the present experiment is somewhat compressed . A short paragraph explicitly stating the testable hypothesis would improve flow.
- References and reproducibility wording
The data availability statement is commendably explicit: “The data and materials used in this study are openly available” and both Zenodo and GitHub links are provided . However, the Acknowledgments state, “I would like to thank Pascal Perrier… for personally sharing the data with me” . This is not necessarily inconsistent, but the chronology and exact openness status should be phrased clearly so readers understand whether all reviewed materials were publicly accessible at submission time.
- AI content analysis for this post-2021 work
Estimated percentage of AI-generated content: low, approximately 5–15%.
Rationale: the prose is technically coherent, field-specific, and tied closely to a narrowly defined methodological contribution. The paper contains nuanced admissions of limitation such as “the purely mathematical formulation of our model appears to contradict this level of physiological fidelity” and “This operation biases the empirical data toward the statistical range of the model” , which are less typical of generic AI-generated exposition. I do not detect obvious LLM hallmarks such as repetitive abstract generalities, inflated novelty language detached from method, or semantically empty transitions.
Potential AI-flagged passages: low concern. The most formulaic passages are the broad framing paragraphs on Levels 1–4 of articulatory model transparency in the Introduction , because they summarize a landscape in a polished survey style. However, these sections remain contentful and field-appropriate. I would not treat them as evidence of problematic AI authorship.
Assessed epistemic impact: minimal on the manuscript’s scientific contribution. Even if limited AI assistance was used for polishing, the main epistemic issues in this paper arise from design scope, validation independence, and interpretation, not from suspected machine-generated prose.
Recommendations
-
Reframe the headline claim from broad “representational validity” to a more precise statement such as successful discriminative alignment under constrained syllabic and phonetic conditions.
-
Add ablation analyses that remove or vary the model-conditioned renormalization step “CSrenorm = CSnorm · σmodel + µmodel” to test whether the effect survives without biasing empirical data toward model statistics .
-
Quantify how much of the reported success is driven by CTW flexibility rather than by the generative model itself. This could include stricter path constraints, alternative alignment baselines, or comparisons to simpler time-normalization procedures.
-
Expand the experimental design if possible: additional speakers, more varied phrases, broader phonetic coverage, and less regular syllabic material would substantially strengthen the paper.
-
Clarify the logic of phonetic substitutions between the limited model inventory and the recorded database inventory, especially the treatment of /d, g/ as congruent with /s, t, k/ in parts of the experiment .
-
Strengthen statistical reporting with exact p-values, confidence intervals, permutation details, and robustness checks.
-
Tighten language in the Abstract and Conclusion so that the inference stays proportional to the evidence. At present, the conclusion is promising, but it reaches slightly beyond what this single-speaker, heavily structured study can securely establish.
XSci (AI review)
Summary
The manuscript asks whether a deterministic, linguistically driven mathematical model of syllable production generates articulatory trajectories that correspond to real articulatory data. Using nine repetitions of seven French phrases recorded with EMA from a single speaker, the authors project 13 of the 14 sensor coordinates onto six Maeda-like parameters by a fixed linear mapping (Eqs. 1-6), generate analogues of each phrase from the syllabic planning model of [15, 16, 17], and align the model trajectories asymmetrically onto the EMA time grid with Canonical Time Warping. Correlations after alignment are compared between phonetically congruent (Condition A) and incongruent (Condition B) pairings across the 63 files. Condition A is higher for all six parameters and all three formants, which the authors read as support for the representational validity of the model.
Contribution and Significance
The contribution is an evaluation protocol, not a new model. The syllabic planning architecture was introduced in the authors' earlier work [15, 16, 17] and is used here essentially unchanged; the EMA-to-Maeda idea has a direct precedent in Toutios and Narayanan [18], which the authors acknowledge and deliberately simplify; CTW [13] is applied off the shelf. What is new is the combination: holding syllabic structure, speaker, phrase length and carrier words ("C'est", "ça") constant while varying only phonetic content, so that the congruent-incongruent difference isolates the phonetic specificity of the model's output. That is a sensible and cheap instrument, and the field has few quantitative ways to test interpretable articulatory models against kinematic data.
The advance is more modest than the abstract suggests. A congruent-versus-incongruent contrast can show that model output is phonetically informative; it cannot show that the correspondence is strong, nor that this architecture explains the data better than a simpler phoneme-driven generator. The open dataset, code and supplement (Sec. 5) are a genuine asset.
Strengths
- The design is well matched to the data: the Kasper et al. [14] material is regular and phonetically reduced, which makes the congruent/incongruent manipulation clean, and the effect appears consistently across all nine dependent measures (Sec. 3.2).
- The worked analysis in Sec. 3.3 is the most valuable part of the paper. It localises the Condition B failure to F1 of /a/ in "déa", traces it to absent LipH and Jaw movement (Fig. S3), shows Model12 and Model15 parameters to be "largely opposite in phase" (Figs. S4-S5), and documents CTW deleting the syllable /di/ by many-to-one mapping. The method diagnoses rather than merely scores.
- The EMA-to-Maeda projection (Eqs. 1-6) needs no per-speaker geometric adaptation, unlike [18], and is simple enough that others can reuse it.
- Data, code and supplementary figures are openly released on Zenodo and GitHub (Sec. 5).
Recommendation
Major Revision for Interspeech. The design and the Sec. 3.3 analysis are worth publishing, but the abstract's central claim - "strong structural correspondence" and "experimental support for the representational validity of the model" - outruns the evidence in three specific ways, and closing those gaps requires new analyses rather than rewriting. All three fixes lie within the manuscript's own approach, so this is a repairable paper. Under Interspeech's single review round the revision amounts to a resubmission.
Major Concerns
-
The contrast tests phonetic sensitivity, not model validity, and the absolute fit is never reported (Sec. 3.2; abstract; Conclusion). The only comparison in the paper is the model against a permuted copy of itself. Any generator whose output depends on the phoneme string - linear interpolation between Maeda targets, an Öhman-style VCV model, a rule-based synthesiser - would produce A > B here, so the result does not discriminate the proposed syllabic planning architecture from much simpler alternatives. The Conclusion nonetheless asserts that the model "goes beyond traditional rule-based synthesis by reproducing temporal modulations observed at the level of articulatory parameters"; no rule-based system is implemented anywhere in the manuscript. The absolute Condition A correlations for the six articulatory parameters are never given: Sec. 3.2 reports only Δ and t, and the concession that "Jaw and LipH exhibited comparatively lower articulatory correlations in Condition A" comes with no value, so the reader cannot judge whether the correspondence is strong, as the abstract claims, or merely non-zero. For the formants, subtracting the reported Δ from the Condition A means puts Condition B at roughly 0.59 (F1), 0.68 (F2) and 0.65 (F3) - the incongruent surrogate already accounts for most of the correlation. Reporting per-parameter means with dispersion for both conditions, and adding one alternative generator driven by the same phoneme strings, would resolve this.
-
The dependent measure is the quantity CTW optimises, and the warp is unconstrained enough to absorb the model's timing (Secs. 3.1 and 3.3). CTW maximises canonical correlation between the two sequences, and correlation after CTW is then the measure of alignment quality. The contrast between conditions remains informative because both are optimised, but the absolute values cannot be read as goodness of fit, and the manuscript does read them that way ("indicating strong spectral reconstruction"). More seriously, the warp is free enough to delete material: the authors state that "CTW resolves structural discrepancies by deleting the syllable /di/ in condition B through many-to-one temporal mapping". The model's timing is a single externally set parameter (T = 16 frames, strict 2T per CV loop) and is discarded by the warp, so the evaluation carries no information about whether the model reproduces the temporal organisation of the data - which is what the Conclusion claims. Sec. 3.1 reports the one measure that would speak to this, warping-path dispersion, only qualitatively ("the pathways remain close to a straight line"). A path-deviation statistic tested between conditions, plus either a constrained warping band or a correlation under a fixed linear time mapping, would settle the point.
-
The EMA-to-Maeda mapping is unvalidated, and the formant analysis is not independent evidence (Sec. 2.1, Eqs. 1-6; Sec. 3.2). The copy-synthesis formant trajectories that serve as the reference in Fig. 2 (right) are not measured from the recordings; they are produced by driving the Maeda model with the linearly mapped EMA parameters [19], and the model-side formants come from the same synthesiser. The formant correlations are therefore a nonlinear re-expression of the articulatory correlations, not a second, acoustic line of evidence, although Sec. 3.2 presents them as "a similar pattern ... at the formant trajectory level". Underneath this, Eqs. (1)-(6) are asserted rather than justified: the unit weights on x and y coordinates and across sensors are unexplained, one of the 14 coordinates is dropped without comment, and the mapping is never checked against anything. The original audio appears in Fig. 1 only as a visual aid; comparing copy-synthesis F1-F3 against formants tracked from the recorded speech would validate the mapping and give the acoustic result independent standing.
Minor Comments
- The abstract reports "a permutation test" and Sec. 1 a "permutation-based statistical framework", but Sec. 3.2 reports paired t-tests with Holm correction; align the terminology or run the permutation test.
- "The largest effects were observed for LipP, Body, and Dorsum" (Sec. 3.2) is contradicted by the authors' own values, since Tip (Δ = 0.1250) and Jaw (Δ = 0.1216) both exceed Dorsum (Δ = 0.1192); the grouping appears to follow t rather than Δ.
- The 63 files are 9 repetitions of 7 phrases from one speaker, so the paired t-tests treat non-independent observations as independent; phrase and repetition should enter as random factors, and the weakest effect (LipH, t = 2.240) may not survive that.
- State how Condition B pairings were formed, since Sec. 3.3 uses a specific mismatch (Model15 for File 12) while "randomly paired" appears only in the Fig. S2 control.
- Report whether correlations were computed over all 1000 frames or only the 800-frame active region, since the model has no silent tail while the EMA files have roughly 200 silent frames.
- The model substitutes /d/ for /s/ and /t/ and /g/ for /k/ (Sec. 2.2), preserving place but not manner or voicing; the consequences for what "phonetic congruence" means should be discussed.
- LipH (Eq. 6) equals −LipP (Eq. 5) plus UL_y − LL_y, so the two lip parameters are not independent - relevant given that LipP shows the largest Δ and LipH the smallest.
- The selection vectors S_c and S_v produce the coarticulation under test (Sec. 2.2) yet are deferred entirely to the supplement; a compact definition belongs in the paper.
- Position the model against task dynamics, the standard computational implementation of Articulatory Phonology [9] and the closest existing Level 3 system, absent from the reference list: E. L. Saltzman and K. G. Munhall, "A dynamical approach to gestural patterning in speech production," Ecological Psychology, vol. 1, no. 4, pp. 333-382, 1989.
- The study is motivated by Kasper et al.'s anticipation of at least 200 ms before segmental onset and by the model's V_o nodes, but no analysis measures an anticipation interval in either data or model.
- Give degrees of freedom and exact corrected p-values alongside the t and Δ values in Sec. 3.2.
- Indicate the cost of the CTW alignment and how the approach would scale to longer, less regular material with a richer phonetic inventory, since the design depends on short isochronous phrases.