XSci (AI review)

October 2, 2026

Referee Report

Manuscript: AQ-OLSR: Adaptive Routing Protocol for UAV Ad Hoc Network Merged with Deep Q-Network (DQN) Algorithm

Field: Computer networking and wireless communications engineering — UAV ad hoc networks (FANETs), simulation-based protocol evaluation with deep reinforcement learning

Recommendation: Major revision

Summary and Recommendation

The manuscript proposes AQ-OLSR, an extension of the Optimized Link State Routing protocol in which the HELLO message interval is selected online by a Deep Q-Network agent rather than held fixed. The decision problem is cast as a Markov Decision Process whose state comprises the expected active connection time of a link, a neighbour's residual energy, and the throughput, end-to-end delay, and packet delivery ratio observed over the preceding interval; the action is the next interval, and the reward is a linear combination of these quantities. The protocol is implemented in ns3-gym and compared against OLSR, ML-OLSR, and EE-Hello-OLSR over a 3 km × 3 km area at 150 m altitude, using Gauss-Markov mobility, IEEE 802.11p, and 512-byte constant-bit-rate traffic over UDP. Two sweeps are reported, one varying speed from 10 to 50 m/s with 25 nodes and one varying node count from 20 to 60 at 20 m/s, with Table 2 summarising improvements of 2.4% to 21.4% over the two adaptive comparators.

Several aspects deserve credit. The exposition of the HELLO-interval trade-off in Section 3.1.1 is clear and correct, the related work in Section 2 treats each comparator's mechanism and limitations substantively, and benchmarking against an adaptive-interval protocol rather than only against stock OLSR reflects sound judgement about what constitutes a meaningful test; the density sweep also addresses a limitation the authors correctly identify in DQN-OLSR. The reviewer nevertheless recommends major revision for three reasons. The learning problem is formulated in a way that does not express the stated objective and leaves necessary elements undefined. The mechanism the protocol implements is narrower than the causal explanations offered for the results, and no control condition tests those explanations. And the evidence is reported without uncertainty, without numerical values, and from an operating envelope too narrow to support the generality claimed. None of this appears to require abandoning the design.

Major Concern Category 1: Formulation of the Learning Problem

1.1 The reward function does not encode the stated objective and its components are not commensurable

Equation (5) defines the reward as k₁ × ECT minus k₂ × Thr minus k₃ × Del minus k₄ × Pdr, with all coefficients set to unity. Read literally, an agent maximising this quantity is penalised for high throughput and high packet delivery ratio, which inverts the objective stated in Section 3.1.1 and contradicts the whole of Section 4, where AQ-OLSR is reported to lead on both; only the delay term carries a defensible sign. Independently, the components occupy incompatible units and radically different scales, from a ratio bounded in [0,1] to a byte rate of order 10⁴, so unit weights do not balance the objective but let the largest term dominate. The authors should state the corrected function, normalise the components, report each term's realised magnitude during training, and show through a sensitivity analysis over k₁ to k₄ that the results do not depend on one arbitrary weighting.

1.2 The Markov Decision Process is specified incompletely and mixes quantities of different scope

Section 3.1.2 introduces the model as "the triplet (S, A, P, R)," which names four elements, then defines three; the transition kernel is never specified. More consequentially, the Markov property is assumed rather than argued: because throughput, delay, and delivery ratio are averaged over the preceding HELLO interval, and that interval is itself the previous action, identical network conditions yield different observations depending on the prior action, a dependence on action history the state does not capture. The scope is also inconsistent, since connection time and residual energy belong to a particular link and neighbour while the remaining components are network-wide aggregates, yet the input is fixed at [1 × 5] and the manuscript never says which neighbour populates the first two slots. Episode semantics are undefined, and the non-stationarity arising when many nodes learn simultaneously is unacknowledged.

1.3 The agent architecture and training protocol are too incompletely specified to establish convergence or permit reproduction

The action space is never defined as a set. Deep Q-Networks address discrete actions, emitting one value per admissible action, yet Figure 4 shows a [1 × 1] scalar output while Section 3.1.3 describes selecting the action with the highest Q value, and the provision that a negative T_Hello be corrected to one second implies an unbounded continuous output; these descriptions cannot all hold. The training protocol omits the number of episodes, the replay buffer capacity and minibatch size, the target-network synchronisation period, the exploration schedule, and the optimiser; the learning rate of 0.004 and discount factor of 0.908 are unjustified; and no learning curve or reward trace establishes convergence. No separation is stated between training and evaluation configurations, which leaves the generalisation claim in Section 1 asserted rather than demonstrated.

1.4 The information the agent consumes does not appear to be observable at an individual UAV

Three state components, and therefore the reward, are defined over network-wide quantities. Equation (4) requires the total packets sent by all sources, which no node can observe locally, and under UDP a node cannot determine how many of its own packets arrived. If these quantities were drawn from simulator-global state, as ns3-gym makes straightforward, the evaluated system is not the distributed protocol described, and the signalling needed to gather that information in deployment would offset the claimed savings; if they were estimated locally, Equations (2) to (4) misdescribe the implementation. Either answer is publishable, but the reader must be told which holds, ideally through a subsection stating where each component is measured and how it reaches the decision-maker.

Major Concern Category 2: Protocol Mechanism and the Attribution of Results

2.1 The causal explanations in Section 4 describe mechanisms the protocol does not implement

The action space is a single scalar. The agent does not select next hops, weight links, alter MPR selection, or modify the shortest-path computation. Yet the discussion of Figure 12 states that the adaptive route "can effectively cut needless hops, skip low-energy UAVs, and ultimately decrease the total energy used," and both results subsections attribute the delivery advantage to the DQN "adaptively updat[ing] the routing decision." No mechanism in Section 3 permits hop reduction or energy-aware forwarding; residual energy enters only as an input feature, and nothing explains how observing it becomes a routing preference. If further modifications exist they are missing from the methodology and should be added with pseudocode; if not, the discussion should be confined to effects a HELLO-interval policy can produce. Stating in one place the complete set of OLSR behaviours AQ-OLSR changes would resolve this.

2.2 The interaction between a variable HELLO interval and OLSR's timing machinery is not analysed

OLSR couples the HELLO interval to the validity time carried in the message, which receivers use to expire link and neighbour entries and on which link-state classification and MPR maintenance depend. Whether AQ-OLSR updates the advertised validity time in step with the learned interval, and how it ensures neighbours receive that update before it takes effect, is unstated; without this, lengthening an interval causes premature link expiry and the route flapping the protocol aims to prevent. Because each node runs its own policy, neighbours will generally select different intervals, and the manuscript does not discuss how heterogeneous intervals interact or whether asymmetric link perceptions result. Relatedly, Section 4.1 attributes OLSR's overhead to TC exchange although AQ-OLSR modifies only HELLO signalling; reporting MPR set size, MPR churn, and the HELLO-versus-TC split of overhead would sharpen this considerably.

2.3 No control condition isolates the contribution of the learned policy

Section 4 attributes every result to the DQN, and the design contains nothing capable of testing that attribution. There is no fixed-interval control set at the policy's mean selected value, which would show whether learning contributes beyond offline tuning; no tabular Q-learning control, which would test whether depth does any work in a five-dimensional state space; and no ablation over the state vector, so the contribution of connection time and residual energy, the two features constituting the stated novelty, remains unmeasured. The learned policy is also never characterised: plotting the selected interval against UAV speed, alongside the interval EE-Hello-OLSR's analytic rule would produce, would show whether the agent found a non-trivial relationship or rediscovered the inverse-speed heuristic. Claims such as freed control airtime being filled by data are measurable as channel-occupancy breakdowns but are asserted instead.

2.4 The novelty and performance claims are not calibrated to what the comparison establishes

Section 2 defines the gap relative to DQN-OLSR as adapting a different control message, adding two state features, and introducing a density sweep. Each step is reasonable, but the manuscript describes the outcome as "an innovative" and "a new routing protocol," and DQN-OLSR is absent from the evaluation, so the size of the step cannot be assessed. Including it as a baseline is the direct remedy; failing that, the contribution should be stated precisely as an extension of that approach, which remains publishable. Likewise, the abstract's "significant advantage" spanning "2.4% to 21.4% ... for all scenarios and metrics" uses a word no reported analysis supports and a range that excludes both energy and OLSR. Some treatment of failure modes, including whether a safe fallback interval is enforced if the policy diverges, would strengthen the engineering case.

Major Concern Category 3: Experimental Design, Evidence, and Reporting

3.1 The evaluation reports no measure of statistical uncertainty

Table 1 states that five simulation runs were performed, and no other element of statistical practice appears: no confidence intervals or error bars, no hypothesis test, and no reported seeds or replicate-generation procedure, despite random placement and a stochastic mobility process. This matters because of the size of the claimed effects. Improvements of 3.1% in delivery ratio and 3.7% in throughput over EE-Hello-OLSR sit comfortably within the inter-replicate variation typical of packet-level wireless simulation, and with five runs the standard error of a mean is roughly 45% of the sample standard deviation. It is also unstated whether the runs involved five independently trained policies or five evaluations of one, which are very different claims about variance. Twenty to thirty replicates with intervals and paired significance tests would place the comparison on defensible ground; where differences do not survive, saying so would strengthen the paper.

3.2 No numerical results appear in the text, and the summary table is incomplete

Sections 4.1 and 4.2 describe ten figures across roughly 1,800 words without stating a single value or unit. Readers learn that overhead is lowest and energy consumption "significantly less" but never what these quantities are or in what units, and definitions drift, with throughput specified as a byte rate in Equation (2) but described in Section 4.1 as a cumulative packet count. Table 2, the sole numerical artefact, omits OLSR and omits energy, lacks a sub-header for the mobility block paralleling the "Density (Scalability)" row, and does not state how its percentages were aggregated across the nine speed or five density settings, so the abstract's headline range cannot be audited. A complete table giving means and intervals in stated units, for every protocol, metric, and operating point, would resolve this.

3.3 The physical-layer and energy models are specified too sparsely for reproduction

Table 1 lists a MAC standard, antenna type, frequency, range, and channel capacity, none of which is an independent parameter in a packet-level simulation. Communication range emerges from transmit power, antenna gains, propagation loss, and receiver sensitivity, none of which is given, so a reader cannot tell whether links break at a hard threshold or degrade through fading, a distinction central here because expected connection time is the first state and reward component and is far more predictable under the former. The stated 2.4 GHz frequency is also inconsistent with IEEE 802.11p, which occupies 5.9 GHz, and the 2 Mbps capacity matches no standard 802.11p modulation and coding scheme. The energy model is absent entirely, with no initial energy, no transmit, receive, or idle power, and no energy source model, which makes the energy curves uninterpretable.

3.4 The chosen operating point complicates interpretation of the throughput and delivery results

Ten sources sending 512-byte packets at four packets per second offer roughly 164 kbit/s against a 2 Mbps channel, leaving the network far below saturation. In that regime delivery ratio is governed by route availability rather than congestion, and throughput is essentially determined by delivery ratio since offered load is fixed, so the two curves are views of one quantity rather than independent confirmations; the light load also means freed control airtime has little to carry, which likely explains why large overhead reductions yield small end-to-end gains. A load sweep toward saturation would show where the mechanism becomes material. On the connectivity side, 25 nodes over 9 km² with a 300 m range implies a sparse and possibly partitioned network, yet no neighbour degree, connectivity fraction, or hop-count statistics are reported. The density sweep additionally scales active flows with node count, confounding density with offered load.

3.5 Mobility is modelled by a single under-specified process, and generality claims exceed the envelope examined

Table 1 reports only "GM Model" and a 0.5 s update interval, omitting the Gauss-Markov memory parameter, the speed and direction statistics, and the boundary rule, although the correlation structure they define determines how predictable link lifetimes are and hence how favourable the setting is to the proposed state representation. Altitude is a single value, so the topology is effectively planar despite the three-dimensional framing. More broadly, traffic type, packet size, rate, transport, channel, and MAC are each fixed at one value while the setting is described as "a generic UAV ad hoc network environment" and the protocol recommended for real-time missions, a recommendation requiring delay tails and jitter, neither of which is reported. Training under one regime and evaluating under another would test the generalisation claim directly and would be among the most valuable additions available.

3.6 The energy result is never connected to an operationally meaningful quantity

Energy is the paper's organising motivation: the abstract closes on it, Section 1 states it as an objective, residual energy is a state component, and two figures are devoted to it. Yet it is reported only qualitatively, carries no unit, and is the one metric excluded from Table 2, so the magnitude of the saving is nowhere recorded. More fundamentally, it is never placed against a platform's total energy budget. Propulsion dominates onboard power consumption for both rotary- and fixed-wing UAVs, and whether reduced control-plane transmission meaningfully extends endurance depends on the ratio of radio to propulsion power, which cannot be evaluated because no platform, battery capacity, or flight time is specified. Reporting energy in joules, including it in Table 2, and expressing the saving as a fraction of mission energy or as additional flight time would let readers judge the claim.

Minor Issues

1. Undefined and inconsistently characterised comparator. ML-OLSR serves as a baseline throughout Section 4 and appears in both tables but is never defined, described, or cited, and has no reference-list entry. It is also characterised as "a load-aware algorithm" in Section 4.1 and "a mobility-aware algorithm" in Section 4.2. A citation, a mechanism description, and the parameters used are needed, along with resolution of the discrepancy.

2. Mischaracterisation of the comparison set. Section 3.1.4 states that AQ-OLSR was compared against "a reactive protocol, a proactive protocol, and an adaptive hello message protocol." None of the three comparators is reactive; all are proactive link-state variants. The sentence should be corrected and the absence of a reactive baseline such as AODV either rectified or justified.

3. Errors in the defining equations. Equation (3), delay as T_Hello divided by received packets, expresses the reciprocal of a reception rate rather than a latency, and enters both state and reward. Equation (1) adds a distance to the reciprocal of a speed, which is dimensionally inconsistent and does not match the surrounding description; since it reproduces the baseline from [23], an incorrect rendering would bias the comparison.

4. Errors in the learning equations. Equation (6) applies the tabular Q-learning assignment rule to a neural approximator, where the learning rate should parameterise gradient descent on the loss in Equation (8), and Equation (7) defines the target network's value through its own maximum rather than by periodic parameter copying, although the prose describes the intended behaviour correctly. The three should be made mutually consistent.

5. Notational and definitional gaps. The state vector indexes residual energy to the neighbour while item (b) indexes it to the local node before describing the neighbour's energy. Expected connection time is defined as the time until distance "extends to communication range," where exceeding the range is evidently meant, and no estimator is specified. No cold-start rule is given for the first decision, when the performance components are undefined.

6. Inconsistencies between abstract, contributions, and body. The abstract lists only two comparators while three are evaluated; contribution 2 promises throughput and delay while five metrics are reported; Section 4 refers to "five different routing protocols" where four are used. Equation (5) also carries t+1 subscripts while the preceding sentence refers to the current interval, which determines whether the reward is observable at decision time.

7. Unreported simulation configuration. Absent are the node placement distribution and seeds, the source-destination pairing and flow start times, the baseline OLSR timer and willingness settings, and the ns-3 version. No warm-up period is stated, which matters because the agent explores before it exploits, and a fixed 600-second duration corresponds to very different numbers of domain crossings across the speed sweep, so stationarity at the low-speed end should be confirmed.

8. Measurements that would support stated mechanisms but are not reported. Hop-count distributions would test the claim of reduced hops, channel-occupancy breakdowns the claim that freed airtime carries data, and delay percentiles and jitter the real-time recommendation. All are readily available from the simulator, as are the connectivity statistics noted above.

9. Choices stated without justification. The 8–16–8–4 hidden-layer widths are unexplained for a five-dimensional input. The exploration policy is described both as stochastic and as "a greedy decent approach" without being named or scheduled. The 150 m altitude is attributed to a source concerning urban low-altitude navigation. Section 3.1.4 also justifies the chosen ranges by matching [23] while the next paragraph lists several changed parameters, so what comparability survives should be clarified.

10. Manuscript preparation. Residual thesis cross-references appear throughout, with "Figure 4.13," "Figure 4.14," and "Figure 4.19" used for Figures 8, 9, and 14, equation references numbered 3.7 to 3.13 alongside (2) to (8), and one caption reading "Figure Error! No text of specified style in document." Section 4.2 states that "the conventional OLSR procedure exhibits the best level of overhead," contradicting the surrounding text, and items (c) to (e) of Section 3.1.2 end mid-sentence. Several passages in Section 1 appear affected by automated paraphrasing, including "device gaining knowledge of machine learning," "Classical on-diamond ... routing algorithms," and "improved verbal exchange." The author's email contains a comma and is invalid, references [16] and [31] duplicate the same work, reference [33] contains unrendered markup, and figure axis units are given nowhere.

11. Framing of the closing material. The conclusion reproduces several sentences from Sections 4.1 and 4.2 nearly verbatim rather than synthesising, and the stated future work describes the present contribution; the forward-looking element, improving MPR selection, deserves to stand alone. A data and code availability statement releasing the ns3-gym environment, the protocol module, and the training scripts would substantially enhance credibility.

Missing and Inadequately Integrated Literature

Cited but Inadequately Integrated

Mnih et al. (2015) is cited to justify the two-network architecture, but the stabilisation machinery that work establishes as necessary is not engaged with: replay is mentioned once and target synchronisation only as occurring "after a certain amount of time." Garcia and Rachelson (2013) is cited for the MDP formalism without being used to discharge its obligations, namely specifying the transition kernel, distinguishing episodic from continuing tasks, and establishing that the chosen representation preserves the Markov property, all of which bear on concern 1.2.

Zeng et al. (2023) is the most consequential case. DQN-OLSR is the closest prior work and the reference point from which the gap is derived, yet it is absent from the evaluation, and the manuscript's own criticism of it, that it is compared only against standard protocols and omits a scaling scenario, is precisely what this work could remedy; its use of MPR change rate as a state feature is also directly relevant to the unexplained TC-overhead attribution in 2.2. Mahmud and Cho (2019) supplies the principal comparator and the parameter ranges but its design is not examined, and reporting how the two policies differ in the intervals they select across the speed sweep would show where the learned policy departs from the analytic heuristic. Lansky et al. (2022), cited twice under two reference numbers, is a systematic review of this design space and could position the state, action, and reward design against existing RL routing protocols, making the novelty claim concrete. The authors' own review of topology-based protocols and mobility models (2021) is cited only for the single- versus multi-UAV distinction although its substance concerns the mobility-model sensitivity raised in 3.5, and Jha et al. (2022) is cited for the wide variation in QoS requirements across UAV applications without that observation being allowed to scope the paper's single-profile recommendations.

Truly Missing Papers

RFC 3626 (Clausen and Jacquet, 2003) is not cited although the paper extends and evaluates OLSR. It defines HELLO_INTERVAL, the validity time field, neighbour hold times, link sensing, and MPR selection, and supplies the two-second default the manuscript adopts without attribution; a protocol contribution should be stated as a delta against the normative document. Henderson et al. (2018) documents how substantially deep reinforcement learning results vary across seeds and implementation details and sets out the reporting standards concern 3.1 calls for. Camp, Boleng, and Davies (2002) is the standard reference for the Gauss-Markov model and for the sensitivity of ad hoc routing results to mobility model choice, and would let the authors justify rather than assert their parameterisation. Bettstetter (2002) gives the analytical relationship between node density, transmission range, and connectivity, which would establish rather than assume that the 20-to-60-node configurations form connected networks.

On the energy argument, Zeng, Zhang, and Lim (2016) sets out the energy constraints specific to UAV communication platforms and Zeng and Zhang (2017) develops an explicit propulsion power model, so together they would let the saving in Figures 12 and 17 be expressed as a fraction of total platform energy. Finally, Arafat and Moh (2022) presents a Q-learning topology-aware FANET routing protocol whose state representation overlaps substantially with the present one, and Lakew et al. (2020) offers a current survey that would broaden the introduction's characterisation of the field.

Arafat, M. Y., and S. Moh. 2022. A Q-Learning-Based Topology-Aware Routing Protocol for Flying Ad Hoc Networks. IEEE Internet of Things Journal 9(3):1985-2000.
Bettstetter, C. 2002. On the Minimum Node Degree and Connectivity of a Wireless Multihop Network. Proc. ACM MobiHoc 2002:80-91.
Camp, T., J. Boleng, and V. Davies. 2002. A Survey of Mobility Models for Ad Hoc Network Research. Wireless Communications and Mobile Computing 2(5):483-502.
Clausen, T., and P. Jacquet. 2003. Optimized Link State Routing Protocol (OLSR). IETF RFC 3626.
Henderson, P., R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger. 2018. Deep Reinforcement Learning That Matters. Proc. AAAI Conference on Artificial Intelligence 32(1):3207-3214.
Lakew, D. S., U. Sa'ad, N.-N. Dao, W. Na, and S. Cho. 2020. Routing in Flying Ad Hoc Networks: A Comprehensive Survey. IEEE Communications Surveys and Tutorials 22(2):1071-1120.
Zeng, Y., R. Zhang, and T. J. Lim. 2016. Wireless Communications with Unmanned Aerial Vehicles: Opportunities and Challenges. IEEE Communications Magazine 54(5):36-42.
Zeng, Y., and R. Zhang. 2017. Energy-Efficient UAV Communication with Trajectory Optimization. IEEE Transactions on Wireless Communications 16(6):3747-3760.

Conclusion and Path Forward

The central insight of this manuscript, that the HELLO interval is a control variable whose best setting depends on link dynamics, node energy, and realised performance, and that this dependence is a reasonable target for a learned policy, is sound, and implementing it in a packet-level simulator coupled to a reinforcement learning loop is a credible way to investigate it. The overhead reductions reported are large enough to be interesting in their own right, and the related work reflects a genuine command of the design space. The reviewer's reservations concern how the problem is formalised, how the mechanism is described, and how the evidence is gathered and reported, rather than whether the idea merits investigation.

A revision addressing the following would, in the reviewer's judgement, transform the paper. The reward function should be corrected in sign, normalised, and subjected to sensitivity analysis; the MDP, the action set, and the training protocol should be stated completely, with evidence of convergence and an explicit separation between training and evaluation configurations; and the measurement and signalling architecture for each state component should be specified so that the deployability question is settled. The causal language in the results should be confined to effects the implemented mechanism can produce, supported by MPR and overhead-decomposition measurements and by an analysis of how a variable interval interacts with OLSR validity times across asynchronous neighbours. Empirically, replicate counts sufficient for interval estimation, numerical results in stated units, a complete parameter table, a load sweep, reported connectivity statistics, a density sweep with flow count held fixed, at least one additional mobility model, delay percentiles, and an energy result expressed against a stated platform budget would place the conclusions on firm ground, as would adding DQN-OLSR as a baseline alongside fixed-interval and ablation controls. The reviewer expects the overhead advantage to survive this scrutiny comfortably and some smaller end-to-end effects not to; a paper reporting that outcome carefully would make a more durable contribution than one claiming more on thinner evidence. The question the authors have chosen is a valuable one and deserves this level of care.