The prompt picks one mission — hover, track, or navigate; the climb and landing come with it. A vocabulary that small gives every phase a predicate to enter, a monitor to check, and a parameter to repair.
One sentence in, a verified scenario out
Six prompts, compiled and executed. Each clip shows one run four ways, on one clock.
Police–bus pursuit
track
Drone tracks a bus from 40 meters and radios the ego when a police car chases the bus through an intersection.
A trigger the prompt only implies, synthesized by the planner and then checked against when it fired.
Multi-agent tracking
track
A van drives ahead of the ego car and a police follows behind it while two background cars circulate; a drone tracks the van from 35 meters.
Four coupled agents and an aerial follower — the case that presses on the 8.6 m/s flight ceiling.
Ascend, then track
track
A drone ascends to 25 meters and then follows a truck as it drives through the streets.
“Then” is a gate, not a hint: tracking cannot begin before the climb completes, and the monitor checks that order on the trace.
Cooperative occlusion warning
hover
A parked truck hides a pedestrian, who starts crossing; a hovering drone spots the pedestrian and warns the ego, which then brakes.
The truck hides the pedestrian for 77% of the crossing; the drone keeps 98% visibility. The warning lands 200 ms after detection; the ego stops 9.1 m short.
Survey patrol
navigate
In clear weather at midday, a drone flies a rectangular survey route at 30 meters altitude over the streets around the ego car.
Waypoints that fall inside buildings are snapped to clear airspace before routing, so the route flies at the altitude asked for.
Night survey patrol
navigate
In wet roads at night, a drone flies a rectangular survey route at 40 meters altitude over the streets around the ego car.
Weather and time of day are requirements like any other — wet roads, night, 40 m — set from the prompt and scored one by one.
Highlights
Language-driven air–ground simulation, recast as verified compilation.
A sentence becomes a scenario that is executable and verified — compilation with verification, not code generation.
The language model gets two bounded roles — retrieval-grounded parsing, guarded repair proposals. Every simulator-bound quantity is grounded and validated by deterministic code.
A typed Air–Ground Scenario Graph holds agents, aerial missions, events, communication links, success conditions and their cross-domain dependencies in one object.
Completion is not realization. Structured execution reaches 100% completion with no observed runtime failures — and 34–58% of those runs still miss requirements silently.
Runtime monitoring and localized repair add 10–20 points of verified pass over static verification alone, reaching 64–84% across five language models, with prompt fidelity unchanged.
AURORA-Bench: 200 prompts across 12 capability categories, 904 annotated requirements, probes on both sides of the feasibility boundary.
200 / 904
benchmark prompts, annotated requirements
64–84%
verified scenarios under AURORA, five LLMs
34–58%
silent failures without runtime verification
+10–20 pp
verified-pass gain from monitoring and repair
The framework
Five stages, one rule for which of them the language model may touch.
The language model may propose what a scenario contains, but never what it measures.
Understanding
Typed parsing with rejection
Free text is decoded into a typed language, retrieval-augmented (RAG) over a knowledge base built by measuring the simulators, not by reading documentation about them.
Reasoning
Decided before anything runs
Where the scenario can exist, when its events fire, whether its requirements are satisfiable at all — each answered by construction, not sampling.
Orchestration
Run, monitor, repair what failed
Both simulators advance under a shared logical clock, trace monitors score the outcome, and a bounded repair ladder — numeric adjustment, localized optimization, guarded language-model edit — touches only the parameters that could have caused the violation.
The AURORA pipeline. A schema-bound parse over a simulator knowledge base yields the typed AGSG (1–2); deterministic modules ground, plan and verify it at three levels (3); CARLA and AirSim run under a shared logical clock (4); runtime monitoring drives the bounded repair ladder (5).
The Air–Ground Scenario Graph
One object for every downstream stage to quantify over.
Nodes: the environment, ground agents, the drone, communication links, events, success conditions. Edges: placement, tracking, links, triggers, and what each success condition measures.
It splits what a scenario contains from the numbers configuring it: structure, plus a named parameter vector boxed by its schema and labeled by author.
Types make feasibility decidable, named coordinates make failure localizable, provenance makes repair accountable.
Addressability: “which variables could have caused this failure” is a reachability query, not a judgment.
Grounded graphs from executed runs. Color gives the node type, edge style the relation; measures edges link each success node to what it checks. (a) Cooperative occlusion warning. (b) Survey patrol. (c) Tracking with a proximity warning.
Case studies
One prompt through every representation the pipeline produces: language, graph, plan, execution, measurement.
Aggregate metrics hide the difference between a scenario that satisfied its criteria and one that satisfied them for the wrong reason. Every execution therefore archives a full artifact set — specification, grounded graph, per-tick trace, frames, renderings, timelines — and every number traces back to it.
One tracking request, five representations.(a, b) 3-D and top-down views. (c) The grounded graph — ascent gate, synthesized trigger, comm link. (d) Automaton and executed schedule: the trigger fires 3.1 s early, the warning lands 9.4 s after its planned firing. (e) Chase gap, ego speed, tracking error.
The warning reaches the ego vehicle before the drone establishes close tracking. That ordering is the whole argument for separating communication from mission performance: a delivered message does not establish sustained tracking, and a single completion label would collapse the two. Requirement-level monitoring keeps them apart, and the graph then says which components to look at when one of them is unmet.
Grounded scenarios across the three mission modes. Requested routes are preserved as asked; a waypoint falling in blocked airspace is corrected during grounding, not flown.
In clear weather at midday, a drone flies a rectangular survey route at 30 meters altitude over the streets around the ego car.
A drone ascends to 40 meters and then follows a motorcycle as it drives through the streets.
A bus drives ahead of the ego car and an ambulance follows behind it while two background cars circulate; a drone tracks the bus from 35 meters.
A van tailgates the ego car closely, closing from 45 meters; a tracking drone alerts the ego when the van comes within 20 meters.
Prompt to execution. Six onboard UAV frames sampled across each run — qualitative evidence that the requested spatial and tracking relations hold, alongside the trace-based verification of events and messages.
Repair, localized. A tracking violation with the tracked-fraction margin just negative is moved positive in two iterations — a numerical adjustment, then a single guarded edit — without regenerating the scenario.
Failures routed to the right stage. An infeasible patrol route is returned to grounding for waypoint snapping rather than forced through small parameter edits. Quantitative violations belong to the repair loop; structural geometric defects do not.
AURORA-Bench
200 prompts, 904 checkable requirements, probes on both sides of feasibility.
Organized by the capability a prompt demands, not the scene it describes — twelve categories, from spatial relations and sequencing to cooperative perception, communication failure, safety-critical and compound behavior.
Annotation is per requirement: a reference specification, a feasible realization where one exists, success criteria, permissible variation ranges. A mean of 4.5 requirements per prompt is what separates satisfying eleven of twelve from satisfying all twelve.
Bounded on both sides: nine infeasible probes to refuse, nine boundary-feasible ones to accept, and twelve paraphrase groups that must parse identically.
End-to-end evaluation runs a 50-prompt subset — 40 behavioral scenarios across nine categories, five boundary-feasible probes, five infeasible probes — under a fixed simulation seed, with retrieval history frozen before each campaign.
Composition of AURORA-Bench.(a) 200 prompts in twelve capability categories draw on 904 requirements of nine types. (b–e) Agent types, missions, temporal structure, success criteria. (f) Requirements per prompt, mean 4.5. (g) Entropy per annotation channel.
Results
Six configurations, five language models, three questions kept apart: does it run, does it preserve the request, does it realize it?
The ladder adds one component at a time — direct code generation (B1), simulator documentation (B2), simulator-grounded retrieval (B3), the typed AGSG and managed executor (B4), static verification (B5), and runtime monitoring with localized repair (AURORA). B1–B3 emit code but no executable specification, so they can be scored for completion and never for realization: their successful runs are completed-unverifiable.
five models · scroll the table →
Configuration
GPT-4o
GPT-5.5
GPT-5.4-mini
Gemini 3.1 Pro
Gemini 3.8 Flash
Method
C.↑
RF↓
P↑
SF↓
Fid.↑
C.↑
RF↓
P↑
SF↓
Fid.↑
C.↑
RF↓
P↑
SF↓
Fid.↑
C.↑
RF↓
P↑
SF↓
Fid.↑
C.↑
RF↓
P↑
SF↓
Fid.↑
B1: Direct code generation
0.42
0.58
—
—
—
0.82
0.18
—
—
—
0.70
0.30
—
—
—
0.80
0.20
—
—
—
0.72
0.28
—
—
—
B2: + Documentation
0.68
0.32
—
—
—
0.92
0.08
—
—
—
0.86
0.14
—
—
—
0.94
0.06
—
—
—
0.92
0.08
—
—
—
B3: + Grounded retrieval
0.66
0.34
—
—
—
0.92
0.08
—
—
—
0.78
0.22
—
—
—
0.98
0.02
—
—
—
0.98
0.02
—
—
—
B4: + AGSG & executor
1.00
0
0.58
0.42
0.966
1.00
0
0.42
0.58
1.000
1.00
0
0.50
0.50
1.000
1.00
0
0.66
0.34
0.993
1.00
0
0.62
0.38
0.984
B5: + Static verification
0.98
0
0.56
0.42
0.980
1.00
0
0.48
0.52
1.000
1.00
0
0.60
0.40
0.995
0.98
0
0.70
0.28
0.995
1.00
0
0.54
0.46
0.971
AURORA
0.98
0
0.76
0.22
0.980
1.00
0
0.64
0.36
1.000
1.00
0
0.70
0.30
0.995
0.98
0
0.84
0.14
0.995
1.00
0
0.74
0.26
0.971
End-to-end performance, 50-prompt subset. C. completion · RF runtime failure · P verified pass · SF silent failure · Fid. prompt fidelity; best per column in accent. Completion below B4 is pre-execution rejection, not execution breakdown.
Configuration comparison across models.(a) Completion for the code-generation baselines. (b) Verified passes for the AGSG-based configurations.
Execution reliability does not imply realization
Simulator knowledge improves code execution but never eliminates runtime failure: documentation raises completion for all five models, retrieval adds mixed further gains. The decisive change is structural. Introducing the typed AGSG and managed executor takes completion to 100% and removes observed runtime failures across every model — and leaves 34–58% of scenarios silently failing, complete and wrong. Completion alone therefore cannot evaluate a generated scenario.
Runtime feedback supplies the gain that structure does not. Static verification alone shifts verified pass only unevenly, because finding a feasible configuration before execution does not make a trace-dependent requirement hold during it. Adding runtime monitoring and localized repair improves verified pass by 10–20 percentage points over B5 across all five models, reaching 64–84%. Fidelity is unchanged from B5 to AURORA for every model, so the gain does not come from a broad relaxation of what was asked — though individual edits still warrant inspection when they touch thresholds.
Where repair helps (GPT-4o). (a) Verified pass 52% → 76%, silent failure 40% → 22%. (b) Completion and pre-repair verification per category. (c) Verified pass by capability, before and after repair.
Repair effectiveness and what it costs
Repair routes each violation to the stage that can fix it — the two cases above show the split: a quantitative margin goes to the repair loop, a structurally infeasible route back to grounding. Neither path regenerates the whole scenario.
The cost is dominated by the simulator, not the model. Median repair episodes take 194–249 s, almost all of it repeated co-simulation; diagnosis and edit generation contribute little, and static checks and kinematic preview are close to free. The way to make repair cheaper is to avoid re-execution, not to accelerate edit generation.
Computational cost.(a) Pipeline-stage latency. (b) Repair-episode cost and its per-iteration components. Log scale — simulator re-execution dominates.
A verified pass is satisfaction of the encoded monitored conditions — not of the prompt itself.
Parsing can reinterpret an infeasible request, and repair can relax a monitored threshold; provenance weighting discourages both without prohibiting them. Read the verified-pass rate alongside prompt fidelity and the edit history — which is why every repair is logged.
Citation
@article{wu2026aurora,
title={AURORA: A Natural Language-Driven Agentic Framework for Understanding, Reasoning, and Orchestrating Reliable Air-Ground Co-Simulation},
author={Wu, Keshu and Zhang, Hao and Gan, Rui and Gao, Xiangbo and Li, Xiaopeng and Tu, Zhengzhong and Zhou, Yang},
journal={arXiv preprint arXiv:2609.19527},
year={2026}
}
