aRieL: Reinforcement Learning Plans Ariel Telescope Surveys

A preprint from University College London researchers applies reinforcement learning to a problem that sounds mundane but is hard in practice: deciding which exoplanets the European Space Agency's Ariel telescope should observe, and in what order, across a multi-year survey. The paper, posted to arXiv on 29 September 2026, introduces aRieL, a simulated observing environment paired with a Proximal Policy Optimisation (PPO) agent. Its authors report that the learned policy finds a strong compromise between competing survey goals and beats several simple scheduling baselines, though they are explicit that this is a feasibility demonstration, not a replacement for mission planning tools.

What the researchers built

According to the arXiv paper, Ariel is planned to launch in 2031 with a nominal four-year operational life, and it will characterise the atmospheres of hundreds of transiting planets using simultaneous visible and infrared spectroscopy. Targets are assigned to science tiers that require different numbers of transit or eclipse observations, and the pool of candidates is larger than what can be observed. The authors note that completing every target in the current Mission Candidate Sample to its maximum tier would need 144.1% of the 3.5-year scheduling horizon they adopt, before counting slewing or idle time. The problem is oversubscribed by design, so a scheduler must choose which parts of the planet population to prioritise.

Earlier Ariel scheduling work, the paper says, relied on combinatorial and metaheuristic optimisation that builds and iteratively repairs a complete schedule. The aRieL approach instead learns a sequential policy: at each step the agent looks at the currently available observation opportunities and the state of the mission, then picks the next target. Each candidate is described by static features such as planetary and stellar properties and tier, plus dynamic features such as progress toward its tier and the timing of the next reachable observation.

Because the set of candidates changes constantly and has no natural ordering, the authors use a Set Transformer architecture. The actor produces one score per available observation, and the critic pools the candidate set into an order-independent summary to estimate the value of the current state. The final configuration handles up to 72 candidates at a time with three attention layers.

What the results show

The agent was compared with four baselines: random selection, a policy that picks whichever target becomes available next, a greedy policy that maximises immediate reward, and a hill-climbing policy whose scoring weights were tuned over 100 iterations. On the baseline candidate sample, the paper reports that the RL policy completed 696 Tier 1 targets (85.5%), finished all 129 Tier 3 targets, and reached 100% coverage of the population bins the authors define. Its observing efficiency was 72.6%, close to the 74.2% of the next-available policy, which is built to maximise efficiency.

The authors also tested a larger catalogue by adding 613 extrapolated TESS planet candidates, a rough proxy for discoveries made before launch. There, the RL policy completed 930 Tier 1 targets (65.2%), compared with 730 (51.2%) for the next best policy, with 98.7% Tier 3 completion, 74.4% efficiency and 95.4% population coverage. The paper is candid that RL did not win everywhere: the next-available policy completed more targets overall, and hill-climbing posted a higher Tier 2 completion rate in the expanded scenario.

One side finding is notable for anyone working with reward design. The greedy policy, which grabs the highest immediate reward, was frequently outperformed by random selection. The authors attribute this to its tendency to chase locally valuable targets that require long idle periods, which cuts observing throughput. That is a small, concrete illustration of why long-horizon planning matters in this setting.

Steering the policy and testing generalisation

Two further experiments speak to flexibility. First, the authors trained additional agents on one-year episodes with altered rewards for the different tiers and for non-observing time, and report that the learned strategies shifted accordingly. Improving one part of the survey generally pulled observing time away from others, so the approach can be used to expose trade-offs, not only to generate a schedule. Second, a policy trained on the baseline scenario was run without retraining on a variant in which Tier 3 targets must be revisited every year. It kept near-complete Tier 3 coverage, but at the cost of the wider Tier 1 and Tier 2 survey.

This matters because mission priorities and the candidate list are expected to change before and during operations. A learned policy that responds to the current state, rather than a schedule computed once, could in principle be re-evaluated quickly: the paper reports that generating a full 3.5-year episode took 20 seconds on the authors' hardware.

Caveats the authors state

The limitations section is substantial, and readers should weigh the headline numbers against it. The authors say their results should not be read as a direct performance comparison with earlier Ariel schedulers, because those studies use different target samples, constraints, and objectives. Notably, earlier samples required roughly 70% of the projected mission time, whereas the sample used here requires 144%, and the earlier tools account for operational details such as calibration, station keeping, and the field of view at the L2 orbit that this environment simplifies.

Only one training seed was reported, so the results describe a single learned policy rather than the variance of PPO training. The paper also identifies a modelling bias: because observation durations differ, a fixed discount factor discounts by number of decisions rather than elapsed time, which can favour trajectories with fewer, longer actions. A duration-aware discount is left to future work, since the standard Stable-Baselines3 implementation the authors used assumes a fixed factor. Stellar variability and observational uncertainty are also not modelled, and the paper states that the results do not establish a globally optimal Ariel schedule.

Why it matters beyond astronomy

The training cost is a quiet point of interest. The authors say the models were trained on a MacBook Pro with an Apple M3 chip, with the final policy taking about 15 hours for 2.5 million environment interactions. For a field where RL headlines usually involve large clusters, a long-horizon, multi-objective scheduling problem tackled on a consumer laptop is a useful data point, with the caveat that more realistic environments would raise the cost. The authors themselves note that adding operational detail would increase state-space complexity and require more training.

The work was also covered by Astrobiology.com on 14 October, which restates the abstract. The paper says the aRieL environment is released publicly alongside it, which lets other groups test their own methods on the same task. That matters because the authors acknowledge that a fair comparison with metaheuristic schedulers would require identical catalogues, objectives and constraints.

What to watch next

The preprint has not, as far as the sources opened here show, completed peer review; its header marks it as a preprint. The next steps the authors point to are broader testing across mission futures, incorporating observational uncertainty, and comparing against more heavily optimised schedulers. If those comparisons hold up, the more interesting contribution may be the framing rather than the numbers: treating a telescope's target selection as a learnable sequential decision whose priorities can be changed by editing the reward, then reading the resulting trade-offs directly from the schedule.