Seminar - Enabling Autonomous and Safe Earth Observation through Transformer-Based Reinforcement Learning Architectures - Nov. 6

Anaïs Cheval 
PhD Student, Smead Aerospace
Friday, Nov. 6 | 10:40 A.M. | AERO 111

Abstract: Earth observation satellites are essential for the timely detection and response to dynamic events such as floods, wildfires and storms. Task planning is a key component of these systems: it decides which task a spacecraft executes, when, and with which resources to maximize the scientific and commercial return of a mission. Existing task-planning methods rely either on exact optimization on the ground, which cannot respond to events arising in flight, or on heuristics executed onboard, which afford responsiveness at the expense of optimality. Reinforcement learning (RL) promises the reactivity of the heuristics with the near-optimality of exact optimization.

Prior RL work for spacecraft tasking relies exclusively on multilayer perceptrons (MLPs), standard feedforward neural networks, which process all candidate tasks as a single input vector. This seminar shows the performance limitations of MLPs and the potential of transformers for task planning. Transformers, a key component of today’s large language models (LLMs), encode each task as a distinct token and capture the dependencies among them through attention layers. Transformer-based RL task planners are presented for medium-resolution sweeping satellites, which detect events, and for high-resolution super-agile satellites, which take detailed imagery of these events. Through these applications, the transformer-based RL architectures are shown to adapt to a wide variety of scenarios, including diverse resource constraints, imaging modes from area coverage to discrete targets, and communication rules. However, neural network policies lack formal safety guarantees. To remove this barrier to deployment, this seminar will also present a reachability-based probabilistic shield that runs at execution time on top of the RL policy and filters its unsafe actions with minimal interference. This research is developed in the open-source Python package BSK-RL, which is built on the high-fidelity spacecraft simulator Basilisk.

Bio: Anaïs Cheval is a Ph.D. student in Aerospace Engineering Sciences at the University of Colorado Boulder, advised by Prof. Hanspeter Schaub in the Autonomous Vehicle Systems Laboratory. Her research focuses on spacecraft autonomy, task planning, guidance and control, and machine learning. Prior to the Ph.D., she earned a double master’s degree in aerospace engineering and in applied mathematics and control theory, completed across three universities: Centrale Graduate School of Engineering, KTH Royal Institute of Technology, and Delft University of Technology.