3rd collaboration workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'25)

Session

Talks

Apr 3, 2025, 2:00 PM

CSSB Building 15 - Lecture Hall (DESY)

Conveners

Talks: RL for particle accelerators I

• Georg Hoffstaetter (Cornell Univ.)

Talks: RL for particle accelerators II

• Auralee Edelen (SLAC)

Talks: General applications for particle accelerators

• Verena Kain (CERN)

Talks: General applications of RL

• Jason St. John (Fermilab)

6. Opportunities for Reinforcement Learning in Accelerator Automation

Jonathan Edelen (RadiaSoft LLC)

4/3/25, 2:00 PM

Talk

For more than half a decade, RadiaSoft has developed machine learning (ML) solutions to problems of immediate, practical interest in particle accelerator operations. These solutions include machine vision through convolutional neural networks for automating neutron scattering experiments and several classes of autoencoder networks for de-noising signals from beam position monitors and...

23. Reinforcement Learning for In-Spill Optimization of the Mu2e Resonant Extraction: Compensating Non-Stationarity

Jason St. John (Fermilab)

4/3/25, 2:20 PM

Poster + Talk

We present design considerations and challenges for the fast machine learning component of a third-order resonant beam extraction regulation system being commissioned to deliver steady beam rates to the mu2e experiment at Fermilab. Dedicated quadrupoles drive the tune toward the 29/3 resonance each spill, extracting beam at kV multiwire septa. The overall Spill Regulation System consists of...

5. Autonomous Trajectory Steering of DC Beams at CERN's SPS Transfer Lines Using Reinforcement Learning

Adrian Menor de Onate (CERN)

4/3/25, 2:40 PM

Poster

The slow extracted beams at the CERN Super Proton Synchrotron (SPS) are transported over several 100 m long transfer lines to three targets in the North Area Experimental Hall. The experiments require to eliminate intensity fluctuations over the roughly 5 s particle spill and hence to debunch the extracted beams. In this environment, secondary emission monitors (SEMs) have to replace the...

1. Exploring Reinforcement Learning for Optimal Bunch Merge in the AGS

Georg Hoffstaetter (Cornell Univ.)

4/3/25, 3:30 PM

Poster + Talk

In BNL’s Booster, the beam bunches can be split into two or three smaller bunches to reduce their space-charge forces. They are then merged back after acceleration in the Alternating Gradient Synchrotron (AGS). This acceleration with decreased space-charge forces can reduce the final emittance, increasing the luminosity in RHIC and improving proton polarization. Parts of this procedure have...

9. Offline Reinforcement Learning-Based Control of LEIR Injection Efficiency via Data-Driven Surrogate Modeling

Mr Borja Rodriguez Mateos (CERN)

4/3/25, 3:50 PM

Talk

Aging of the stripper foil and unexpected machine shutdowns are the primary causes for reduction of the injected intensity from CERN’s Linac3 into the Low Energy Ion Ring (LEIR). As a result, the set of optimal control parameters that maximizes beam intensity in the ring tends to drift, requiring daily adjustments to the machine control settings. This paper explores the design of a...

4. Geoff: Applications and Developments in 2024

Dr Penny Madysa (GSI Helmholtzzentrum für Schwerionenforschung)

4/4/25, 9:00 AM

Talk

The complexity of the GSI/FAIR accelerator facility demands a high level of automation in order to maximize time for physics experiments. Accelerator laboratories world-wide are exploring a variety of techniques to achieve this, from classical optimization to reinforcement learning.

Geoff, the Generic Optimization Framework & Frontend, is an open-source framework that harmonizes access to...

15. Reinforcement Learning for Laser Alignment

Matthew Schwab (FSU Jena)

4/4/25, 9:20 AM

Poster + Talk

Manual alignment of optical systems can be time consuming and the achieved performance of the system varies depending on the operator doing the alignment. A reinforcement learning approach using the PPO algorithm was used to train agents to align simple two-mirror optical setups, as well as a full regenerative laser amplifier. The goal is to produce agents that can reproducibly align the setup...

13. Generalizing LLMs for Accelerators with RL

Antonin Sulc (LBNL)

4/4/25, 9:40 AM

Talk

Recent advances in fine-tuning large language models (LLMs) with reinforcement learning (RL) techniques have demonstrated their ability to generalize, unlike the often-used Supervised Fine-Tuning (SFT).
Many aspects of particle accelerators, such as beam parameters, have well-defined objectives, making them ideal candidates for RL-driven optimization.

In this work, we explore the...

45. 6D Phase Space Beam Modelling Using Point Cloud Approach

Hayg Guler (IJCLab)

4/4/25, 10:00 AM

8. A Benchmark for Deep Reinforcement Learning-Based Control of Liquid-Propellant Rocket Engines

Kai Dresia

4/4/25, 10:50 AM

Poster + Talk

Deep reinforcement learning (DRL) has demonstrated great potential for controlling and regulating complex real-world systems such as nuclear fusion reactors such as tokamaks and particle accelerators. Another promising application is the DRL-based control of liquid-propellant rocket engines (LPREs), which have been a focus of research at the German Aerospace Center (DLR) for the past six...

22. Multi-Agent Reinforcement Learning to achieve reliable wireless communication

Sabrina Pochaba

4/4/25, 11:10 AM

Poster + Talk

Reinforcement learning (RL) is gaining more and more importance in the field of machine learning (ML). One subfield of RL is Multi-Agent RL (MARL). Here, several agents learn to solve a problem simultaneously rather than a single agent. For this reason, this approach is suitable for many real-world problems.
Since learning in a multiple agent scenario is highly complex, further conflicts can...

20. Reinforcement Learning-Driven Optimal Control of Perfect Entangling Two-Qubit Gates

Leander Grech (University of Malta)

4/4/25, 11:30 AM

Poster + Talk

Noisy intermediate-scale quantum (NISQ) computers promise a new paradigm for what is possible in information processing, with the ability to tackle complex and otherwise intractable computational challenges, by harnessing the massive intrinsic parallelism of qubits. Central to realising the potential of quantum computing are perfect entangling (PE) two-qubit gates, which serve as a critical...

31. RL for beam phase space distribution tuning at LCLS

Auralee Edelen (SLAC)

4/4/25, 11:50 AM

Talk

Beams at LCLS require precise shaping in position-momentum phase space to meet the needs of different users. In particular, the shape of the longitudinal phase space needs to be customized, while ensuring the transverse phase space meets the requirements for Free Electron Laser (FEL) lasing. We present results of using RL for longitudinal phase space shaping, and compare these with approaches...