3rd collaboration workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'25)

Session

Poster session

Not scheduled

CSSB Building 15 - Lecture Hall (DESY)

35. Advanced Optimization Strategies for the ESS-Bilbao Proton Injector, LEBT and RFQ Tuning

Ibon Bustinduy (ESS-BILBAO), Juan Luis Muñoz (ESS-Bilbao), Konrad Altenmüller (ESS-Bilbao)

Poster

The ESS-Bilbao injector is a multipurpose machine that will accelerate protons up to 3 MeV. It will be used to produce neutrons by means of a Beryllium target. The first part of the injector has been running smoothly for more than a decade. This is formed by a proton source of the Electron Cyclotron Resonance (ECR) type that posseses unique characteristics. The subsequent Low Energy Transport...

14. Autonomous Optimization of Longitudinal Triple Splitting in the Proton Synchrotron Using Reinforcement Learning

Joel Axel Wulff (CERN)

Poster

Achieving precise bunch spacing in the Large Hadron Collider (LHC) relies on advanced RF manipulations in the Proton Synchrotron (PS). Multiple RF systems covering a large range of revolution harmonics (7 to 21, 42, 84) allow performing bunch splitting manipulations. To minimize bunch-by-bunch variations in intensity, longitudinal emittance, and shape, precise tuning of relative RF amplitude...

34. Cheetah – A High-speed Differentiable Beam Dynamics Simulation for Machine Learning Applications

Jan Kaiser (DESY), Chenran Xu (IBPT)

Poster

Machine learning has emerged as a powerful solution to the modern challenges in accelerator physics. However, the limited availability of beam time and the high computational cost of simulation codes pose significant hurdles in generating the necessary data for training state-of-the-art machine learning models. Furthermore, optimisation methods can be used to tune accelerators and perform...

2. Comparative Study of Machine Unlearning Techniques for Computer Vision and NLP Models

Mahule Roy (National Institute of Technology Karnataka Surathkal)

Poster

Machine unlearning is an emerging field in machine learning that focuses on efficiently removing the influence of specific data from a trained model. This capability is critical in scenarios requiring compliance with data privacy regulations or when erroneous data needs to be removed without retraining from scratch. In this study, I explore the importance of machine unlearning as a way to...

12. Do you really think about consequences? Bridging Classical Control and Reinforcement Learning for Delayed Outcome Optimisation

Olga Mironova (PLUS University Salzburg)

Poster

This study explores advanced strategies for optimal control in systems with delayed consequences, using beam steering in the AWAKE electron line at CERN as a benchmark. We formulate the task as a constrained optimization problem within a continuous, primarily linear Markov Decision Process (MDP), incorporating measured system parameters and realistic termination criteria. A wide range of...

36. Efficient Accelerator Operation with Artificial Intelligence Based Optimization Methods

Evangelos Matzoukas

Poster

Tuning injectors is a challenging task for the operation of accelerator facilities and synchrotron light sources,
particularly during the commissioning phase. Efficient tuning of the transfer line is essential for ensuring
optimal beam transport and injection efficiency. This process is further complicated by challenges such as
beam misalignment in quadrupole magnets, which can degrade beam...

37. Efficient data-driven model predictive control for online accelerator tuning

Chenran Xu (IBPT)

Poster

Reinforcement learning (RL) is a promising approach for the online control of complex, real-world systems, with recent success demonstrated in applications such as particle accelerator control. However, model-free RL algorithms often suffer from sample inefficiency, making training infeasible without access to high-fidelity simulations or extensive measurement data. This limitation poses a...

38. Feedback Optimization on the EuXFEL Electron Dump Beamline

Christian Hespe (DESY)

Poster

At the European XFEL, the main beam dump serves to absorb all electron bunches that are not required for the downstream scientific experiments. Due to the large beam power of the accelerator, controlling the dump temperature is a crucial component in its operation. Currently, this is done in an open-loop feed-forward manner. However, due to unforeseen drifts and changes in the setup of the...

32. High-Speed Differentiable Simulations as an Enabler for Reinforcement Learning-based FEL Tuning Agents

Jan Kaiser (DESY)

Poster

The photon pulse intensity is one of the key performance metrics of Free Electron Laser (FEL) facilities and has a direct impact on their experimental yield. To date, FEL intensity tuning is a time-consuming manual task that requires expert human operators to have significant skill and experience. Autonomous tuning methods have been demonstrated to reduce setup times and improve the attained...

10. Leveraging Reinforcement Learning for Safe Navigation of Mobile Robots in Accelerator Tunnels

André Dehne (HAW Hamburg)

Poster

The integration of mobile autonomous robots in accelerators introduces potential risks to the facility itself, including collisions with critical components, cables, and infrastructure. Such incidents could compromise the functionality and safety of the accelerator, necessitating robust solutions to mitigate these risks. This paper explores how Reinforcement Learning (RL) can be leveraged to...

7. Model Predictive Control with Gaussian Processes for Safe and Efficient RL: A Case Study at the CERN SPS

Ferdinand Ferber (CERN)

Poster

Classical, model-free Reinforcement Learning (RL) has achieved impressive results in areas where interactions with the environment are inexpensive, such as computer games or simulations. However, in many real-world applications, such as robotics or autonomous particle accelerators, interactions with the system are costly, which creates a need for sample-efficient RL algorithms. In addition,...

3. Parallel Simulations for Faster Reinforcement Learning of Accelerator Control

Finn O'Shea (SLAC National Accelerator Laboratory)

Poster

Reinforcement Learning methods typically require a large number of interactions with the environment to learn anything useful. This makes learning with sophisticated accelerator simulations difficult because of the total time required to train. On the other hand, learning with environments based on these accelerator codes is potentially very useful because they contain a lot of knowledge...

16. Robust Multi-Turn Injection at SIS18 via Gaussian Process Model Predictive Control

Benjamin Halilovic

Poster

In advanced accelerator facilities like the heavy-ion synchrotron SIS18 at GSI in Darmstadt, ensuring stable and efficient multi-turn injection is crucial for achieving high-intensity beams. However, conventional control methods often lack the adaptability needed to handle rapidly changing beam dynamics, leading to suboptimal performance. To address this limitation, a data-driven Gaussian...

27. Studies on Virtual Platform for the HALF Beamline

Xueting Wu (USTC)

Poster

The autonomous alignment and optimization of synchrotron beamlines pose significant challenges. Traditionally, manual alignment is time-consuming and experience-dependent process, often requiring extensive diagnostic efforts and data collection. With the construction of the Hefei Advanced Light Facility (HALF) underway, the development of a virtual platform for beamlines will be an invaluable...

18. Toward a Structured Reinforcement Learning Pipeline for Real-World Systems

Georg Schäfer

Poster

Recent advances in reinforcement learning (RL) have shown great potential for managing complex systems in robotics, manufacturing, and beyond. However, translating RL successes from controlled experiments to real-world scenarios remains a significant challenge due to the absence of a standardized engineering pipeline that prioritizes thorough problem formulation. While data science and control...

21. Towards Few-Shot Reinforcement Learning in Particle Accelerator Control

Simon Hirlaender (PLUS University Salzburg)

Poster

This paper investigates the automation of particle accelerator control using few-shot reinforcement learning (RL), a promising approach to rapidly adapt control strategies with minimal training data. With the advent of advanced diagnostic tools and increasingly complex accelerator schedules, ensuring reliable performance has become critical. We focus on the physics simulation of the AWAKE...

33. Towards Lattice-Agnostic Reinforcement Learning Agents for Transverse Beam Tuning

Jan Kaiser (DESY), Chenran Xu (IBPT)

Poster

Reinforcement learning (RL) has been successfully applied to various online tuning tasks, often outperforming traditional optimization methods. However, model-free RL algorithms typically require a high number of samples, with training processes often involving millions of interactions. As this time-consuming process needs to be repeated to train RL-based controllers for each new task, it...

17. Towards Robust and Adaptive Decision-Making in Autonomous Robots: A Framework for Synchronized Digital Twins

Hannes Voß (University of Applied Sciences Hamburg)

Poster

The use of autonomous mobile robots in dynamic and uncertain environments requires adaptive and robust decision-making. Synchronized digital twins — real-time virtual counterparts of physical systems — offer a promising approach to improving planning, increasing robustness, and enhancing adaptability. However, developing such systems presents significant challenges, including balancing...