2nd collaboration workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'24)

Session

Posters

Feb 6, 2024, 3:30 PM

Blue lecture hall (Universität Salzburg (Paris-Lodron-Universität))

37. Applying Reinforcement Learning to IFMIF-DONES HVAC optimisation

Antonio Manjavacas (University of Granada)

Poster

As a critical radiological facility, the International Fusion Materials Irradiation Facility - DEMO Oriented Neutron Source (IFMIF-DONES) will implement effective measures to ensure the safety of its personnel and the environment. To enable the proper implementation of these measures, the ISO 17873 standard has been adopted throughout the design process of the facility. The proposed dynamic...

57. Beam Trajectory Control with Lattice-Agnostic Reinforcement Learning

Chenran Xu (IBPT)

Poster

In recent work, it has been shown that reinforcement learning (RL) is capable of outperforming existing methods on accelerator tuning tasks. However, RL algorithms are difficult and time-consuming to train, and currently need to be retrained for every single task. This makes fast deployment in operation difficult and hinders collaborative efforts in this research area. At the same time, modern...

72. Cheetah: Bridging the Gap Between Machine Learning and Particle Accelerator Physics with High-Speed, Differentiable Simulations

Jan Kaiser (DESY), Chenran Xu (IBPT)

Poster

The optimisation and control of particle accelerators present significant challenges due to the limited availability of beam time, high computational costs, and the complexity of the underlying physics. Machine learning has emerged as a powerful tool to address these challenges, but its application is hindered by the scarcity of high-quality data and the computational intensity of traditional...

47. Data-Driven Model Predictive Control for Automated Optimization of Injection into the SIS18 Synchrotron

Sabrina Appel (GSI)

Poster

In accelerator labs like GSI/FAIR, automating complex systems is key for maximising physics experiment time. This study explores the application of a data-driven model predictive control (MPC) to refine the multi-turn injection (MTI) process into the SIS18 synchrotron, departing from conventional numerical optimisation methods. MPC is distinguished by its reduced number of optimisation steps...

71. Learning to Do or Learning While Doing: Reinforcement Learning and Bayesian Optimisation for Online Continuous Tuning

Jan Kaiser (DESY), Chenran Xu (IBPT)

Poster

In the pursuit of optimising particle accelerators, the choice of method for autonomous tuning is critical for enhancing performance and operational efficiency. This study delves into comparing deep reinforcement learning-trained optimisers (RLO) and Bayesian optimisation (BO) for this purpose, motivated by the need to address the complex, dynamic nature of accelerators. Through simulation and...

46. Optimising Quantum Gate Fidelity with Deep Reinforcement Learning

Dr Leander Grech (University of Malta)

Poster

Noisy intermediate-scale quantum (NISQ) computers work by applying a set of quantum gates to an initial ground state, to transform it into a final state that represents the solution to complex computational problems, such as molecular energy evaluation or optimising for the shortest routes in the travelling salesman problem. The effective realisation of NISQ computers requires the...

56. Rapid Adaptation in Accelerator Control: Leveraging Meta-Reinforcement Learning for Handling Dynamic System Variations

Simon Hirlaender (PLUS University Salzburg), Sabrina Pochaba

Poster

In typical reinforcement learning applications for accelerators, system dynamics often vary, leading to
decreased performance in trained agents. In certain scenarios, this performance degradation is severe,
necessitating retraining. However, employing meta-reinforcement learning in conjunction with an
appropriate simulation can enable an agent to rapidly adapt to environmental changes. This...

55. Safe and fast Model-based Reinforcement Learning based on Gaussian Processes demonstrated on CERN AWAKE

Simon Hirlaender (PLUS University Salzburg)

Poster

Reinforcement Learning (RL) is emerging as a valuable method for controlling and optimizing particle accelerators, learning through direct experience without a pre-existing model. However, its low sample efficiency limits its application in real-world scenarios. This paper introduces a model-based RL approach using Gaussian processes to address this efficiency challenge. The proposed RL agent...

48. Samlpe Alignment in Neutron Beamlines Using Reinforcement Learning

Jonathan Edelen (RadiaSoft LLC)

Poster

RadiaSoft has been developing machine learning (ML) methods for automating processes within the accelerator landscape for the past five years. One critical area of this work has been the full automation of sample alignment at neutron and x-ray beamlines to ensure both high quality experimental data and efficient use of operator hours. Historically, sample alignment has been a manual or a...

50. The RL4AA'23 kick-off workshop at KIT

Dr Andrea Santamaria Garcia (KIT)

Poster

Reinforcement Learning (RL) is a unique learning paradigm that is particularly well-suited to tackle complex control tasks, can deal with delayed consequences, and learns from experience without an explicit model of the dynamics of the problem. These properties make RL methods extremely promising for applications in particle accelerators, where the dynamically evolving conditions of both the...

70. Zero-shot Optimiser Learning for Particle Accelerators under Partial Observability

Jan Kaiser (DESY)

Poster

In the quest to harness the full potential of particle accelerators for scientific research, the need for precision and efficiency in their operation is paramount. Traditional tuning methods, while effective, fall short in optimising performance swiftly and accurately, leading to underutilisation of valuable beam time. This study applies deep reinforcement learning to autonomously tune...