1st collaboration workshop on Reinforcement Learning for Autonomous Accelerators (RL4AA'23)
Building 345
Institute of Beam Physics and Technology (KIT)
Reinforcement learning (RL) is a powerful learning paradigm of machine learning (ML). It is particularly suited to tackle control problems in large environments, can learn from experience without the need of a model of the dynamics, and can deal with delayed consequences.
Capturing your control problem as a meaningful Markov Decision Process (MDP) is not trivial. Additional challenges arise in the training in terms of stability and evaluation. Other practical aspects include reproducibility, efficiency, implementation, deployment in hardware, or choosing the most suitable algorithm for your problem.
RL applications in particle accelerators are very promising, but have been deployed in real machines only a handful of times. This workshop aims at lowering the barrier in applying RL and making it a more widely used tool.
We will take time for valuable discussions on the topic of RL applied to particle accelerators and to improve our common knowledge. We will also exchange ideas about promising future avenues and help consolidate a community to advance in this area of research in accelerator controls.
-
-
8:00 AM
→
8:19 AM
KIT-Shuttle to Campus Nord 19m Shuttle stop (https://maps.app.goo.gl/YEYs2qa4TdyRmKAR9?g_st=ic)
Shuttle stop
https://maps.app.goo.gl/YEYs2qa4TdyRmKAR9?g_st=ic
-
8:19 AM
→
9:00 AM
Registration of participants at reception + walking to IBPT 41m CN Anmeldung (https://maps.app.goo.gl/zvk55ETKapvNs38U6?g_st=ic)
CN Anmeldung
https://maps.app.goo.gl/zvk55ETKapvNs38U6?g_st=ic
https://maps.app.goo.gl/zvk55ETKapvNs38U6?g_st=ic
- 9:00 AM → 9:10 AM
-
9:10 AM
→
9:30 AM
Introduction to IBPT activities 20m KARA hall seminar room (https://goo.gl/maps/bWuXfGLFZaRVvhuV8)
KARA hall seminar room
https://goo.gl/maps/bWuXfGLFZaRVvhuV8
Speaker: Johannes Steinmann (KIT / IBPT) -
9:30 AM
→
11:00 AM
Tour of the accelerator facilities: KARA (Karlsruhe Research Accelerator)
-
11:00 AM
→
12:00 PM
Tour of the accelerator facilities: FLUTE (Ferninfrarot Linac- und Test-Experiment)
-
12:00 PM
→
1:30 PM
Lunch break 1h 30m Building 145 (Canteen "Casino")
Building 145
Canteen "Casino"
-
1:30 PM
→
3:25 PM
Meet & greet speed talks KARA hall seminar room (https://goo.gl/maps/bWuXfGLFZaRVvhuV8)
KARA hall seminar room
https://goo.gl/maps/bWuXfGLFZaRVvhuV8
-
1:30 PM
Sabrina Appel 5m
-
1:35 PM
Awal Awal 5m
-
1:40 PM
Niky Bruchon 5m
-
1:45 PM
Conrad Caliari 5m
-
1:50 PM
Felipe Donoso 5m
- 1:55 PM
-
2:00 PM
Jan Henry Hetzel 5m
-
2:05 PM
Simon Hirländer 5m
- 2:10 PM
-
2:15 PM
Reuf Kozlica 5m
-
2:20 PM
Stephan Kötter 5m
-
2:25 PM
Sabrina Pochaba 5m
-
2:30 PM
Andrea Santamaria Garcia 5m
-
2:35 PM
Michael Schenk 5m
-
2:40 PM
Pierre Schnizer 5m
-
2:45 PM
Alexander Schütt 5m
- 2:50 PM
-
2:55 PM
Waheedullah Sulaiman Khail 5m
- 3:00 PM
-
3:05 PM
Discussions, Q&A 20m
-
1:30 PM
-
3:25 PM
→
3:50 PM
Coffee break 25m
-
3:50 PM
→
4:50 PM
Open discussion: preparation of advanced session KARA hall seminar room (https://goo.gl/maps/bWuXfGLFZaRVvhuV8)
KARA hall seminar room
https://goo.gl/maps/bWuXfGLFZaRVvhuV8
-
5:05 PM
→
5:26 PM
KIT-Shuttle back 21m
-
7:00 PM
→
10:00 PM
Workshop dinner L'Osteria Karlsruhe
L'Osteria Karlsruhe
Zähringerstraße 69, 76133 Karlsruhe
-
8:00 AM
→
8:19 AM
-
-
8:30 AM
→
8:52 AM
KIT-Shuttle to CN 22m Shuttle stop (https://maps.app.goo.gl/YEYs2qa4TdyRmKAR9?g_st=ic)
Shuttle stop
https://maps.app.goo.gl/YEYs2qa4TdyRmKAR9?g_st=ic
-
9:00 AM
→
12:45 PM
Introduction to RL workshop KARA hall seminar room (https://goo.gl/maps/bWuXfGLFZaRVvhuV8)
KARA hall seminar room
https://goo.gl/maps/bWuXfGLFZaRVvhuV8
Introductory talks and hands-on tutorials
- 9:00 AM
-
9:40 AM
Advanced concepts in RL 1hSpeaker: Dr Simon Hirländer (University of Salzburg)
-
10:40 AM
Coffee break 20m
-
11:00 AM
Python tutorial: application to an accelerator problem 1h 45m
Please find the instructions for the tutorial here: https://github.com/RL4AA/RL4AA23
Speakers: Dr Andrea Santamaria Garcia (KIT), Chenran Xu (IBPT), Jan Kaiser (DESY), Dr Simon Hirländer (University of Salzburg)
-
12:45 PM
→
2:00 PM
Lunch break 1h 15m Building 145 (Canteen "Casino")
Building 145
Canteen "Casino"
-
2:00 PM
→
3:00 PM
IBPT seminar talk by Dr. Simon Hirländer "Reinforcement Learning in particle accelerator control: are we there yet?" 1h KARA hall seminar room (https://goo.gl/maps/bWuXfGLFZaRVvhuV8)
KARA hall seminar room
https://goo.gl/maps/bWuXfGLFZaRVvhuV8
Speaker: Dr Simon Hirländer (University of Salzburg) -
3:00 PM
→
3:30 PM
Coffee break 30m
-
3:30 PM
→
6:00 PM
Discussion on advanced RL topics
-
3:30 PM
Discussion Group 1: Modeling, limitations, and methods 35mSpeaker: Simon Hirländer
-
4:05 PM
Discussion Group 2: Challenges I 35mSpeaker: Chenran Xu (IBPT)
-
4:40 PM
Discussion Group 3: Challenges II 35mSpeaker: Michael Schenk
-
5:15 PM
Discussion Group 4: Community 35mSpeaker: Jan Kaiser
-
5:50 PM
Farewell 10mSpeaker: Andrea Santamaria Garcia (KIT)
-
3:30 PM
-
8:30 AM
→
8:52 AM