Registration Room: Aula

Welcome

• Achim Streit (KIT-SCC)

Room: Aula Welcome

Introduction to HUF 2024

• Doris Ressmann (Karlsruhe Institute of Technlology)

Room: Aula

Support Update

• Jonathan Procknow

Room: Aula

KIT's Site report

• Dorin Lobontu

Room: Aula Dorin

IN2P3 Site Status

• Pierre-Emmanuel BRINETTE (IN2P3 / CNRS)

Room: Aula ## Updates on HPSS at IN2P3 Computing Center - Infrastructure and HPSS upgrade - ARM architecture support

Transitioning HPSS Monitoring from Nagios to VictoriaMetrics

• Basil Lalli (NERSC - LBNL)

Room: Aula Last year at NERSC we retired our long-standing nagios-based HPSS monitoring deployment in favor of VictoriaMetrics, Loki and Alertmanager. We would like to share our experience and lessons learned on the way. - Motivation for making this transition - Limitations of Nagios-style monitoring - How does VictoriaMetrics address these? - General overview of our monitoring deployment - 3rd party exporters - Custom exporters/"plugins" - Demonstration of some of the dashboards we use and alerts we generate. - Future areas of improvement - Standardizing our HPSS-specific data collection - Service discovery

HPSS monitoring at KIT

• Preslav Konstantinov (KIT)

Room: Aula Preslav

Introduction to GridKa Tour

• Andreas Petzold (KIT)

Room: Aula

GridKa Tour 1 Room: Aula

GridKa Tour 2 Room: Aula

HPSS Release Roadmap (Restricted)

• Michael Meseke

Room: Aula

Implementing a Virtualized HPSS Deployment for Testing and Development

• Forrest Greenwood (HPSS Subsriber)

Room: Aula As part of our efforts to upgrade our site to HPSS 10.3, Indiana University recently began development of a virtualized HPSS environment that we can use to quickly iterate on testing and development initiatives without tying up limited bare-metal hardware resources. This virtual-machine environment is patterned on the implementation created by IBM's HPSS Support team for use at the recent HPSS 10.3 Training held in May 2024. Topics to be discussed include provisioning the VM, installing and configuring a virtual tape library using the mhVTL open-source software package, installing and configuring both DB2 and HPSS, and possibly a sampling of the sorts of issues we intend to test using this environment. If technical affordances permit, this presentation could potentially include a live demonstration of the VM running on an external flash drive. Otherwise, we would be happy to present using the traditional static PowerPoint.

HPSS S3 Scalability With Rubin LFA S3 Store Use Case

• Guangwei Che

Room: Aula SLAC National Accelerator Laboratory Technology and Inovation Department Scientific Computing Systems With a growing demand on HPSS S3 support from SLAC science user’s community, we eagerly started testing HPSS S3 interface since the pre-GA release in July 2023. From the initial fragile and immature version to today’s more robust and resilient state, we worked directly with HPSS S3 developer’s team to troubleshoot and triage many challenging issues faced with the scalability, data IO performance and the large and deeply nested data structure handling for very small files from Rubin LFA ceph S3 store use case. In this presentation we’ll tell our stories in the journey of bring HPSS S3’s capability to a next level.

Testing HPSS S3 Interface at MPCDF

• Elena Summer (Max Planck Computing and Data Facility (MPCDF))

Room: Aula Starting from version 10.3, HPSS has an S3 interface. We at MPCDF have installed it on our test system to try it out in several usage scenarios including cloud sync - using Ceph Cloud Sync module as well as rclone, generating presigned URLs, and just using different S3 clients. Among our test actions, we are trying out put, get, remove S3 objects as well as getting S3 objects from tape. This talk covers test scenarios, their setup and results, encountered issues and their fixes.

BOF Monitoring Room: Aula

NeRSC Site Report

• Owen James (NERSC/LBNL)

Room: Aula NERSC Site Report HUF24 Karlsruhe Institute of Technology Campus Nord from 09-12 September 2024 Abstract Topics to discuss - NERSC Stats - PBs, etc - Upgrade HPSS 7.4.3 to 9.3 New RHEL8 Core Servers. New FS7300 metadata arrays. Update existing movers to RHEL8. Updated PAM auth module to work with NERSC Auth. - Install 4th TS4500 tape library 16 Frame 1188+ slots. testing 10.0.1 firmware with SSL for REST over Ethernet. TS1160 drives JE media, while we figure out TS1170/JF. Total Theoretical capacity 950PB on JE, 2.37EB on JF media. - Issues deploying TS1170 in our air cooled environment - Monitoring update (brief, specific talk to follow) - REST over ethernet testing

Spectra Logic

• Matt Starr

Room: Aula

HPSS Object Storage Class Deep Dive

• Greg Thorsness

Room: Aula 2024 HUF Presentations by IBM !!duration 1h

Have I right-sized my disk cache?

• Francis Dequenne (LBL)

Room: Aula Abstract: For most sites, the HPSS disk cache is a critical component of the HPSS configuration, helping boosting performances of storing and retrieving data from the archive. However, it may be a bit of a black art to assess how big the disk cache should be, especially in environments that have grown over the tears. This talk will present a couple of tools that have been developed at NERSC, that allow us to assess the effectiveness of an existing cache, and give some insight on the impact of increasing or decreasing that disk cache size.

Upcoming HPSS Features (Restricted)

• Michael Meseke

Room: Aula 2024 HUF Presentations by IBM !! duration: 1,5h

Burning Issues (Restricted)

• Jonathan Procknow

Room: Aula 2024 HUF Presentations by IBM

Staging ~2 Million Files from Tape for a User

• Geoff Cleary (LLNL)

Room: Aula Imagine you turn on your work laptop or arrive at the office and find this message from the customer service team: "we have a user that is trying to retrieve *many* files from the HPSS archive. At the current retrieval rate, we estimate it will take 6 months for the user to retrieve all the files in the dataset. Can you help?" What do you do? How do you proceed? What features does HPSS offer to help with this situation? I'll answer those questions and more as we examine LLNL's approach to retrieving nearly 2 million files across 10's of tape volumes with tools like `quaid`, SQLite, and RabbitMQ (along with a bit of custom Python code).

BOF Client Interfaces Room: Aula

Group Foto Room: Aula

Exploring storage technologies for HPSS disk caches

• Andreas Petzold

Room: Aula At KIT we operate HPSS as a tape system for the GridKa WLCG Tier-1 and for the Baden-Württemberg Data Archive service. Performance limitations of the HPSS disk cache systems led us to explore new technology options for the disk cache, based on classic storage systems with SSDs, and storage servers with local NVMe devices. We will present details on the different possible solutions, including benchmarks.

Managing Data Throughout Its Lifecycle: Lessons Learned and Future Directions

• Charles McClary (HPSS Subscriber)

Room: Aula Abstract: Data lifecycle management poses significant challenges, particularly in academic and research environments where data accumulation is rapid and perpetual. This presentation delves into the complexities surrounding data retention and abandonment, highlighting the prevalent issues of data hoarding and the lack of structured deletion policies. Specifically, it addresses the dilemma wherein users, especially researchers, find little incentive to delete data, leading to a cluttered and often inaccessible data landscape. Furthermore, the departure of users from institutions like Indiana University (IU) exacerbates the problem, as data may be left behind with no clear ownership or accessibility. Indiana University is tackling these issues gradually. We'll discuss our efforts to address data management and abandonment through: New usage constraints: Instituting new quotas with tiered growth guidelines. Simplified Archiving and Movement: Providing user-friendly tools to archive and migrate data to appropriate storage tiers. Data Management Education: Empowering researchers with best practices for data stewardship. Insuring allocation value: Requiring annual renewal of desired resources. The "Digital Will" Concept: Developing a system where departing users can designate data inheritors and define deletion policies. By examining the successes and pitfalls of these initiatives, this presentation provides valuable insights into effective data lifecycle management strategies. It underscores the importance of fostering a culture of responsible data stewardship while leveraging technological innovations to facilitate seamless data management throughout its lifecycle.

HPSS Core Servers on Commodity Hardware or: How We Learned to Love Databases on ZFS

• Herb Wartens

Room: Aula At LLNL we have been using commodity hardware more and more to serve our parallel filesystems and archival storage clusters. We wanted to explore how to use this same hardware for our HPSS Core Server systems. In order to make the system as reliable as possible, ZFS emerged as the underlying filesystem we wanted to utilize for its reliability and other advanced features. How would traditional databases perform on top of ZFS? Could we design a production-worthy system using this hardware?

ZKM Tour Room: Lorenzstraße 19 Karlsruhe 76135

Generative AI and HPSS

• Greg Thorsness

Room: Aula 2024 HUF Presentations by IBM !! duration 1h

MPCDF Site Report

• Manuel Panea (Max Planck Computing and Data Facility)

Room: Aula We will present our activities with HPSS since the last HUF, including our upgrade to HPSS 10.3.

Restful SSM

• Fabi Adams

Room: Aula 2024 HUF Presentations by IBM !!duration 1h

SSC Site Report

• Tarak Patel

Room: Aula SSC Site Report will be a recap of previous HUF presentations and focus on : - Solution overview - Review of components throughout upgrades - HPNLS (High Performance Nearline Storage) architecture - HPSS and RHEL Upgrade - HPSS monitoring - User tools and environment

JAXA Site Report

• Naoyuki FUJITA (Japan Aerospace Exploration Agency(JAXA))

Room: Aula The recent operation status of JAXA HPSS "J-SPACE", the plans and issues for its replacement in 2025, and monitoring functionality will be reported.

Closing HUF 2024

• Doris Ressmann (Karlsruhe Institute of Technlology)

Room: Aula