Self-Adaptive dCache

Self-Adaptive dCache

Heatmap von dCache-Pool-Zugriffen eines Nutzers oder einer Nutzerin im Verlauf einer Woche. Farbabstufungen veranschaulichen die Auslastung der Pools. Von Grün nach Rot steigt die Auslastung an.

Experiments at Deutsches Elektronen-Synchrotron (DESY) produce large-scale data which need to
be stored and distributed globally for examination and analysis. The dCache framework allows to
set-up and control distributed data storages and is conjointly developed at DESY, the Nordic Data
Grid Facility (NeIC, NDGF), and the Fermi National Accelerator Laboratory.

The dCache system is a fundamental building block of the WLCG storage infrastructure,
supporting all major experiments in the fields of High Energy and Astro-Physics. The
majority of the large data centres for the LHC computing uses dCache and overall 60%
of the LHC data is stored on dCache systems. Hence dCache is an essential component
for all results published by the LHC collaborations, e.g. the discovery of the Higgs
Boson, but also the Flavour-Anomalies that are the best indicator for New Physics. DESY alone
operates about 100PiB of disk storage using dCache for High Energy Physics,
accelerator research and operations as well as research with photons. Through dCache
most of the data produced by the latter two are archived on tape as well. Also, the data taken
during the Corona research in 2020/2021 are stored and archived on dCache. Thus, managing the
flow of petabyte-scale data is a challenging and pivotal task. While dCache allows operating
networks of storage nodes, which is of central importance, it still requires manual administration
and configuration.

The development of self-adaptive systems is an active research field, which aims at engineering
computational systems which can adapt themselves at run-time. Besides providing frameworks for
implementing these systems it is particularly challenging to enable run-time reasoning about
appropriate adjustments in changing execution contexts. A prime application example is the
management of computational resources, e.g. data centers. This project aims at initiating
sustainable collaborations between DESY and HAW in order to study how reliable adaptive systems
can be engineered for automating the administration of dCache-based storage systems.
Within the project a testbed shall be created as a starting point and basis for the ongoing
development of autonomous self-adaptive algorithms.

 

Duration
-
Budget
11.700
Funding
Hamburg’s Ministry of Science, Research, Equality and Districts (BWFGB)
Unit
Faculty of Engineering and Computer Science
x