8.4 EOS File Storage¶

8.4.1 EOS File System Overview¶

EOS is a distributed file system for EB-level data storage based on the XRootD framework. It currently provides massive data storage services for multiple experiments, including LHAASO and HXMT.

Current EOS file system information is shown below.

Instance	Mount Point	Instance Server Address	Space	Usage
LHAASO local	/eos	eos01.ihep.ac.cn	2.39 PB	LHAASO local offline data processing
LHAASO Daocheng	/eos/daocheng	lhmteos01.ihep.ac.cn	152.83 TB	LHAASO Daocheng online data processing
HXMT	/mnt/hxmt	hxmteos01.ihep.ac.cn	382.25 TB	HXMT experiment storage

8.4.2 EOS File System Usage¶

1) EOS Usage¶

EOS currently provides two access modes: client FUSE mode and XRootD mode.

Access Methods¶

a) FUSE access

This method accesses the EOS system through a mount point, similar to accessing a local file system. For different experiment storage instances, mount points are shown in the table above. For example, the mount directory for the LHAASO experiment is /eos. On login nodes, data in the EOS file system can be accessed through the mount point.

b) XRootD access

This method accesses the EOS system through the XRootD protocol.

root://EOS_MGM_URL//filepath

EOS_MGM_URL is the server address of the instance. Users can use XRootD commands or EOS commands. For example, to list a file in the LHAASO instance:

xrdfs root://eos01.ihep.ac.cn ls /eos/user/file.txt

Or:

eos root://eos01.ihep.ac.cn ls /eos/user/file.txt

Note: If user jobs access EOS files through XRootD, the experiment software must support the XRootD protocol.

2) View User Resource Quota¶

Command format:

eos quota your_eos_path

Example:

eos quota /eos/user/z/zhangsan
By user ...
# _______________________________________________________________________________________________
# ==> Quota Node: /eos/user/z/zhangsan/
# _______________________________________________________________________________________________
user       used bytes logi bytes used files aval bytes aval logib aval files filled[%]  vol-status ino-status
lihaibo    800 GB  800 GB  10 k-   1.00 TB    200GB    25M -  9.08       ok         ok
        (used space)          (number of used files) (space quota) (number of file quota)
By group ...
# _______________________________________________________________________________________________
# ==> Quota Node: /eos/user/z/zhangsan/
# _______________________________________________________________________________________________
# ...............................................................................................
group      used bytes logi bytes used files aval bytes aval logib aval files filled[%]  vol-status ino-status
u07        800 GB  800 GB  10 k-   0 B        0 B        0 -        100.00     ignored    ignored

3) View EOS Recycle Bin Usage¶

Command format:

eos recycle help
Support ls, purge, restore three operations.

View files in a user's personal recycle bin:

eos recycle ls
# Deletion Time            UID      GID      SIZE         TYPE          RESTORE-KEY      RESTORE-PATH
# ==============================================================================================================================
Sat Dec 29 16:32:03 2018   zhangsan  u07      13           file          0000000008b0f7bf /eos/user/z/zhangsan/test.txt

Clean files in a user's personal recycle bin:

eos recycle purge

Restore files from a user's personal recycle bin:

eos recycle restore [--force-original-name|-f] [--restore-versions|-r] <recycle-key>

Example:

eos recycle restore 0000000008b0f7bf

Note: Files in the EOS recycle bin are currently kept for only three days.

🤖 AI 助手