9.2.2 Lanzhou University

1 Lanzhou University Computing Environment¶

1.1 Account Application¶

The Lanzhou University computing cluster uses an application and approval process for user accounts. Users can apply for an account by filling in personal information through this form. Accounts are valid for one year.

For any account-related issue, contact lzuhep_exp_pub@outlook.com.

Login nodes are the only entry points for users to edit files, debug programs, and submit jobs. Password login and SSH key login are supported.

No.	Login node OS	Count	Domain name
1	AlmaLinux 9.4 (64 bit)	1	lzulogin01.hep.lzu.edu.cn
2	AlmaLinux 9.4 (64 bit)	1	lzulogin02.hep.lzu.edu.cn
3	AlmaLinux 9.4 (64 bit)	1	lzulogin03.hep.lzu.edu.cn
4	AlmaLinux 9.4 (64 bit)	1	lzulogin04.hep.lzu.edu.cn

Users can log in through username@lzulogin.hep.lzu.edu.cn; the cluster will route the session to the login node with the lowest current load.

1.3 Password Change¶

Password requirements:

Do not use weak passwords. Use at least 10 characters with four or more character types, including uppercase letters, lowercase letters, digits, and special characters.
Do not reuse the same password on public external systems such as WeChat, QQ, websites, or apps.
Change the password regularly, at least every six months.
Do not lend your account or password to others.

Run kpasswd on a login node to change the password.

If you need to reset the password, contact lzuhep_wangyh@lzu.edu.cn.

1.4 Job Submission and Query¶

The Lanzhou University computing cluster uses HTCondor as its job scheduler. HTCondor supports high-throughput workloads and is suitable for high-energy physics computing. For more details, refer to the HTCondor job guide.

1.4.1 Load the Job Environment¶

Before using HTCondor, add the following commands to your personal bashrc:

$ export PATH=/cvmfs/common.ihep.ac.cn/software/hepjob/bin/:$PATH
$ export PATH=/cvmfs/container.ihep.ac.cn/bin:$PATH

Programs expected to run for more than one hour should be submitted as batch jobs. The job environment must be loaded before submission.

1.4.2 Submit CPU Jobs¶

$ hep_sub job.sh
$ hep_sub job.sh -os SL7
$ hep_sub job.sh -g lzuhep

1.4.3 Submit GPU Jobs¶

$ hep_sub job.sh -gpu 1 -cpu 1

The gpu01 node has two NVIDIA RTX2080 Ti GPUs and is intended only for foreground GPU job debugging.

1.4.4 Query Job Status¶

$ hep_q -u username
$ hep_q -i 9527.0

1.4.5 Delete Jobs¶

$ hep_rm 9527.0
$ hep_rm -a
$ hep_rm 9527.0 -forcex
$ hep_rm 9527

1.4.6 Release Held Jobs¶

$ hep_q -u username -hold
$ hep_release -a

1.4.7 Wall Time¶

The default job wall-time limit is 100 hours. If a job is expected to run longer, submit it to the long queue:

$ hep_sub job.sh -wt long

The long queue limit is 30 days. Contact the administrator in advance if you need to submit many long jobs.

1.4.8 Memory¶

The default memory limit is 3 GB. Specify memory in MB when needed:

$ hep_sub job.sh -mem 3000

1.4.9 CPU Threads¶

CPU jobs use one thread by default:

$ hep_sub job.sh -cpu 1

A single job cannot span multiple compute nodes, so the requested thread count must not exceed the number of threads on one node.

1.4.10 Job Information Environment Variables¶

The following variables can be used in job scripts:

$_CONDOR_IHEP_JOB_ID
$_CONDOR_IHEP_REMOTE_HOST
$CUDA_VISIBLE_DEVICES
$_CONDOR_AssignedGPUs

1.5 File Storage Service¶

This section describes user file storage and CVMFS software storage available on the cluster.

1.5.1 User File Storage¶

Directory	Purpose	Accessible from	Quota
/lzufs/home/USERNAME	User home directory	All nodes	200 GB
/lzufs/user/USERNAME	User data	All nodes	21 TB

1.5.2 CVMFS Software Storage¶

CernVM-FS provides scalable, reliable, and low-maintenance software distribution for high-energy physics applications. It is implemented as a POSIX read-only file system in user space and is mounted under /cvmfs for jobs to read software.

The CVMFS system currently provides software repositories for multiple experiments:

Path	Purpose
/cvmfs/bes3.ihep.ac.cn	BES3 software
/cvmfs/bes.ihep.ac.cn	BES software
/cvmfs/cepc.ihep.ac.cn	CEPC analysis software
/cvmfs/dcomputing.ihep.ac.cn	Distributed computing analysis software
/cvmfs/gluex.ihep.ac.cn	GlueX analysis software
/cvmfs/heps_ap.ihep.ac.cn	HEPS software
/cvmfs/juno.ihep.ac.cn	JUNO analysis software
/cvmfs/lhaaso.ihep.ac.cn	LHAASO analysis software
/cvmfs/mlgpu.ihep.ac.cn	GPU machine learning software
/cvmfs/lhcb.cern.ch	LHCb software

🤖 AI 助手