2. Quick Start¶

User Account¶

Users need a computing environment account before they can use computing resources.

Account Application¶

Users with a unified authentication account can apply for an AFS account at https://login.ihep.ac.cn.
New users without unified authentication can apply online at https://login.ihep.ac.cn/regist.jsp.

Note

The cluster password is the same as the unified authentication password. No separate password setup is required. If the unified authentication password is changed, the cluster password will be synchronized.

Linux, macOS, and Windows can all be used to log in to the cluster. Linux or macOS is recommended.

For Windows 10 and later, Windows Terminal is recommended. For other Windows versions, install third-party SSH client software. For more details, refer to Account Management.

Linux/MacWindows

ssh -l [username]@lxlogin.ihep.ac.cn

# Enter the following command in an SSH client
ssh -l [username]@lxlogin.ihep.ac.cn

File Management¶

File Directories¶

After login, users enter their home directory by default, such as /afs/ihep.ac.cn/users/xxx/xxx.

The home directory contains a .bashrc file, which is executed automatically when the user logs in.

Recommended personal file storage locations:

Directory	Stored content	Per-user quota
/afs/ihep.ac.cn/users/xxxx/[username]	Login configuration files	500 MB
/scratchfs/xxxx	Temporary user files	500 GB
/workfs/xxxx	Personal user files	50 GB

Experiment data should be stored in the corresponding experiment directories. See File Systems for detailed storage information and quotas.

File Upload and Download¶

ftpsftp

# Upload files
lftp ftp://lxlogin.ihep.ac.cn/afs -u [username] [password]

# Download files
lftp> mget [filename]

# Upload files
sftp -oPort=20 lxlogin.ihep.ac.cn

# Download files
sftp> get [filename]

Job Submission¶

HTCondor Jobs¶

HTCondor is an open-source batch system widely used in high-energy physics computing. It supports large-scale computing clusters and is flexible, making it especially suitable for high-throughput job scheduling at IHEP. Based on the cluster environment, the Computing Center has optimized HTCondor functionality and performance, developed centralized management and monitoring tools, and deployed a high-throughput scheduling system based on HTCondor. The HTCondor system currently supports most IHEP experiments, including BES, JUNO, DYW, LHAASO, and HXMT.

bash userstcsh users

$ export PATH=/afs/ihep.ac.cn/soft/common/sysgroup/hep_job/bin/:$PATH

$ setenv PATH /afs/ihep.ac.cn/soft/common/sysgroup/hep_job/bin/:$PATH

Prepare a job script.

Upload the job script to the corresponding directory, such as /workfs/xxx/test.sh, or enter the directory and create the script with vi test.sh.

A simple shell script:
```
#!/bin/sh
echo "Hello, World!"
```

Set executable permission for the script.

# Check script permissions
$ ls -l test.sh
-rw-r--r-- 1 xxx xxx 23 08-27 16:54 test.sh

# Modify script permissions
$ chmod +x test.sh

# Check permissions again
$ ls -l test.sh
-rwxr--r-- 1 xxx xxx 23 08-27 16:54 test.sh

Submit the job.

$ hep_sub test.sh -g [group_name] # group_name is the resource group to use

Check job status.

$ hep_q -u [username] # username is the user whose jobs you want to query

Example output:

JOBID  OWNER    SUBMITTIME         RUN_TIME      ST    PRI SIZE        CMD
5519.0  xxx      2019-08-27 16:54   0+00:00:00     0       0.0         test.sh

JOBID is the job ID, OWNER is the job owner, SUBMITTIME is the submission time, RUN_TIME is the running time, ST is the status, PRI is the priority, SIZE is the virtual memory usage, and CMD is the job program name.

Cancel jobs.

Delete jobs by job ID. Multiple jobs can be deleted at the same time. For example, to delete jobs 5519 and 5520:
```
$ hep_rm 5519.0 5520.0
```
Delete all jobs of the current user:
```
$ hep_rm -a
```

Slurm Jobs¶

Slurm is an open-source cluster manager and job scheduling system. It supports multiple types of compute nodes, including GPU and CPU nodes.

Slurm job submission workflow:

Prepare a job script.

Use an editor such as vim or emacs to edit the job script. Sample job scripts are available under /cvmfs/common.ihep.ac.cn/software/slurm_sample_script.
```
$ ssh <AFS_user_name>@lxlogin.ihep.ac.cn
$ cd /cvmfs/common.ihep.ac.cn/software/slurm_sample_script
$ ls -lht
```
Figure 1-4 Slurm template directory

The simplest sample script can be viewed with:
```
$ cat slurm_sample_script_2.sh
```
Figure 1-5 Slurm job script template

In the script:
1. Define the required variables, such as VAR1 and VAR2.
2. If you need to run your own script, use srun and replace /path/to/your/script with the actual script file name.
3. If you need to run a parallel program, use srun first to trigger resource allocation, then use mpiexec to run the parallel program. Replace /path/to/your/mpi_program with the actual path of your parallel program.
Submit the job.

After editing the script, submit it with:
```
$ hep_sub -slurm -part <partition_name> -g <group_name> -np <cpu_core_number> slurm_sample_script_2.sh
```
Where:
1. <partition_name> is the queue name. Available queues can be queried with:
```
$ hep_clus -slurm
```
  Example output:
  
  Figure 1-6 Example output of hep_clus
  
  Available queue names are shown in the first column. If you are not sure which queue to use, submit to the comp queue, which is the default cluster queue.
2. <group_name> is the group to which the job belongs. For example, if the job belongs to the offline group of the BES experiment, use -g offline.
3. <cpu_core_number> is the number of CPU cores required by the job. For example, if 30 CPU cores are required, use -np 30.
To submit slurm_sample_script_2.sh to the comp queue, for the offline group, requesting 30 CPU cores:
```
$ hep_sub -slurm -part comp -g offline -np 30 slurm_sample_script_2.sh
```
Note: the queue name and job group name must be provided, otherwise job submission will fail.
Check job status.

After submission, hep_sub returns a job ID. Use this ID to query the job:
```
$ hep_q -slurm -i <job_id>
```
<job_id> is the ID returned by hep_sub.
Get job results.

After the job finishes, if no output directory is specified, the default job result is saved in the submission directory. The output file format is <job_id>.out. For example, if the job ID is 1234, the output file is 1234.out.

Software Usage¶

The platform uses the module command to load software environments. For detailed usage, refer to Environment Configuration.

The following example uses Python.

Load the Python environment:

$ module load python/3.7

Verify the Python environment:

$ python
Python 3.7.0 (default, Aug 27 2019, 16:54:48)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Exit Python:

>>> exit()
$

FAQ¶

See FAQ.

🤖 AI 助手