Index

3.2 Local Computing Cluster Overview

IHEP provides two main local computing clusters:

HTCondor computing cluster
Slurm computing cluster

The HTCondor cluster supports high-throughput computing and uses HTCondor as its workload management system. Most jobs on this cluster are single-core serial jobs.

The Slurm cluster supports high-performance computing and uses Slurm as its workload management system. Most jobs on this cluster are multi-core parallel jobs, including CPU and GPU workloads.

This section introduces basic operations such as job submission, job query, and job management for these clusters.

HTCondor Quotas per Experiment¶

The HTCondor cluster uses a three-level group hierarchy (Experiment.Group.Queue) for resource partitioning and quota management. For example, LHAASO.lhaaso.default refers to the default queue of the lhaaso group under the LHAASO experiment.

Level 1 (Experiment): An experiment or project, e.g., LHAASO, JUNO, BES.
Level 2 (Group / Account): A subdivision inside an experiment based on user group or use case, e.g., dqarun, offlinerun, physics under BES.
Level 3 (Queue): Usually one of default (normal jobs), mid (medium-duration jobs), special (special jobs), or long (long-running jobs). Some experiments only use a subset of these queues.

The tables below summarize each experiment's HTCondor quotas (unit: cores / job slots). The data is sourced from the configuration files under /etc/condor/config.d/41-GROUP-*.conf on the config manager node cm01.

Per-Experiment Quota Summary¶

Experiment	Experiment Total Quota
AliCPT	120
ATLAS	1840
BES	20000
CC (u07)	25000
CEPC	832
CMS	836
COMET	1
CSNS	10
DYW	568
EP	128
eXTP	256
GECAM	576
HEPS	10
HERD	1268
HXMT	1936
JUNO	5936
LHAASO	12480
LHCb	584
MoEDAL	100
OTHERS	760
Panda	224
COMMON	131

Detailed Quotas per Experiment¶

The tables below list the quota breakdown for each experiment. A - means that queue is not configured for the group.

About quota discrepancies

For some experiments, the sum of Level-2 (group) quotas differs from the Level-1 (experiment) total quota (e.g., under ATLAS the atlas group total is shown as 140 while its default queue is as high as 1425). This is a consequence of the GROUP_ACCEPT_SURPLUS mechanism in the configuration, which lets groups borrow idle resources from others. Refer to the configuration files for the authoritative values.

AliCPT¶

Experiment Total Quota: 120

Group	Group total	default	mid	special	long
alicpt	120	88	12	10	10

ATLAS¶

Experiment Total Quota: 1840

Group	Group total	default	mid	special	long
atlas	140	1425	43	10	60
combination	200	20	176	10	50

BES¶

Experiment Total Quota: 20000

Group	Group total	default	mid	special	long
dqarun	500	430	50	10	10
offlinerun	1200	1060	120	10	10
physics	17104	15544	1500	50	10

CC¶

Experiment Total Quota: 25000

Group	Group total	default	mid	special	long
u07	25000	24000	100	10	10

CEPC¶

Experiment Total Quota: 832

Group	Group total	default	mid	special	long
higgs	832	750	62	10	10

CMS¶

Experiment Total Quota: 836

Group	Group total	default	mid	special	long
cms	836	736	80	10	10

COMET¶

Experiment Total Quota: 1

Group	Group total	default	mid	special	long
comet	1	1	1	1	1

CSNS¶

Experiment Total Quota: 10

Group	Group total	default	mid	special	long
csns	10	10	-	-	-

DYW¶

Experiment Total Quota: 568

Group	Group total	default	mid	special	long
dyw	568	348	200	10	10

EP¶

Experiment Total Quota: 128

Group	Group total	default	mid	special	long
ep	128	116	12	10	10

eXTP¶

Experiment Total Quota: 256

Group	Group total	default	mid	special	long
extp	1	1	1	1	1

GECAM¶

Experiment Total Quota: 576

Group	Group total	default	mid	special	long
gecam	576	499	57	10	10

HEPS¶

Experiment Total Quota: 10

Group	Group total	default	mid	special	long
heps	10	10	-	-	-

HERD¶

Experiment Total Quota: 1268

Group	Group total	default	mid	special	long
herd	768	700	68	5	5
herdrun	500	500	-	-	-

HXMT¶

Experiment Total Quota: 1936

Group	Group total	default	mid	special	long
hxmt	1936	1400	500	20	16

JUNO¶

Experiment Total Quota: 5936

Group	Group total	default	mid	special	long
juno	2000	1900	50	30	20
dqmtest	0	0	0	0	0
dqmjuno	1024	1000	14	-	10
junospecial	0	0	0	0	0
junodc	1048	1048	0	0	0
junogns	1664	1600	50	10	4
tao	200	180	10	1	1

LHAASO¶

Experiment Total Quota: 12480

Group	Group total	default	mid	special	long
lhaaso	12480	5480	2100	400	4500
lhaasorun	1	1	1	1	1

LHCb¶

Experiment Total Quota: 584

Group	Group total	default	mid	special	long
lhcb	584	506	58	10	10

MoEDAL¶

Experiment Total Quota: 100

Group	Group total	default	mid	special	long
moedal	100	90	5	-	5

OTHERS¶

Experiment Total Quota: 760

Group	Group total	default	mid	special	long
ucas	760	664	76	10	10
guest	20	20	-	-	-

Panda¶

Experiment Total Quota: 224

Group	Group total	default	mid	special	long
panda	224	184	20	10	10

COMMON (Shared Test and Short Queues Across Experiments)¶

Group Total Quota: 131

The COMMON group is a special case: its Level 3 does not represent a queue tier (default/mid/…), but rather experiment codes, used for test jobs (test.*) and short jobs (short.*) shared across experiments.

Group	Group total	Quota per sub-queue
COMMON.test	11	10
COMMON.short	120	100

Sub-queues (Level 3): alicpt, atlas, physics, cms, dyw, ep, extp, gecam, hxmt, juno, lhaaso, lhcb, panda, guest, higgs, u07, comet, herd

Querying live configuration

The data above reflects the current configuration on the cluster management node cm01, and may change according to experiment needs or resource reallocations. To fetch the latest values, run the following on an account with administrator privileges:

ssh cm01
ls /etc/condor/config.d/41-GROUP-*.conf
cat /etc/condor/config.d/41-GROUP-<experiment>.conf

🤖 AI 助手