slurm_report_two

CS SLURM Cluster Report - 8 Weeks

CS SLURM Cluster Report - 8 Weeks

Report generated for jobs run on the CS SLURM cluster from 2024-09-15 through 2024-11-09.

Job total during this query range: 275,695

Job total since August 1st 2024: 607,152

This page is updated every Sunday at 5:00pm EST.

SLURM Scheduler System Output

--------------------------------------------------------------------------------
Cluster Utilization 2024-09-15T00:00:00 - 2024-11-09T23:59:59
Usage reported in TRES Hours/Percentage of Total
--------------------------------------------------------------------------------
  Cluster      TRES Name               Allocated                  Down         PLND Down                     Idle             Planned                 Reported 
--------- -------------- ----------------------- --------------------- ----------------- ------------------------ ------------------- ------------------------ 
       cs            cpu          581608(13.14%)          12824(0.29%)          0(0.00%)          3379460(76.35%)      452140(10.22%)          4426032(73.16%) 
       cs            mem      5350893591(15.48%)       32603151(0.09%)          0(0.00%)      29191280037(84.43%)            0(0.00%)      34574776778(73.13%) 
       cs       gres/gpu           78196(38.39%)             21(0.01%)          0(0.00%)           125471(61.60%)            0(0.00%)           203688(73.16%) 

* Total Cluster Resources Avaialble by Partition
 (Note, TRES is short for Trackable RESources)
PartitionName=cpu
   TRES=cpu=1494,mem=12594000M,node=41
PartitionName=gpu
   TRES=cpu=2226,mem=19710000M,node=37,gres/gpu=159
PartitionName=nolim
   TRES=cpu=354,mem=864000M,node=7
PartitionName=gnolim
   TRES=cpu=424,mem=2008000M,node=16,gres/gpu=48

SLURM Usage by Partition

PartitionName	total_jobs	cputime(HH:MM:SS)	completed	cancelled	failed	timeout	out_of_memory	node_fail
gpu	263953	432454:07:50	12191	1889	249717	105	44	7
cpu	7739	107616:19:42	6309	477	899	17	37	0
gnolim	3988	44900:41:56	1894	1262	824	3	5	0
nolim	15	39:43:26	10	1	3	0	1	0

SLURM Usage by Advisor Group

slurm-cs-undefined, users that have CS accounts but are not CS students
slurm-cs-unassigned, users that are CS students but do not have a listed CS advisor

GroupName	total_jobs	cputime(HH:MM:SS)	cpu	gpu	nolim	gnolim	completed	cancelled	failed	timeout	out_of_memory	node_fail
slurm-cs-zezhou-cheng	599	139815:41:18	0	599	0	0	489	63	43	4	0	0
slurm-cs-yu-meng	87	96043:06:34	0	87	0	0	31	20	17	13	2	4
slurm-cs-tianhao-wang	374	71021:52:50	7	294	0	73	92	126	140	11	4	1
slurm-cs-lu-feng	6497	59896:21:14	0	4055	0	2442	2980	1507	2009	1	0	0
slurm-cs-chen-yu-wei	2942	57652:40:38	2942	0	0	0	2234	73	634	1	0	0
slurm-cs-hongning-wang	42	28233:49:18	0	42	0	0	9	17	8	6	0	2
slurm-cs-haiying-shen	4005	24763:29:54	1158	2847	0	0	3916	88	0	1	0	0
slurm-cs-shangtong-zhang	42	22518:17:40	20	22	0	0	8	6	16	11	1	0
slurm-cs-kevin-skadron	1953	18223:56:20	194	1749	10	0	1316	122	489	24	2	0
slurm-cs-yen-ling-kuo	180	17544:18:02	4	176	0	0	42	34	89	10	5	0
slurm-cs-undefined	12561	11623:59:30	1993	9189	4	1375	8139	667	3734	7	14	0
slurm-cs-yangfeng-ji	244395	9108:02:44	244	244151	0	0	3	625	243766	0	1	0
slurm-cs-daniel-graham	322	6920:43:14	81	241	0	0	153	55	95	17	2	0
slurm-cs-ferdinando-fioretto	504	4613:50:43	453	51	0	0	294	145	52	9	4	0
slurm-cs-unassigned	98	3811:50:44	19	79	0	0	28	4	62	1	3	0
slurm-cs-farzad-farnoud	28	3580:53:08	4	24	0	0	7	9	8	0	4	0
slurm-cs-sebastian-elbaum	671	3118:34:02	465	117	0	89	527	10	127	2	5	0
slurm-cs-madhur-behl	168	3008:17:02	101	65	0	2	83	28	42	1	14	0
slurm-cs-adwait-jog	63	1715:41:37	40	22	1	0	20	13	9	4	17	0
slurm-cs-ashish-venkat	12	858:57:12	12	0	0	0	6	0	5	0	1	0
slurm-cs-xiaozhu-lin	38	731:51:56	0	38	0	0	17	2	16	1	2	0
slurm-cs-geoffrey-fox	25	192:05:16	0	25	0	0	4	4	13	1	3	0
slurm-cs-mary-lou-soffa	58	12:07:58	0	58	0	0	0	11	44	0	3	0
slurm-cs-miaomiao-zhang	24	00:17:48	2	22	0	0	6	0	18	0	0	0
slurm-cs-rich-nguyen	7	00:06:12	0	0	0	7	0	0	7	0	0	0

SLURM Usage by NodeName

Nodename	total_jobs	cputime(HH:MM:SS)	completed	cancelled	failed	timeout	out_of_memory	node_fail
cheetah06	580	133121:08:24	493	58	28	1	0	0
serval01	156	125704:49:10	47	39	41	21	2	6
jaguar03	10208	24297:02:56	729	113	9358	6	2	0
cheetah04	1156	22372:55:10	190	22	932	11	1	0
cortado01	1203	21198:41:22	933	31	239	0	0	0
cortado08	1125	20758:04:04	843	25	256	0	1	0
puma01	379	18959:39:08	360	11	1	5	2	0
jaguar01	124	16289:14:40	30	34	53	3	4	0
bigcat01	1803	14644:40:30	1459	157	176	4	7	0
cheetah05	1410	13340:12:48	600	65	733	9	3	0
jaguar04	80	13174:37:52	34	24	16	6	0	0
lotus	6091	11457:42:40	496	64	5521	4	5	1
sds02	12258	6923:35:48	682	122	11448	4	2	0
hydro	413	6364:21:58	372	16	20	0	5	0
sds01	7921	6278:34:58	605	77	7238	1	0	0
puma02	64	6164:01:36	7	16	32	4	5	0
lynx08	624	5384:45:16	458	44	110	2	10	0
jaguar02	5569	4754:31:54	980	123	4460	6	0	0
ai02	212	4430:55:48	88	94	30	0	0	0
ai03	210	4344:54:36	88	90	32	0	0	0
ai04	198	4218:04:16	86	85	27	0	0	0
ai01	212	3988:19:00	99	79	32	1	1	0
jaguar05	11676	3765:20:50	469	117	11076	9	5	0
ai08	288	3692:38:30	151	111	26	0	0	0
ai07	306	3579:22:48	168	105	33	0	0	0
ai05	293	3550:20:14	158	89	45	1	0	0
cheetah01	1012	3464:20:30	781	69	150	10	2	0
ai06	263	3138:37:30	138	81	43	1	0	0
ai09	428	2945:49:04	183	94	151	0	0	0
panther01	134	2939:53:44	96	20	12	1	5	0
ai10	412	2897:42:26	167	93	152	0	0	0
adriatic01	10984	2891:29:52	535	67	10381	1	0	0
cheetah02	4135	2813:32:18	308	35	3790	0	2	0
adriatic02	10378	2470:02:58	551	56	9771	0	0	0
adriatic03	9782	2460:58:22	546	52	9184	0	0	0
adriatic04	9600	2326:51:16	512	52	9036	0	0	0
adriatic06	9402	2268:22:12	495	42	8863	2	0	0
jaguar06	1583	2256:37:14	362	28	1192	1	0	0
adriatic05	9778	2108:54:00	509	46	9222	0	1	0
affogato13	11420	2086:15:44	208	60	11151	1	0	0
jinx01	284	2083:07:16	142	87	55	0	0	0
affogato11	11736	2059:54:50	202	66	11466	0	2	0
jinx02	265	1983:38:12	137	84	43	0	1	0
affogato15	11646	1968:21:08	197	63	11385	1	0	0
affogato14	11463	1955:15:46	191	62	11210	0	0	0
lynx02	6517	1662:41:56	103	18	6395	0	1	0
cheetah03	3820	1571:03:10	307	29	3483	1	0	0
lynx01	7919	1480:12:26	115	19	7784	1	0	0
lynx05	5589	1439:56:20	97	19	5472	1	0	0
lynx11	15296	1252:38:38	130	45	15121	0	0	0
lynx12	16125	1231:58:52	123	50	15952	0	0	0
lynx10	11491	1195:31:42	102	46	11341	1	1	0
lynx03	7016	1191:03:18	101	20	6893	0	2	0
lynx09	269	1181:33:48	197	40	32	0	0	0
titanx03	148	1173:28:42	75	42	30	0	1	0
lynx04	6836	1169:06:16	109	19	6706	0	2	0
cortado04	103	1137:02:00	94	8	0	1	0	0
cortado03	104	1112:13:38	95	8	1	0	0	0
cortado02	114	1062:08:28	103	9	2	0	0	0
titanx02	158	1006:10:02	64	43	49	0	2	0
slurm3	87	994:53:38	80	4	3	0	0	0
slurm2	95	978:33:22	83	11	0	0	1	0
titanx04	140	974:45:22	73	40	27	0	0	0
slurm4	79	961:01:46	77	1	0	0	1	0
cortado05	93	944:33:04	81	8	3	1	0	0
titanx05	153	892:48:10	77	41	35	0	0	0
cortado09	141	880:28:16	134	6	1	0	0	0
lynx06	6637	766:40:44	131	22	6483	0	1	0
lynx07	6439	727:22:04	116	15	6307	0	1	0
cortado10	121	598:45:20	110	8	2	0	1	0
slurm1	87	589:11:56	71	13	1	0	2	0
affogato02	43	533:46:48	38	4	1	0	0	0
cortado06	46	525:57:32	34	8	2	2	0	0
affogato04	51	522:14:54	47	4	0	0	0	0
slurm5	41	490:41:40	36	4	0	0	1	0
affogato01	78	350:19:00	65	12	1	0	0	0
cortado07	52	344:39:22	41	8	2	1	0	0
affogato10	40	321:32:10	36	3	0	0	1	0
affogato06	30	305:24:28	28	2	0	0	0	0
affogato07	39	294:53:02	37	2	0	0	0	0
affogato05	29	290:01:12	26	3	0	0	0	0
affogato08	33	270:19:28	31	2	0	0	0	0
affogato09	32	261:40:10	30	2	0	0	0	0
struct08	21	248:32:14	18	3	0	0	0	0
struct03	19	245:53:02	19	0	0	0	0	0
struct02	21	245:24:06	21	0	0	0	0	0
struct01	19	241:37:12	19	0	0	0	0	0
struct06	18	238:01:08	18	0	0	0	0	0
struct04	18	237:22:54	18	0	0	0	0	0
struct10	18	236:06:10	18	0	0	0	0	0
struct09	19	235:42:38	19	0	0	0	0	0
struct07	18	234:11:50	18	0	0	0	0	0
struct05	19	233:00:02	19	0	0	0	0	0
heartpiece	4	23:08:30	2	1	0	0	1	0
affogato03	28	18:27:22	27	0	1	0	0	0
doppio03	1	16:30:48	1	0	0	0	0	0
doppio04	1	00:03:36	0	0	1	0	0	0
epona	6	00:00:30	6	0	0	0	0	0
doppio05	1	00:00:02	1	0	0	0	0	0
doppio01	0	00:00:00	0	0	0	0	0	0
doppio02	0	00:00:00	0	0	0	0	0	0

Table of Contents

CS SLURM Cluster Report - 8 Weeks

SLURM Scheduler System Output

SLURM Usage by Partition

SLURM Usage by Advisor Group

SLURM Usage by NodeName