slurm_report_two_months

CS SLURM Cluster Report - 8weeks

Report generated for jobs run on the CS SLURM cluster from 2024-09-22 through 2024-11-16.

Job total during this query range: 294,951

Job total since August 1st 2024: 627,060

This page is updated every Sunday at 5:00pm EST.

SLURM Scheduler System Output

--------------------------------------------------------------------------------
Cluster Utilization 2024-09-22T00:00:00 - 2024-11-16T23:59:59
Usage reported in TRES Hours/Percentage of Total
--------------------------------------------------------------------------------
  Cluster      TRES Name               Allocated                  Down         PLND Down                     Idle             Planned                 Reported 
--------- -------------- ----------------------- --------------------- ----------------- ------------------------ ------------------- ------------------------ 
       cs            cpu          701045(13.53%)          20377(0.39%)          0(0.00%)          3920364(75.66%)      539909(10.42%)          5181696(85.65%) 
       cs            mem      6583051083(16.26%)       49964695(0.12%)          0(0.00%)      33851329000(83.62%)            0(0.00%)      40484344778(85.62%) 
       cs       gres/gpu           91948(38.56%)            105(0.04%)          0(0.00%)           146411(61.40%)            0(0.00%)           238464(85.65%) 

* Total Cluster Resources Avaialble by Partition
 (Note, TRES is short for Trackable RESources)
PartitionName=cpu
   TRES=cpu=1680,mem=17094000M,node=44
PartitionName=gpu
   TRES=cpu=2226,mem=19710000M,node=37,gres/gpu=159
PartitionName=nolim
   TRES=cpu=168,mem=480000M,node=4
PartitionName=gnolim
   TRES=cpu=424,mem=2008000M,node=16,gres/gpu=48

SLURM Usage by Partition

PartitionName	total_jobs	cputime(HH:MM:SS)	completed	cancelled	failed	timeout	out_of_memory	node_fail
gpu	271712	494663:37:58	17683	2243	251602	119	56	9
cpu	18603	134362:54:13	15959	906	1595	100	43	0
gnolim	4578	53229:20:02	2000	1545	1025	3	5	0
nolim	58	542:46:16	42	11	4	0	1	0

SLURM Usage by Advisor Group

slurm-cs-undefined, users that have CS accounts but are not CS students
slurm-cs-unassigned, users that are CS students but do not have a listed CS advisor

GroupName	total_jobs	cputime(HH:MM:SS)	cpu	gpu	nolim	gnolim	completed	cancelled	failed	timeout	out_of_memory	node_fail
slurm-cs-zezhou-cheng	602	147423:48:14	0	602	0	0	489	63	45	5	0	0
slurm-cs-yu-meng	89	105163:01:46	0	89	0	0	32	20	17	14	2	4
slurm-cs-tianhao-wang	397	81871:57:14	7	309	0	81	92	133	155	12	4	1
slurm-cs-chen-yu-wei	8030	74281:29:30	8030	0	0	0	6665	117	1186	62	0	0
slurm-cs-lu-feng	7553	70240:59:18	0	4591	0	2962	3395	1857	2300	1	0	0
slurm-cs-hongning-wang	46	36804:04:14	0	46	0	0	11	17	8	8	0	2
slurm-cs-undefined	23694	30500:34:42	6820	15480	4	1390	17319	877	5471	7	18	2
slurm-cs-haiying-shen	4005	24763:29:54	1158	2847	0	0	3916	88	0	1	0	0
slurm-cs-shangtong-zhang	53	23005:50:04	28	25	0	0	9	8	23	11	2	0
slurm-cs-kevin-skadron	3085	18883:31:24	653	2396	34	2	2292	166	595	29	3	0
slurm-cs-yen-ling-kuo	181	18506:51:52	4	177	0	0	42	34	89	11	5	0
slurm-cs-yangfeng-ji	244455	9347:59:36	304	244151	0	0	45	625	243766	18	1	0
slurm-cs-adwait-jog	660	8203:54:48	459	156	10	35	211	398	25	5	21	0
slurm-cs-daniel-graham	330	7570:39:52	81	249	0	0	161	55	95	17	2	0
slurm-cs-farzad-farnoud	53	7569:43:22	4	49	0	0	10	22	13	0	8	0
slurm-cs-madhur-behl	203	4838:06:34	102	79	10	12	95	31	56	7	14	0
slurm-cs-ferdinando-fioretto	520	4636:41:55	455	65	0	0	295	153	58	9	5	0
slurm-cs-unassigned	98	3811:50:44	19	79	0	0	28	4	62	1	3	0
slurm-cs-sebastian-elbaum	730	3571:38:42	465	176	0	89	542	20	158	2	8	0
slurm-cs-ashish-venkat	12	858:57:12	12	0	0	0	6	0	5	0	1	0
slurm-cs-xiaozhu-lin	38	731:51:56	0	38	0	0	17	2	16	1	2	0
slurm-cs-geoffrey-fox	25	192:05:16	0	25	0	0	4	4	13	1	3	0
slurm-cs-mary-lou-soffa	58	12:07:58	0	58	0	0	0	11	44	0	3	0
slurm-cs-miaomiao-zhang	27	07:16:10	2	25	0	0	8	0	19	0	0	0
slurm-cs-rich-nguyen	7	00:06:12	0	0	0	7	0	0	7	0	0	0

SLURM Usage by NodeName

Nodename	total_jobs	cputime(HH:MM:SS)	completed	cancelled	failed	timeout	out_of_memory	node_fail
serval01	162	143394:59:18	50	39	41	24	2	6
cheetah06	591	141379:11:58	501	58	30	2	0	0
cheetah04	1199	30640:25:22	210	29	946	11	1	2
jaguar03	10340	29549:25:44	833	127	9365	8	7	0
cortado01	3266	26231:00:22	2764	55	426	21	0	0
cortado08	1880	26071:13:06	1337	30	489	23	1	0
bigcat01	2984	21021:07:08	2385	256	318	17	8	0
jaguar01	136	20935:52:48	36	37	53	5	5	0
puma01	389	19139:44:38	366	15	1	5	2	0
cheetah05	2021	13363:03:38	1131	80	796	11	3	0
jaguar04	87	13286:10:12	39	25	17	6	0	0
lotus	6242	11847:16:16	622	78	5532	4	5	1
sds01	8036	10083:33:56	693	99	7242	2	0	0
sds02	13222	7994:50:10	972	157	12086	5	2	0
hydro	1384	7304:32:18	1304	38	24	12	6	0
lynx08	1980	6912:38:14	1791	54	113	12	10	0
puma02	572	6600:28:38	189	42	331	4	6	0
jaguar02	5969	5669:31:54	1238	149	4576	6	0	0
ai03	240	5311:43:46	96	98	46	0	0	0
ai02	245	5264:04:42	97	104	44	0	0	0
ai04	247	4888:29:10	97	109	41	0	0	0
cheetah02	4363	4778:41:52	486	63	3812	0	2	0
ai01	245	4630:04:54	110	83	50	1	1	0
jaguar05	12026	4585:25:28	731	136	11145	9	5	0
ai05	342	4497:42:24	162	117	62	1	0	0
ai08	327	4335:50:04	155	131	41	0	0	0
cheetah01	1228	4116:42:06	889	78	246	11	4	0
ai07	353	3927:00:22	177	119	57	0	0	0
ai06	315	3719:44:14	147	108	59	1	0	0
panther01	501	3628:46:12	416	59	20	1	5	0
ai09	478	3484:13:02	189	122	167	0	0	0
adriatic01	11329	3435:58:44	822	80	10426	1	0	0
adriatic02	10688	3425:44:48	821	65	9802	0	0	0
ai10	454	3284:08:00	175	119	160	0	0	0
adriatic03	10093	2941:33:52	813	61	9219	0	0	0
adriatic04	9852	2785:39:04	727	56	9069	0	0	0
jaguar06	1829	2776:21:28	488	39	1300	2	0	0
cheetah03	4032	2677:59:54	488	38	3505	1	0	0
adriatic06	9676	2630:12:18	733	53	8888	2	0	0
adriatic05	10058	2509:33:28	747	51	9259	0	1	0
jinx01	329	2481:50:24	149	110	70	0	0	0
affogato13	11620	2445:23:00	378	67	11173	1	1	0
affogato11	11947	2424:31:56	372	75	11497	0	3	0
jinx02	304	2418:09:22	143	107	53	0	1	0
affogato15	11837	2313:32:56	365	69	11402	1	0	0
affogato14	11663	2310:45:58	361	68	11233	0	1	0
lynx09	1599	2305:07:06	1511	50	35	3	0	0
lynx02	6594	1997:41:50	173	20	6400	0	1	0
lynx01	8011	1805:32:44	197	22	7791	1	0	0
slurm3	167	1566:58:26	141	22	4	0	0	0
lynx03	7076	1516:35:38	153	23	6898	0	2	0
lynx05	5651	1481:18:18	155	21	5474	1	0	0
titanx03	175	1436:45:10	81	57	36	0	1	0
lynx11	15425	1348:41:52	239	50	15136	0	0	0
cortado03	145	1343:16:50	112	26	7	0	0	0
lynx12	16252	1325:57:02	229	55	15968	0	0	0
cortado09	708	1323:58:24	664	36	8	0	0	0
cortado02	157	1322:20:52	122	27	8	0	0	0
lynx10	11602	1301:57:46	188	50	11362	1	1	0
lynx04	6929	1299:43:52	193	23	6711	0	2	0
cortado04	144	1288:37:12	115	22	6	1	0	0
titanx02	181	1252:41:14	66	58	55	0	2	0
slurm4	147	1197:42:00	126	19	1	0	1	0
titanx04	156	1189:52:30	76	49	31	0	0	0
slurm1	610	1117:00:39	543	47	16	0	4	0
titanx05	169	1107:00:44	80	50	39	0	0	0
cortado05	125	1087:11:24	101	17	6	1	0	0
affogato04	75	1086:21:44	60	9	6	0	0	0
slurm2	98	991:08:34	83	11	2	0	2	0
cortado10	305	924:17:22	273	26	5	0	1	0
lynx06	6742	866:23:30	216	26	6499	0	1	0
lynx07	6554	825:41:12	207	19	6327	0	1	0
affogato02	186	791:53:04	178	4	4	0	0	0
cortado06	82	769:09:24	57	15	8	2	0	0
slurm5	68	573:33:58	57	10	0	0	1	0
affogato01	252	514:04:40	232	12	7	1	0	0
cortado07	86	474:11:56	60	11	14	1	0	0
struct08	230	426:41:54	221	9	0	0	0	0
struct09	228	413:13:34	222	6	0	0	0	0
struct03	21	399:53:50	20	0	0	0	1	0
struct10	223	385:05:26	217	6	0	0	0	0
affogato05	43	374:30:40	37	3	3	0	0	0
affogato10	49	335:18:38	42	3	3	0	1	0
affogato06	44	316:26:16	34	2	8	0	0	0
struct02	26	313:31:50	26	0	0	0	0	0
affogato07	50	307:51:42	42	2	6	0	0	0
struct01	25	307:06:34	25	0	0	0	0	0
struct07	22	296:32:06	22	0	0	0	0	0
struct04	23	294:21:34	23	0	0	0	0	0
struct05	24	293:45:06	24	0	0	0	0	0
struct06	22	291:48:36	22	0	0	0	0	0
affogato08	42	283:43:00	37	2	3	0	0	0
affogato09	41	274:39:50	36	2	3	0	0	0
heartpiece	17	264:22:56	6	9	1	0	1	0
doppio01	7	199:47:18	7	0	0	0	0	0
affogato03	114	62:28:04	111	0	3	0	0	0
doppio02	8	33:02:00	7	1	0	0	0	0
epona	9	03:37:36	9	0	0	0	0	0
bigcat04	0	00:00:00	0	0	0	0	0	0
bigcat05	0	00:00:00	0	0	0	0	0	0
bigcat06	0	00:00:00	0	0	0	0	0	0

Table of Contents

CS SLURM Cluster Report - 8weeks

SLURM Scheduler System Output

SLURM Usage by Partition

SLURM Usage by Advisor Group

SLURM Usage by NodeName