{2025.06}[2025b] GROMACS 2025.4 with CUDA-12.9.1 by bedroge · Pull Request #1482 · EESSI/software-layer

bedroge · 2026-04-22T12:11:23Z

Requires:

9 out of 87 required modules missing:

* Catch2/2.13.10-GCCcore-14.3.0 (Catch2-2.13.10-GCCcore-14.3.0.eb)
* gfbf/2025b (gfbf-2025b.eb)
* hypothesis/6.136.6-GCCcore-14.3.0 (hypothesis-6.136.6-GCCcore-14.3.0.eb)
* spin/0.14-GCCcore-14.3.0 (spin-0.14-GCCcore-14.3.0.eb)
* pybind11/3.0.0-GCC-14.3.0 (pybind11-3.0.0-GCC-14.3.0.eb)
* SciPy-bundle/2025.07-gfbf-2025b (SciPy-bundle-2025.07-gfbf-2025b.eb)
* networkx/3.5-gfbf-2025b (networkx-3.5-gfbf-2025b.eb)
* mpi4py/4.1.0-gompi-2025b (mpi4py-4.1.0-gompi-2025b.eb)
* GROMACS/2025.3-foss-2025b-CUDA-12.9.1 (GROMACS-2025.3-foss-2025b-CUDA-12.9.1.eb)

bedroge · 2026-04-23T08:55:49Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-rug for:arch=x86_64/amd/zen5,accel=nvidia/cc120

eessi-bot-rug · 2026-04-23T08:55:59Z

New job on instance eessi-bot-rug for repository eessi.io-2025.06-software
Building on: amd-zen5 and accelerator nvidia/cc120
Building for: x86_64/amd/zen5 and accelerator nvidia/cc120
Job dir: /scratch/hb-eessibot/SHARED/jobs/2026.04/pr_1482/28614014

date	job status	comment
Apr 23 08:55:57 UTC 2026	submitted	job id `28614014` awaits release by job manager
Apr 23 08:56:57 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 23 09:19:02 UTC 2026	running	job `28614014` is running
Apr 23 09:49:33 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-28614014.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ❌ found message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen5-accel-nvidia-cc120-17769376410.tar.zst` size: 0 MiB (22 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120 no other files in tarball
Apr 23 09:49:33 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node %device_type=gpu /b88eedf0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node %device_type=gpu /8c8bf48b @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node %device_type=gpu /6d7a17a9 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node %device_type=gpu /e5a16ba0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node %device_type=gpu /634d019c @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node %device_type=gpu /e9b09ad8 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node /b1ea69c1 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node /a317b8da @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node /a102bba0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node /7bd54429 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node /84994f87 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node /d58e51e9 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-28614014.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2026-04-23T10:12:59Z

The build succeeded, but it fails in the CUDA sanity check:

== 2026-04-23 11:47:13,762 easyblock.py:3849 INFO CUDA sanity check detailed report:
12 files missing one or more CUDA compute capabilities:
  lib/libgromacs.so.10.0.0
  lib/libgromacs.so.10
  lib/libgromacs.so
  lib/libgromacs_mpi.so.10.0.0
  lib/libgromacs_mpi.so.10
  lib/libgromacs_mpi.so
  lib64/libgromacs.so.10.0.0
  lib64/libgromacs.so.10
  lib64/libgromacs.so
  lib64/libgromacs_mpi.so.10.0.0
  lib64/libgromacs_mpi.so.10
  lib64/libgromacs_mpi.so
12 files with device code for more CUDA Compute Capabilities than requested:
  lib/libgromacs.so.10.0.0
  lib/libgromacs.so.10
  lib/libgromacs.so
  lib/libgromacs_mpi.so.10.0.0
  lib/libgromacs_mpi.so.10
  lib/libgromacs_mpi.so
  lib64/libgromacs.so.10.0.0
  lib64/libgromacs.so.10
  lib64/libgromacs.so
  lib64/libgromacs_mpi.so.10.0.0
  lib64/libgromacs_mpi.so.10
  lib64/libgromacs_mpi.so
12 files missing PTX code for the highest configured CUDA Compute Capability:
  lib/libgromacs.so.10.0.0
  lib/libgromacs.so.10
  lib/libgromacs.so
  lib/libgromacs_mpi.so.10.0.0
  lib/libgromacs_mpi.so.10
  lib/libgromacs_mpi.so
  lib64/libgromacs.so.10.0.0
  lib64/libgromacs.so.10
  lib64/libgromacs.so
  lib64/libgromacs_mpi.so.10.0.0
  lib64/libgromacs_mpi.so.10
  lib64/libgromacs_mpi.so

I guess it may be related to the 120f that we're using, as the binaries do seem to have support for sm_120:

Fatbin elf code:
================
arch = sm_120

bedroge · 2026-04-24T07:43:24Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-rug for:arch=x86_64/amd/zen5,accel=nvidia/cc120

eessi-bot-rug · 2026-04-24T07:43:36Z

New job on instance eessi-bot-rug for repository eessi.io-2025.06-software
Building on: amd-zen5 and accelerator nvidia/cc120
Building for: x86_64/amd/zen5 and accelerator nvidia/cc120
Job dir: /scratch/hb-eessibot/SHARED/jobs/2026.04/pr_1482/28630885

date	job status	comment
Apr 24 07:43:34 UTC 2026	submitted	job id `28630885` awaits release by job manager
Apr 24 07:44:50 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 24 07:46:53 UTC 2026	running	job `28630885` is running
Apr 24 08:17:23 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-28630885.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen5-accel-nvidia-cc120-17770184860.tar.zst` size: 32 MiB (33916242 bytes) entries: 760 modules under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/modules/all `GROMACS/2025.4-foss-2025b-CUDA-12.9.1.lua` software under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/software `GROMACS/2025.4-foss-2025b-CUDA-12.9.1` reprod directories under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120/reprod `GROMACS/2025.4-foss-2025b-CUDA-12.9.1/20260424_081438UTC` other under 2025.06/software/linux/x86_64/amd/zen5/accel/nvidia/cc120 no other files in tarball
Apr 24 08:17:23 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node %device_type=gpu /b88eedf0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node %device_type=gpu /8c8bf48b @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node %device_type=gpu /6d7a17a9 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node %device_type=gpu /e5a16ba0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node %device_type=gpu /634d019c @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node %device_type=gpu /e9b09ad8 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node /b1ea69c1 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node /a317b8da @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node /a102bba0 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_node /7bd54429 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_node /84994f87 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_node /d58e51e9 @BotBuildTests:gpu_rtx_pro_6000+default [Skipping GPU test : only 1 GPU available for this test case] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-28630885.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2026-04-24T11:43:00Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/amd/zen4,accel=nvidia/cc90
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/intel/cascadelake,accel=nvidia/cc70
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-vsc-ugent for:arch=x86_64/amd/zen3,accel=nvidia/cc80
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-jsc for:arch=aarch64/nvidia/grace,accel=nvidia/cc90
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-rug for:arch=x86_64/intel/skylake_avx512,accel=nvidia/cc70

eessi-bot-surf · 2026-04-24T11:43:08Z

New job on instance eessi-bot-surf for repository eessi.io-2025.06-software
Building on: intel-icelake and accelerator nvidia/cc80
Building for: x86_64/intel/icelake and accelerator nvidia/cc80
Job dir: /projects/eessibot/eessi-bot-surf/jobs/2026.04/pr_1482/22223686

date	job status	comment
Apr 24 11:43:06 UTC 2026	submitted	job id `22223686` will be eligible to start in about 20 seconds
Apr 24 11:43:20 UTC 2026	received	job awaits launch by Slurm scheduler
Apr 24 11:45:00 UTC 2026	running	job `22223686` is running
Apr 24 13:01:50 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-22223686.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ✅ no message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-intel-icelake-accel-nvidia-cc80-17770356230.tar.zst` size: 0 MiB (22 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80 no other files in tarball
Apr 24 13:01:50 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /15d6e239 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /5471f15a @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /526cd259 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /1dc400ef @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /9715dde6 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /416eaee1 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /ed938ed4 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /8d24cea9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /73a202f1 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /946648aa @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /9eb3f1e9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /7f04eb2b @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-22223686.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

eessi-bot-surf · 2026-04-24T11:43:14Z

New job on instance eessi-bot-surf for repository eessi.io-2025.06-software
Building on: amd-zen4 and accelerator nvidia/cc90
Building for: x86_64/amd/zen4 and accelerator nvidia/cc90
Job dir: /projects/eessibot/eessi-bot-surf/jobs/2026.04/pr_1482/22223689

date	job status	comment
Apr 24 11:43:12 UTC 2026	submitted	job id `22223689` will be eligible to start in about 20 seconds
Apr 24 11:43:24 UTC 2026	received	job awaits launch by Slurm scheduler
Apr 24 11:43:47 UTC 2026	running	job `22223689` is running
Apr 25 11:44:10 UTC 2026	finished	🤷 UNKNOWN (click triangle for detailed information) Job results file `_bot_job22223689.result` does not exist in job directory, or parsing it failed. No artefacts were found/reported.
Apr 25 11:44:10 UTC 2026	test result	🤷 UNKNOWN (click triangle for detailed information) Job test file `_bot_job22223689.test` does not exist in job directory, or parsing it failed.

gpu-bot-ugent · 2026-04-24T11:43:14Z

New job on instance eessi-bot-vsc-ugent for repository eessi.io-2025.06-software
Building on: intel-cascadelake and accelerator nvidia/cc70
Building for: x86_64/intel/cascadelake and accelerator nvidia/cc70
Job dir: /scratch/gent/vo/002/gvo00211/SHARED/jobs/2026.04/pr_1482/40819786

date	job status	comment
Apr 24 11:43:12 UTC 2026	submitted	job id `40819786` awaits release by job manager
Apr 24 11:44:50 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 24 11:46:54 UTC 2026	running	job `40819786` is running
Apr 24 13:09:46 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-40819786.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-intel-cascadelake-accel-nvidia-cc70-17770361340.tar.zst` size: 31 MiB (32681382 bytes) entries: 760 modules under 2025.06/software/linux/x86_64/intel/cascadelake/accel/nvidia/cc70/modules/all `GROMACS/2025.4-foss-2025b-CUDA-12.9.1.lua` software under 2025.06/software/linux/x86_64/intel/cascadelake/accel/nvidia/cc70/software `GROMACS/2025.4-foss-2025b-CUDA-12.9.1` reprod directories under 2025.06/software/linux/x86_64/intel/cascadelake/accel/nvidia/cc70/reprod `GROMACS/2025.4-foss-2025b-CUDA-12.9.1/20260424_130833UTC` other under 2025.06/software/linux/x86_64/intel/cascadelake/accel/nvidia/cc70 no other files in tarball
Apr 24 13:09:46 UTC 2026	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-40819786.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

eessi-bot-jsc · 2026-04-24T11:43:17Z

New job on instance eessi-bot-jsc for repository eessi.io-2025.06-software
Building on: nvidia-grace and accelerator nvidia/cc90
Building for: aarch64/nvidia/grace and accelerator nvidia/cc90
Job dir: /p/project1/ceasybuilders/eessibot/jobs/2026.04/pr_1482/14684643

date	job status	comment
Apr 24 11:43:15 UTC 2026	submitted	job id `14684643` awaits release by job manager
Apr 24 11:44:07 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 24 11:45:10 UTC 2026	running	job `14684643` is running
Apr 24 12:39:58 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-14684643.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-aarch64-nvidia-grace-accel-nvidia-cc90-17770337320.tar.gz` size: 32 MiB (34349669 bytes) entries: 760 modules under 2025.06/software/linux/aarch64/nvidia/grace/accel/nvidia/cc90/modules/all `GROMACS/2025.4-foss-2025b-CUDA-12.9.1.lua` software under 2025.06/software/linux/aarch64/nvidia/grace/accel/nvidia/cc90/software `GROMACS/2025.4-foss-2025b-CUDA-12.9.1` reprod directories under 2025.06/software/linux/aarch64/nvidia/grace/accel/nvidia/cc90/reprod `GROMACS/2025.4-foss-2025b-CUDA-12.9.1/20260424_122736UTC` other under 2025.06/software/linux/aarch64/nvidia/grace/accel/nvidia/cc90 no other files in tarball
Apr 24 12:39:58 UTC 2026	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite produced failures. ReFrame Summary [ FAILED ] Ran 18/30 test case(s) from 30 check(s) (4 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-14684643.out` ❌ found message matching `ERROR:` ❌ found message matching `[\sFAILED\s].Ran . test case`

eessi-bot-rug · 2026-04-24T11:43:18Z

New job on instance eessi-bot-rug for repository eessi.io-2025.06-software
Building on: intel-skylake_avx512 and accelerator nvidia/cc70
Building for: x86_64/intel/skylake_avx512 and accelerator nvidia/cc70
Job dir: /scratch/hb-eessibot/SHARED/jobs/2026.04/pr_1482/28636437

date	job status	comment
Apr 24 11:43:16 UTC 2026	submitted	job id `28636437` awaits release by job manager
Apr 24 11:44:00 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 24 12:08:04 UTC 2026	running	job `28636437` is running
Apr 24 13:09:06 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-28636437.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-intel-skylake_avx512-accel-nvidia-cc70-17770359480.tar.zst` size: 31 MiB (32689771 bytes) entries: 760 modules under 2025.06/software/linux/x86_64/intel/skylake_avx512/accel/nvidia/cc70/modules/all `GROMACS/2025.4-foss-2025b-CUDA-12.9.1.lua` software under 2025.06/software/linux/x86_64/intel/skylake_avx512/accel/nvidia/cc70/software `GROMACS/2025.4-foss-2025b-CUDA-12.9.1` reprod directories under 2025.06/software/linux/x86_64/intel/skylake_avx512/accel/nvidia/cc70/reprod `GROMACS/2025.4-foss-2025b-CUDA-12.9.1/20260424_130523UTC` other under 2025.06/software/linux/x86_64/intel/skylake_avx512/accel/nvidia/cc70 no other files in tarball
Apr 24 13:09:06 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_2_node %device_type=gpu /495ccd0c @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_2_node %device_type=gpu /61fda20d @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_2_node %device_type=gpu /e3d4ae3b @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_2_node %device_type=gpu /ce7fe725 @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_2_node %device_type=gpu /5c339fc9 @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_2_node %device_type=gpu /b4bd1071 @BotBuildTests:gpu_v100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_2_node /c3881e1d @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_2_node /5f02f86c @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_2_node /530b49da @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_2_node /f49f730d @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_2_node /c412ac42 @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_2_node /18861056 @BotBuildTests:gpu_v100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-28636437.out` ✅ no message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

gpu-bot-ugent · 2026-04-24T11:43:20Z

New job on instance eessi-bot-vsc-ugent for repository eessi.io-2025.06-software
Building on: amd-zen3 and accelerator nvidia/cc80
Building for: x86_64/amd/zen3 and accelerator nvidia/cc80
Job dir: /scratch/gent/vo/002/gvo00211/SHARED/jobs/2026.04/pr_1482/15689215

date	job status	comment
Apr 24 11:43:18 UTC 2026	submitted	job id `15689215` awaits release by job manager
Apr 24 11:44:46 UTC 2026	released	job awaits launch by Slurm scheduler
Apr 24 11:48:58 UTC 2026	running	job `15689215` is running
Apr 24 12:55:31 UTC 2026	finished	😁 SUCCESS (click triangle for details) Details ✅ job output file `slurm-15689215.out` ✅ no message matching `FATAL:` ✅ no message matching `ERROR:` ✅ no message matching `FAILED:` ✅ no message matching `required modules missing:` ✅ found message(s) matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-amd-zen3-accel-nvidia-cc80-17770352040.tar.zst` size: 34 MiB (36599032 bytes) entries: 760 modules under 2025.06/software/linux/x86_64/amd/zen3/accel/nvidia/cc80/modules/all `GROMACS/2025.4-foss-2025b-CUDA-12.9.1.lua` software under 2025.06/software/linux/x86_64/amd/zen3/accel/nvidia/cc80/software `GROMACS/2025.4-foss-2025b-CUDA-12.9.1` reprod directories under 2025.06/software/linux/x86_64/amd/zen3/accel/nvidia/cc80/reprod `GROMACS/2025.4-foss-2025b-CUDA-12.9.1/20260424_125258UTC` other under 2025.06/software/linux/x86_64/amd/zen3/accel/nvidia/cc80 no other files in tarball
Apr 24 12:55:31 UTC 2026	test result	😢 FAILURE (click triangle for details) Reason EESSI test suite was not run, test step itself failed to execute. Details ✅ job output file `slurm-15689215.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2026-04-24T14:13:59Z

@casparvl The icelake cc80 build with the Surf bot failed because of:

[1777032154.196060] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)
[1777032154.196107] [gcn12:54944:0]           mpool.c:269  UCX  ERROR Failed to allocate memory pool (name=rc_recv_desc) chunk: Input/output error
[1777032154.196266] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)

Have you encountered this before?

boegel · 2026-04-24T15:28:47Z

@casparvl The icelake cc80 build with the Surf bot failed because of:

[1777032154.196060] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)
[1777032154.196107] [gcn12:54944:0]           mpool.c:269  UCX  ERROR Failed to allocate memory pool (name=rc_recv_desc) chunk: Input/output error
[1777032154.196266] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)

Have you encountered this before?

ulimit -l being set to 8MB causing trouble doesn't seem too crazy to me...

Maybe we just need to add ulimit -l unlimited to bot build (job) script?

bedroge · 2026-05-01T11:30:50Z

@casparvl The icelake cc80 build with the Surf bot failed because of:

[1777032154.196060] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)
[1777032154.196107] [gcn12:54944:0]           mpool.c:269  UCX  ERROR Failed to allocate memory pool (name=rc_recv_desc) chunk: Input/output error
[1777032154.196266] [gcn12:54944:0]           ib_md.c:287  UCX  ERROR ibv_reg_mr(address=0x7febfda00000, length=37748736, access=0xf) failed: Cannot allocate memory : Please set max lo
cked memory (ulimit -l) to 'unlimited' (current: 8192 kbytes)

Have you encountered this before?

ulimit -l being set to 8MB causing trouble doesn't seem too crazy to me...

Maybe we just need to add ulimit -l unlimited to bot build (job) script?

I tried various things with an interactive job on Snellius, but ulimit -l always printed unlimited. Looking at the logs and the job details again, I found that the job itself is also reported as OUT_OF_MEMORY, so maybe that's the real issue here. sacct shows that the job requested 120GB though, which should be more than enough for GROMACS? Also, MaxRSS is 15916602K, i.e. ~16GB, so I don't understand this... From what I can see in the logs, it doesn't seem to use /dev/shm either.

edit: the zen4 job also ran out of memory according to Slurm, but somehow kept running and then timed out after a day.

@casparvl do you have any idea what's going on?

casparvl · 2026-05-04T10:07:49Z

The only thing I can think of: these nodes don't have local disks, so /tmp is essentially also in memory. Could it be that we are writing a lot there? I mean, it'd have to be a whole lot.

casparvl · 2026-05-04T10:18:28Z

edit: the zen4 job also ran out of memory according to Slurm, but somehow kept running and then timed out after a day.

I've seen this happen before. If you have, say, 3 processes running, OOM killer might kill one, leave 2 stray processes that just wait for the other one to do something. And that then runs indefinitely. SLURM doesn't end the job, since you still have running processes.

bedroge · 2026-05-08T13:19:50Z

I've done an interactive build on an A100 node on Snellius with my personal account and on top of EESSI (without a container), that worked fine:

== Build succeeded for 1 out of 1 (total: 30 mins 20 secs)
== Summary:
   * [SUCCESS] GROMACS/2025.4-foss-2025b-CUDA-12.9.1

No memory issues, and the max memory usage was like ~4GB. I'll do another one with the container.

bedroge · 2026-05-08T14:27:34Z

Let me just try this again as well:

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80

eessi-bot-surf · 2026-05-08T14:27:44Z

New job on instance eessi-bot-surf for repository eessi.io-2025.06-software
Building on: intel-icelake and accelerator nvidia/cc80
Building for: x86_64/intel/icelake and accelerator nvidia/cc80
Job dir: /projects/eessibot/eessi-bot-surf/jobs/2026.05/pr_1482/22588119

date	job status	comment
May 08 14:27:42 UTC 2026	submitted	job id `22588119` will be eligible to start in about 20 seconds

casparvl · 2026-05-13T15:18:30Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80

casparvl · 2026-05-13T15:19:29Z

Hmmm, something is wrong. We changed some things in the bot config in our config management system, but for some reason it concludes it shouldn't submit a job based on the above commands. Will dig into why...

ocaisa · 2026-05-13T15:33:10Z

@bedroge I don't know if you are looking at adding the hwloc support to this, but just to mention you would also need a dependency on https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs/h/hwloc-CUDA in that case

bedroge · 2026-06-02T12:22:34Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80

eessi-bot-surf · 2026-06-02T12:22:42Z

New job on instance eessi-bot-surf for repository eessi.io-2025.06-software
Building on: intel-icelake and accelerator nvidia/cc80
Building for: x86_64/intel/icelake and accelerator nvidia/cc80
Job dir: /projects/eessibot/eessi-bot-surf/jobs/2026.06/pr_1482/23379886

date	job status	comment
Jun 02 12:22:40 UTC 2026	submitted	job id `23379886` will be eligible to start in about 20 seconds
Jun 02 12:22:53 UTC 2026	received	job awaits launch by Slurm scheduler
Jun 02 12:24:58 UTC 2026	running	job `23379886` is running
Jun 02 13:44:02 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-23379886.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ✅ no message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-intel-icelake-accel-nvidia-cc80-17804076270.tar.zst` size: 0 MiB (22 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80 no other files in tarball
Jun 02 13:44:02 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /15d6e239 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /5471f15a @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /526cd259 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /1dc400ef @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /9715dde6 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /416eaee1 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /ed938ed4 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /8d24cea9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /73a202f1 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /946648aa @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /9eb3f1e9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /7f04eb2b @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-23379886.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

bedroge · 2026-06-02T13:53:42Z

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-surf for:arch=x86_64/intel/icelake,accel=nvidia/cc80

eessi-bot-surf · 2026-06-02T13:53:51Z

New job on instance eessi-bot-surf for repository eessi.io-2025.06-software
Building on: intel-icelake and accelerator nvidia/cc80
Building for: x86_64/intel/icelake and accelerator nvidia/cc80
Job dir: /projects/eessibot/eessi-bot-surf/jobs/2026.06/pr_1482/23382530

date	job status	comment
Jun 02 13:53:48 UTC 2026	submitted	job id `23382530` will be eligible to start in about 20 seconds
Jun 02 13:54:01 UTC 2026	received	job awaits launch by Slurm scheduler
Jun 02 13:55:16 UTC 2026	running	job `23382530` is running
Jun 02 15:16:46 UTC 2026	finished	😢 FAILURE (click triangle for details) Details ✅ job output file `slurm-23382530.out` ✅ no message matching `FATAL:` ❌ found message matching `ERROR:` ✅ no message matching `FAILED:` ❌ found message matching `required modules missing:` ❌ no message matching `No missing installations` ✅ found message matching `.tar.* created!` Artefacts `eessi-2025.06-software-linux-x86_64-intel-icelake-accel-nvidia-cc80-17804132810.tar.zst` size: 0 MiB (22 bytes) entries: 0 modules under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/modules/all no module files in tarball software under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/software no software packages in tarball reprod directories under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80/reprod no reprod directories in tarball other under 2025.06/software/linux/x86_64/intel/icelake/accel/nvidia/cc80 no other files in tarball
Jun 02 15:16:46 UTC 2026	test result	😁 SUCCESS (click triangle for details) ReFrame Summary [ SKIP ] ( 1/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /15d6e239 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 2/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /5471f15a @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 3/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_allreduce %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /526cd259 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 4/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node %device_type=gpu /1dc400ef @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 5/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node %device_type=gpu /9715dde6 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 6/12) EESSI_OSU_coll %benchmark_info=mpi.collective.osu_alltoall %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node %device_type=gpu /416eaee1 @BotBuildTests:gpu_a100+default [Skipping GPU test : only 1 GPU available for this test case] [ SKIP ] ( 7/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /ed938ed4 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 8/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /8d24cea9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] ( 9/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_latency %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /73a202f1 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (10/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5.1-gompi-2025b-CUDA-12.9.1 %scale=1_4_node /946648aa @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (11/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2025a-CUDA-12.8.0 %scale=1_4_node /9eb3f1e9 @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ SKIP ] (12/12) EESSI_OSU_pt2pt_GPU %benchmark_info=mpi.pt2pt.osu_bw %module_name=OSU-Micro-Benchmarks/7.5-gompi-2024a-CUDA-12.6.0 %scale=1_4_node /7f04eb2b @BotBuildTests:gpu_a100+default [Skipping test : 1 GPU(s) available for this test case, need exactly 2] [ PASSED ] Ran 0/12 test case(s) from 12 check(s) (0 failure(s), 12 skipped, 0 aborted) Details ✅ job output file `slurm-23382530.out` ❌ found message matching `ERROR:` ✅ no message matching `[\sFAILED\s].Ran . test case`

casparvl · 2026-06-02T15:21:24Z

Created a small script in an attempt to figure out the memory consumption:

$ cat query_memory_stats.sh
#!/bin/bash

cgroup_mem_stats() {
    awk '{printf "%s %.2f GB\n", $1, $2/1024/1024/1024}' /sys/fs/cgroup/$(</proc/self/cpuset)/../../../memory.stat
}

cgroup_max_mem() {
    cat /sys/fs/cgroup/$(</proc/self/cpuset)/../../../memory.peak | numfmt --to=iec
}

while true; do
    echo "$(date): cgroup mem: $(cgroup_max_mem), du -sh /tmp: $(du -sh /tmp)"
    cgroup_mem_stats
    sleep 1
done

A snapshot of that output, somewhat very close to the peak memory usage (right before running OOM):

Tue Jun  2 05:13:56 PM CEST 2026: cgroup mem: 120G, du -sh /tmp: 17G    /tmp
anon 15.98 GB
file 103.51 GB
kernel 0.38 GB
kernel_stack 0.00 GB
pagetables 0.04 GB
sec_pagetables 0.00 GB
percpu 0.00 GB
sock 0.00 GB
vmalloc 0.00 GB
shmem 102.80 GB
zswap 0.00 GB
zswapped 0.00 GB
file_mapped 0.06 GB
file_dirty 0.00 GB
file_writeback 0.00 GB
swapcached 0.00 GB
anon_thp 7.47 GB
file_thp 0.00 GB
shmem_thp 0.00 GB
inactive_anon 0.00 GB
active_anon 118.78 GB
inactive_file 0.44 GB
active_file 0.27 GB
unevictable 0.00 GB
slab_reclaimable 0.29 GB
slab_unreclaimable 0.04 GB
slab 0.33 GB
workingset_refault_anon 0.00 GB
workingset_refault_file 0.00 GB
workingset_activate_anon 0.00 GB
workingset_activate_file 0.00 GB
workingset_restore_anon 0.00 GB
workingset_restore_file 0.00 GB
workingset_nodereclaim 0.00 GB
pgscan 0.00 GB
pgsteal 0.00 GB
pgscan_kswapd 0.00 GB
pgscan_direct 0.00 GB
pgscan_khugepaged 0.00 GB
pgsteal_kswapd 0.00 GB
pgsteal_direct 0.00 GB
pgsteal_khugepaged 0.00 GB
pgfault 0.60 GB
pgmajfault 0.01 GB
pgrefill 0.00 GB
pgactivate 0.00 GB
pgdeactivate 0.00 GB
pglazyfree 0.00 GB
pglazyfreed 0.00 GB
zswpin 0.00 GB
zswpout 0.00 GB
zswpwb 0.00 GB
thp_fault_alloc 0.00 GB
thp_collapse_alloc 0.00 GB
thp_swpout 0.00 GB

Looking at this file and the timestamps, the memory consumption increases gradually. It's almost like some process is allocated MPI buffers in shmem, then getting the failure that we see in the log file (that @bedroge mentioned here #1482 (comment) ), and then just trying again and again and again, until the memory fills up.

I'm assuming that if we solve the underlying issue for UCX, this will fix the OOM as well. The tricky part is: when @bedroge manually starts the container, the ulimit -l is already set to unlimited, so this error seems to be a bit misleading.

It might be related to openucx/ucx#6264 which my colleague @satishskamath has also replied to. While that was for way older versions of UCX, and while I think that natively on our system the issue was resolved, the error does seem to come from the same ibv_reg_mr component. We could try to see what happens if we export the environment variables that are suggested as a workaround...

casparvl · 2026-06-02T15:27:14Z

bot:show_config

eessi-bot-aws · 2026-06-02T15:27:20Z

Instance eessi-bot-mc-aws is configured to build on:

Node type x86-64-generic:
- OS: linux
- CPU architecture: x86_64/generic
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-haswell:
- OS: linux
- CPU architecture: x86_64/intel/haswell
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-sapphirerapids:
- OS: linux
- CPU architecture: x86_64/intel/sapphirerapids
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-skylake:
- OS: linux
- CPU architecture: x86_64/intel/skylake_avx512
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-cascadelake:
- OS: linux
- CPU architecture: x86_64/intel/cascadelake
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-icelake:
- OS: linux
- CPU architecture: x86_64/intel/icelake
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-zen2:
- OS: linux
- CPU architecture: x86_64/amd/zen2
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-zen3:
- OS: linux
- CPU architecture: x86_64/amd/zen3
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-zen4:
- OS: linux
- CPU architecture: x86_64/amd/zen4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-generic:
- OS: linux
- CPU architecture: aarch64/generic
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-neoverse_n1:
- OS: linux
- CPU architecture: aarch64/neoverse_n1
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-neoverse_v1:
- OS: linux
- CPU architecture: aarch64/neoverse_v1
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-graviton4:
- OS: linux
- CPU architecture: aarch64/aws/graviton4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']

eessi-bot-surf · 2026-06-02T15:27:21Z

Instance eessi-bot-surf is configured to build on:

Node type cpu_zen2:
- OS: linux
- CPU architecture: x86_64/amd/zen2
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type cpu_zen4:
- OS: linux
- CPU architecture: x86_64/amd/zen4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type gpu_a100:
- OS: linux
- CPU architecture: x86_64/intel/icelake
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
- Accelerators: nvidia/cc80
Node type gpu_h100:
- OS: linux
- CPU architecture: x86_64/amd/zen4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
- Accelerators: nvidia/cc90

gpu-bot-ugent · 2026-06-02T15:27:21Z

Instance eessi-bot-vsc-ugent is configured to build on:

Node type zen5:
- OS: linux
- CPU architecture: x86_64/amd/zen5
- Repositories: ['eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type gpu_a100:
- OS: linux
- CPU architecture: x86_64/amd/zen3
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
- Accelerators: nvidia/cc80
Node type gpu_v100:
- OS: linux
- CPU architecture: x86_64/intel/cascadelake
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
- Accelerators: nvidia/cc70

eessi-bot-deucalion · 2026-06-02T15:27:21Z

Instance eessi-bot-deucalion is configured to build on:

Node type a64fx:
- OS: linux
- CPU architecture: aarch64/a64fx
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']

eessi-bot-jsc · 2026-06-02T15:27:21Z

Instance eessi-bot-jsc is configured to build on:

Node type aarch64-nvidia-grace:
- OS: linux
- CPU architecture: aarch64/nvidia/grace
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-nvidia-gh200:
- OS: linux
- CPU architecture: aarch64/nvidia/grace
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
- Accelerators: nvidia/cc90

eessi-bot-aws-eu-south · 2026-06-02T15:27:22Z

Instance eessi-bot-aws-eu-south is configured to build on:

Node type x86-64-zen4:
- OS: linux
- CPU architecture: x86_64/amd/zen4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type x86-64-zen5:
- OS: linux
- CPU architecture: x86_64/amd/zen5
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']
Node type aarch64-graviton4:
- OS: linux
- CPU architecture: aarch64/aws/graviton4
- Repositories: ['eessi.io-2023.06-compat', 'eessi.io-2023.06-software', 'eessi.io-2025.06-compat', 'eessi.io-2025.06-software']

casparvl · 2026-06-02T15:30:22Z

Seems my bot can't really reply, but is picking up stuff from the PR. Let's see if I can trigger a build, that'll allow us to easily try different environment variables via the site config script of the bot to see if we can fix this.

bot: build repo:eessi.io-2025.06-software instance:eessi-bot-casparvl for:arch=x86_64/intel/icelake,accel=nvidia/cc80

add GROMACS-2025.3-foss-2025b-CUDA-12.9.1.eb

8840a9b

bedroge added accel:nvidia 2025.06-software.eessi.io 2025.06 version of software.eessi.io labels Apr 22, 2026

bedroge mentioned this pull request Apr 24, 2026

{bio}[foss/2025b] GROMACS v2025.3 w/ CUDA 12.9.1 easybuilders/easybuild-easyconfigs#25215

Merged

bump GROMACS 2025.3 to 2025.4

4de00fe

bedroge changed the title ~~{2025.06}[2025b] GROMACS 2025.3 with CUDA-12.9.1~~ {2025.06}[2025b] GROMACS 2025.4 with CUDA-12.9.1 Apr 24, 2026

bedroge mentioned this pull request May 22, 2026

{2025.06}[foss-2024a] Siesta 5.4.2 CUDA 12.6.0 #1506

Open

Conversation

bedroge commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Apr 23, 2026

Uh oh!

eessi-bot-rug Bot commented Apr 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Apr 23, 2026

Uh oh!

bedroge commented Apr 24, 2026

Uh oh!

eessi-bot-rug Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Apr 24, 2026

Uh oh!

eessi-bot-surf Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-surf Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpu-bot-ugent Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-jsc Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eessi-bot-rug Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gpu-bot-ugent Bot commented Apr 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Apr 24, 2026

Uh oh!

boegel commented Apr 24, 2026

Uh oh!

bedroge commented May 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented May 4, 2026

Uh oh!

casparvl commented May 4, 2026

Uh oh!

bedroge commented May 8, 2026

Uh oh!

bedroge commented May 8, 2026

Uh oh!

eessi-bot-surf Bot commented May 8, 2026

Uh oh!

casparvl commented May 13, 2026

Uh oh!

casparvl commented May 13, 2026

Uh oh!

ocaisa commented May 13, 2026

Uh oh!

bedroge commented Jun 2, 2026

Uh oh!

eessi-bot-surf Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedroge commented Jun 2, 2026

Uh oh!

eessi-bot-surf Bot commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

casparvl commented Jun 2, 2026

Uh oh!

casparvl commented Jun 2, 2026

Uh oh!

eessi-bot-aws Bot commented Jun 2, 2026

Uh oh!

eessi-bot-surf Bot commented Jun 2, 2026

Uh oh!

gpu-bot-ugent Bot commented Jun 2, 2026

Uh oh!

eessi-bot-deucalion Bot commented Jun 2, 2026

bedroge commented Apr 22, 2026 •

edited

Loading

eessi-bot-rug Bot commented Apr 23, 2026 •

edited

Loading

eessi-bot-rug Bot commented Apr 24, 2026 •

edited

Loading

eessi-bot-surf Bot commented Apr 24, 2026 •

edited

Loading

eessi-bot-surf Bot commented Apr 24, 2026 •

edited

Loading

gpu-bot-ugent Bot commented Apr 24, 2026 •

edited

Loading

eessi-bot-jsc Bot commented Apr 24, 2026 •

edited

Loading

eessi-bot-rug Bot commented Apr 24, 2026 •

edited

Loading

gpu-bot-ugent Bot commented Apr 24, 2026 •

edited

Loading

bedroge commented May 1, 2026 •

edited

Loading

eessi-bot-surf Bot commented Jun 2, 2026 •

edited

Loading

eessi-bot-surf Bot commented Jun 2, 2026 •

edited

Loading