> For the complete documentation index, see [llms.txt](https://wiki.solids.group/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://wiki.solids.group/alamo/hpc.md).

# Alamo on HPC

This page provides reference information for compiling and running Alamo on high-performance computing (HPC) clusters.

> These instructions are for reference only and may not always work. The Alamo developers do not manage the software on these clusters, and configurations may change over time. If you encounter outdated instructions, [open an issue on GitHub](https://github.com/solidsgroup/Alamo/issues).

## Reference Scripts for Nova

> For more information on Nova or HPC in general, Iowa State University provides an [online HPC guide](https://www.hpc.iastate.edu/guides).

The following environment modules are required to complete the listed tasks.

| Task                   | Required Module(s) |
| ---------------------- | ------------------ |
| Configuring with gcc   | `openmpi gcc`      |
| Configuring with clang | `openmpi llvm`     |
| Compiling with gcc     | `openmpi gcc`      |
| Compiling with clang   | `openmpi llvm`     |
| Running Alamo          | `openmpi gcc`      |

The scripts below automatically handle module management.

The configuration and compilation scripts below can be run line-by-line or in a Bash script. If you use a script, make the file executable:

```bash
chmod +x /path/to/file
```

> As of April 2025, the git installation on Nova is quite old and can occasionally cause fatal errors when configuring. If you get an error about SSL when the configure script tries to check out AMReX, load the git module with `module load git`.

### Clang Configure and Compile Script

```bash
#!/usr/bin/env bash
module purge
module load openmpi llvm
./configure --get-eigen --comp clang++
srun --nodes=1 --cpus-per-task=16 --mem-per-cpu=1G --time=10:00 make -j16
```

### GCC Configure and Compile Script

```bash
#!/usr/bin/env bash
module purge
module load openmpi gcc
./configure --get-eigen
srun --nodes=1 --cpus-per-task=16 --mem-per-cpu=1G --time=10:00 make -j16
```

### Alamo Simulation Slurm Job Script

```bash
#!/usr/bin/env bash
#SBATCH --time=24:00:00
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=36
#SBATCH --mem-per-cpu=1000
#SBATCH --job-name="alamo"
#SBATCH --output="%x-%j-log.txt"
#SBATCH --mail-user=your_email@iastate.edu
#SBATCH --mail-type=BEGIN
#SBATCH --mail-type=END
#SBATCH --mail-type=FAIL

module purge
module load openmpi
srun --mpi=pmix ./path/to/alamo/executable /path/to/input/file
```

The script starts a parallel job on Nova. Modify these parameters as needed:

* `--time`: the wall clock time limit, or maximum job duration
* `--nodes`: number of nodes requested
* `--ntasks-per-node`: number of tasks per node
* `--cpus-per-task`: number of cores per task
* `--mem-per-cpu`: memory allocated per core, in MB
* `--job-name`: job name displayed by `squeue`
* `--output`: log filename; see the [Slurm documentation](https://slurm.schedmd.com/sbatch.html#SECTION_FILENAME-PATTERN) for filename pattern specifications
* `--mail-user`: email for notifications; remove this if not needed
* Executable path: for example, `./bin/alamo-2d-clang++`
* Input file path: the input file for Alamo

> Iowa State University provides a [Slurm job script generator for Nova](https://www.hpc.iastate.edu/guides/nova/slurm-script-generator-for-nova), which can help generate job scripts.

Nova uses a fair-share scheduling system to prioritize job execution based on requested resources and past usage. To reduce wait times, request only necessary resources and set reasonable time limits.

Slurm automatically determines the number of cores based on the `--nodes` and `--ntasks-per-node` values. Refer to the [Nova hardware guide](https://www.hpc.iastate.edu/guides/nova) for appropriate values.

Once modifications are made, submit the job with [`sbatch`](https://slurm.schedmd.com/sbatch.html):

```bash
sbatch /path/to/job_script.sh
```

## Managing Dependencies on an HPC Cluster

Compiling and running Alamo on an HPC cluster differs slightly from doing so on a local machine, primarily because of dependency management. HPC clusters often provide multiple versions of software dependencies for users. To manage these dependencies efficiently, HPC clusters commonly use [Environment Modules](https://modules.sourceforge.net/), or modules, which help users load and unload software as needed. Most HPC clusters provide the required modules for compiling Alamo.

To load an environment module:

```bash
module load module_1 module_2 ...
```

To unload an environment module:

```bash
module unload module_1 module_2 ...
```

To unload all modules:

```bash
module purge
```

While Environment Modules are widely used, another tool called [Spack](https://spack.readthedocs.io/en/latest/index.html) was specifically developed for dependency management in shared computing environments. You may encounter either system while working on an HPC cluster. The instructions on this page assume Environment Modules.

## Configuring Alamo on an HPC Cluster

Configuring Alamo on an HPC cluster is similar to configuring it on a local machine. However, you must ensure that a compiler, `clang`, and Python 3 are available via modules or other means. Additionally, Alamo relies on the [Eigen library](https://eigen.tuxfamily.org/index.php?title=Main_Page), which can either be loaded as a module or installed during configuration with the `--get-eigen` flag:

```bash
./configure --get-eigen ...
```

Using `--get-eigen` is preferred.

## Compiling Alamo on an HPC Cluster

Compiling code can be resource-intensive and time-consuming. Running large, multithreaded operations on the [login node](https://www.hpc.iastate.edu/guides/introduction-to-hpc-clusters/what-is-an-hpc-cluster) of an HPC cluster is generally discouraged. To avoid this, compile within an interactive job or by submitting a batch job.

If the cluster uses the [Slurm Workload Manager](https://slurm.schedmd.com/overview.html), an interactive job can be started with [`salloc`](https://slurm.schedmd.com/salloc.html):

```bash
salloc --nodes=1 --cpus-per-task=16 --mem=16G --time=10:00
```

This command requests a single node with 16 cores and 16 GB of memory for 10 minutes. Once the resources are allocated, your shell will reload, and you can compile with:

```bash
make -j16
```

Replace `16` with the number of cores requested.

Alternatively, if interactivity is not required, submit a non-interactive compilation job using [`srun`](https://slurm.schedmd.com/srun.html):

```bash
srun --nodes=1 --cpus-per-task=16 --mem=16G --time=10:00 make -j16
```

Adjust resource requests as needed. To reduce wait times, specify a shorter duration using the `--time` flag.

## Running Alamo on an HPC Cluster

To verify that a simulation starts correctly, an interactive job may suffice. For full simulations, submit a batch job to the cluster's workload manager. For Slurm-based clusters, use [`sbatch`](https://slurm.schedmd.com/sbatch.html) to submit a job script:

```bash
sbatch /path/to/job_script
```

The sections above include example job scripts that can be modified to suit your needs.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://wiki.solids.group/alamo/hpc.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.