GPU Compilation on Nova

Steps

1. Load the following modules:

   module load git openmpi cuda

2. Check out the Alamo repository:

   git clone https://github.com/solidsgroup/alamo.git

   For this workflow, check out the gpu branch:

   git checkout gpu

3. In the Alamo directory, configure with:

   ./configure --get-eigen --cuda

   The Nova eigen module will not work for this workflow.
4. Compile the currently working example:

   make bin/heat

   Optionally start an interactive compilation session to speed up the build:

   srun --partition=interactive --pty bash

   The modules must be reloaded inside the interactive session.

   You may need to run make more than once; the old wiki noted odd issues when compiling in parallel.
5. With Alamo compiled, start a new interactive session:

   srun --partition=interactive --gres=gpu:a100:1 --pty bash

   Load CUDA:

   module load cuda

6. Run:

   ./bin/heat-3d-cuda80-g++ tests/HeatConduction/input

   The old wiki reported that this did not work because libcuda.so.1 could not be found. Symlinking to the /stub library was attempted, but the stub library could not be used.