Singularity is a Linux container system similar (and compatible to) Docker. It’s advantage over Docker is that is was designed to allow users without superuser privileges to run containers within their environment. I recently encountered the following error when running a Nvidia CUDA application within a Singularity container using “singularity run -nv <container>:
CUDART: cudaGetDeviceCount(&deviceCount); = 999 (unknown error)
Workaround: After running /usr/local/cuda/samples/1_Utilities/deviceQuery/deviceQuery outside of the container, the CUDA application within the container ran without any problem.