Differences

This shows you the differences between two versions of the page.

--- software-list [2023/04/05 09:38] – joebathelt
+++ software-list [2023/04/21 09:45] (current) – jonas
@@ Line 17: / Line 17: @@
 ===== Containerised environments =====
+[Contributed by Joe Bathelt]
 Let's dive into the fascinating world of containers! Are you tired of dealing with version conflicts and complicated installations of different software packages on your high-performance computer cluster (HPC)? Say no more! Containers provide an easy and self-contained environment for your analysis. They allow you to use any software without the need for systems administrators to install it on the HPC. With containers, you can run complex data analyses on large datasets that would take weeks on a desktop computer, in just a matter of hours!
@@ Line 24: / Line 26: @@
 You might have heard of Docker, which is a popular container environment software that is easy to install and widely used in software development and science. But have you heard of Singularity? It's also a container environment software developed specifically for HPCs. And the best part? You don't need admin rights to run it!
-If you're not sure which container to use for your analysis, don't worry! There are already many fully developed and tested pipelines available, such as //fmriprep// [[https://fmriprep.org/en/stable/]], which is a fully-fledged pipeline for processing functional MRI data. You can find other containerised pipelines on NiPreps [[https://www.nipreps.org]], Google, or by reading through recent publications in //NeuroImage// or //Human Brain Mapping//.
+If you're not sure which container to use for your analysis, don't worry! There are already many fully developed and tested pipelines available, such as //fmriprep// [[https://fmriprep.org/en/stable/]], which is a widely used pipeline for processing functional MRI data. You can find other containerised pipelines on [[https://www.nipreps.org]], Google, or by reading through recent publications in //NeuroImage// or //Human Brain Mapping//.
 Most containerised pipelines were implemented in Docker. To run it on the cluster with Singularity, you will need to convert it. Here is an example of how to do this for //fmriprep// (thanks to Carl Hodgetts for testing this out):
-''
+<code>
-docker run --privileged -t --rm -v /var/run/docker.sock:/var/run/docker.sock -v /Users/carlhodgetts/my_images:/output singularityware/docker2singularity:v2.6 nipreps/fmriprep:23.0.1
+docker run --privileged -t --rm \
-''
+    -v /var/run/docker.sock:/var/run/docker.sock \
+    -v /Users/carlhodgetts/my_images:/output \
+    singularityware/docker2singularity:v2.6 nipreps/fmriprep:23.0.1
+</code>
 Once this is done, copy the resulting container to the cluster using //scp// and run the container on the cluster like this:
-''
+<code>
 singularity run --cleanenv /path/to/image/fmriprep.simg \
     path/to/data/dir path/to/output/dir \
@@ Line 39: / Line 44: @@
     participant \
     --participant-label label
-''
+</code>
 Check the //fmriprep// documentation for additional configuration options: [[https://fmriprep.org/en/1.5.9/singularity.html]]
+==== Making your own container ====
-You want to use your own analysis pipeline, I hear you say. No problem, it's easy with neurodocker! It generates a file containing all the required software information for your container. Once you have your file, you can build your container and transfer it to the cluster to run your analysis.
+You want to use your own analysis pipeline, I hear you say. No problem, it's easy with neurodocker! It generates a file containing all the required software information for your container. Once you have your file, you can build your container and transfer it to the cluster to run your analysis. More info here: [[https://www.repronim.org/neurodocker/index.html]]
 Generate the "recipe" file:
-''
+<code>
 neurodocker generate singularity \
     --pkg-manager yum \
@@ Line 54: / Line 60: @@
     --afni method=binaries version=latest \
 > afni-binaries.Singularityfile
-''
+</code>
@@ Line 61: / Line 67: @@
-''
+<code>
 sudo singularity build afni.simg afni-binaries.Singularityfile
-''
+</code>
+Please note that you need admin rights to build the container. This cannot be done by regular users on the HPC; please contact Steve Hammett or Jonas Larsson for assistance. If you have a Mac or Windows PC, you can install a virtual machine with Linux and build the container there. Ensure that your local version of Singularity is the same as the one on the cluster, i.e. Version 2.6
-Please note that you need admin rights to build the container. This cannot be done by regular users on the HPC. If you have a Mac or Windows PC, you can install a virtual machine with Linux and build the container there.
+==== Using your Singularity container ====
-**Using your singularity image**:
+There are different ways to use your singularity container. I divided this by use case here:
-There are different ways to use your singularity container. I divided this here by use case:
 Test your image:
-''singularity shell singularity-image.simg''
+<code>singularity shell singularity-image.simg</code>
-- You can run your singularity container interactively. This is similar to running a virtual machine.
+  * You can run your singularity container interactively. This is similar to running a virtual machine.
-- This is great when you want to check if everything is working as expected.
+  * This is great when you want to check if everything is working as expected.
-- Do not be tempted to make changes in the singularity image. They will not persist. Instead, you should modify the singularity instruction file, i.e. the one generated by neurodocker.
+  * Do not be tempted to make changes in the singularity image. They will not persist. Instead, you should modify the singularity instruction file, i.e. the one generated by neurodocker.
 Run a specific command:
-''singularity exec singularity-image.simg command''
+<code>singularity exec singularity-image.simg command</code>
-- You can execute a specific command within your singularity image. This is like the regular command line for running that command with the added parts for running it via singularity. You can specify inputs and parameters as usual.
+  * You can execute a specific command within your singularity image. This is like the regular command line for running that command with the added parts for running it via Singularity. You can specify inputs and parameters as usual.
-- This is useful when you have a singularity image with several commands that you may like to run
+  * This is useful when you have a singularity image with several commands that you may like to run
 Run a script:
-''singularity run singularity-image.simg''
+<code>singularity run singularity-image.simg</code>
-- You can get singularity to automatically execute default command.
+  * You can get singularity to automatically execute the default command.
-- You need to specific the default script in the singularity build recipe, i.e. the one generated by neurodocker
+  * You need to specify the default script in the singularity build recipe, i.e. the one generated by //neurodocker//
-- This is useful for running a pipeline or series of commands, e.g. the singularity version of fmriprep works like this.
+  * This is useful for running a pipeline or series of commands, e.g. the singularity version of //fmriprep// works like this.