Podman containers

Nextflow integration with Podman containers technology allows you to write self-contained and truly reproducible computational pipelines.

By using this feature any process in a Nextflow script can be transparently executed into a Podman container. This may be extremely useful to package the binary dependencies of a script into a standard and portable format that can be executed on any platform supporting the Podman engine.

Note

This feature requires Nextflow version 20.01.0 or later.

Warning

This is an incubating feature. The use in production environment is not recommended.

Prerequisites

You will need Podman installed on your execution environment e.g. your computer or a distributed cluster, depending on where you want to run your pipeline. Running in rootless mode requires appropriate OS configuration. Due to current Podman limits using cpuset for cpus and memory such is only possible using sudo.

How it works

You won't need to modify your Nextflow script in order to run it with Podman. Simply specify the Podman image from where the containers are started by using the -with-podman command line option. For example:

nextflow run <your script> -with-podman [OCI container image]

Every time your script launches a process execution, Nextflow will run it into a Podman container created by using the specified image. In practice Nextflow will automatically wrap your processes and run them by executing the podman run command with the image you have provided.

Note

A OCI container image can contain any tool or piece of software you may need to carry out a process execution. Moreover the container is run in such a way that the process result files are created in the hosting file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.

If you want to avoid entering the Podman image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the nextflow.config file:

process.container = 'nextflow/examples:latest'
podman.enabled = true

In the above example replace nextflow/examples:latest with any Podman image of your choice.

Read the Configuration page to learn more about the nextflow.config file and how to use it to configure your pipeline execution.

Warning

Nextflow automatically manages the file system mounts each time a container is launched depending on the process input files. Note, however, that when a process input is a symbolic link file, the linked file must be stored in the same folder where the symlink is located, or any its sub-folder. Otherwise the process execution will fail because the launched container won't be able to access the linked file.

Multiple containers

It is possible to specify a different container image for each process definition in your pipeline script. Let's suppose you have two processes named foo and bar. You can specify two different container images for them in the Nextflow script as shown below:

process foo {
  container 'image_name_1'

  '''
  do this
  '''
}

process bar {
  container 'image_name_2'

  '''
  do that
  '''
}

Alternatively, the same containers definitions can be provided by using the nextflow.config file as shown below:

process {
    withName:foo {
        container = 'image_name_1'
    }
    withName:bar {
        container = 'image_name_2'
    }
}
podman {
    enabled = true
}

Read the Process scope section to learn more about processes configuration.

Advanced settings

Podman advanced configuration settings are described in Scope podman section in the Nextflow configuration page.