kevin/ma
1
0
Fork 0
Code Issues 1 Packages Activity

add nextflow d30e48d

Browse Source
This commit is contained in:
Kevin
2026-04-29 23:01:54 +02:00
parent d0b12d668d
commit 97cc9058d3
2840 changed files with 730250 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

347
nextflow/docs/config.md Normal file
View File

@@ -0,0 +1,347 @@
(config-page)=
# Configuration
## Configuration files
When you launch a pipeline script, Nextflow detects configuration files from multiple sources and applies them in the following order (from lowest to highest priority):
1. `$NXF_HOME/config` (defaults to `$HOME/.nextflow/config`)
2. `nextflow.config` in the project directory
3. `nextflow.config` in the launch directory
4. Config files specified with `-c <config-files>`
:::{tip}
You can use the `-C <config-file>` option to specify a fixed set of configuration files and ignore all other files.
:::
(config-syntax)=
## Syntax
Nextflow configuration uses a similar syntax as Nextflow scripts. Configuration options must be set declaratively, but the value of a config option can be an arbitrary expression.
A config file may contain any number of [assignments](#assignments), [blocks](#blocks), and [includes](#includes). You can add comments just like in scripts.
### Assignments
A config assignment sets a config option to a value:
```groovy
workDir = 'work'
docker.enabled = true
process.maxErrors = 10
```
The config option consists of an *option name* prefixed by any number of *scopes*. Scopes group related config options. See {ref}`config-options` for the full set of config options.
The value is typically a literal value, such as a number, boolean, or string. However, you can use any {ref}`expression <syntax-expressions>`:
```groovy
params.helper_file = "${projectDir}/assets/helper.txt"
```
### Blocks
You can specify config options in the same scope as a block:
```groovy
// dot syntax
docker.enabled = true
docker.runOptions = '-u $(id -u):$(id -g)'
// block syntax
docker {
enabled = true
runOptions = '-u $(id -u):$(id -g)'
}
```
As a result, deeply nested config options can be assigned in multiple ways. The following three assignments are equivalent:
```groovy
executor.retry.maxAttempt = 5
executor {
retry.maxAttempt = 5
}
executor {
retry {
maxAttempt = 5
}
}
```
### Includes
You can include configuration files in other configuration files with the `includeConfig` keyword:
```groovy
process.executor = 'sge'
process.queue = 'long'
process.memory = '10G'
includeConfig 'path/extra.config'
```
Relative paths are resolved against the location of the including file.
:::{note}
Config files should only be included at the top level or in a [profile](#config-profiles). This ensures the included config file is valid on its own and in the context in which it is included.
:::
## Constants and functions
The following constants are globally available in a Nextflow configuration file:
`baseDir: Path`
: :::{deprecated} 20.04.0
:::
: Alias for `projectDir`.
`launchDir: Path`
: The directory where the workflow was launched.
`projectDir: Path`
: The directory where the main script is located.
`secrets: Map<String,String>`
: Map of pipeline secrets. See {ref}`secrets-page` for more information.
The following functions are globally available in a Nextflow configuration file:
`env( name: String ) -> String`
: :::{versionadded} 25.04.0
:::
: Get the value of the environment variable with the specified name in the Nextflow launch environment.
(config-params)=
## Parameters
You can define pipeline parameters in config files using the `params` scope:
```groovy
// dot syntax
params.max_cpus = 64
params.publish_mode = 'copy'
// block syntax
params {
max_cpus = 64
publish_mode = 'copy'
}
```
:::{note}
When including a config file, the included config is evaluated with the parameters defined before the include. The included config cannot see parameters defined after the include.
:::
You should declare parameters in the config file only when other config options use them. When you use a parameter in the script, you should declare it there and override it in config profiles as needed:
```nextflow
// main.nf
params.input = null
```
```groovy
// nextflow.config
params {
publish_mode = 'copy'
}
workflow.output.mode = params.publish_mode
profiles {
test {
params.input = "${projectDir}/test/input.txt"
}
}
```
See {ref}`cli-params` for information about how Nextflow resolves pipeline parameters at runtime.
(config-process)=
## Process configuration
You can use the `process` scope to specify {ref}`process directives <process-reference>` separately from the pipeline code.
For example:
```groovy
process {
executor = 'sge'
queue = 'long'
clusterOptions = '-pe smp 10 -l virtual_free=64G,h_rt=30:00:00'
}
```
This configuration executes all processes using the SGE executor with the given settings.
(config-process-selectors)=
### Process selectors
You can use the `withLabel` selector to configure all processes annotated with a {ref}`process-label` directive:
```groovy
process {
withLabel: big_mem {
cpus = 16
memory = 64.GB
queue = 'long'
}
}
```
This configuration assigns 16 CPUs, 64 GB of memory, and the `long` queue to all processes with the `big_mem` label.
You can use the `withName` selector to configure a specific process by its name:
```groovy
process {
withName: hello {
cpus = 4
memory = 8.GB
queue = 'short'
}
}
```
The `withName` selector matches both:
- Processes defined with that name
- Processes included under that alias
When you include a process with an alias, selectors for the alias take priority over selectors for the original name. For example, if you define a process as `hello` and include it as `sayHello`, both `withName: hello` and `withName: sayHello` apply, with `withName: sayHello` taking priority.
:::{tip}
You don't need to enclose label and process names in quotes unless they contain special characters (`-`, `!`, etc.) or are keywords or built-in type identifiers.
:::
(config-selector-expressions)=
### Selector expressions
You can use regular expressions in process selectors to apply the same configuration to all processes matching the pattern:
```groovy
process {
withLabel: 'hello|bye' {
cpus = 2
memory = 4.GB
}
}
```
This configuration requests 2 CPUs and 4 GB of memory for processes labeled as `hello` or `bye`.
You can negate a selector expression by prefixing it with the special character `!`:
```groovy
process {
withLabel: 'hello' { cpus = 2 }
withLabel: '!hello' { cpus = 4 }
withName: '!align.*' { queue = 'long' }
}
```
This configuration sets 2 CPUs for processes labeled as `hello` and 4 CPUs for all processes *not* labeled as `hello`. It also specifies the `long` queue for processes whose name does *not* start with `align`.
(config-selector-priority)=
### Selector priority
Nextflow applies process configuration settings in the following order (from lowest to highest priority):
1. Process configuration settings (without a selector)
2. Process directives in the process definition
3. `withLabel` selectors matching any of the process labels
4. `withName` selectors matching the process name
5. `withName` selectors matching the process included alias
6. `withName` selectors matching the process fully qualified name
For example:
```groovy
process {
cpus = 4
withLabel: hello { cpus = 8 }
withName: bye { cpus = 16 }
withName: 'aloha:bye' { cpus = 32 }
}
```
This configuration:
- Sets 4 CPUs for all processes (unless otherwise specified in their process definition)
- Sets 8 CPUs for processes annotated with the `hello` label
- Sets 16 CPUs for any process named `bye` (or imported as `bye`)
- Sets 32 CPUs for any process named `bye` (or imported as `bye`) invoked by a workflow named `aloha`
(config-profiles)=
## Config profiles
Configuration files can define one or more *profiles*. A profile is a set of configuration settings that can be selected at runtime using the `-profile` command line option.
Configuration profiles are defined in the `profiles` scope. For example:
```groovy
profiles {
standard {
process.executor = 'local'
}
cluster {
process.executor = 'sge'
process.queue = 'long'
process.memory = '10GB'
}
cloud {
process.executor = 'cirrus'
process.container = 'cbcrg/imagex'
docker.enabled = true
}
}
```
This configuration defines three profiles: `standard`, `cluster`, and `cloud`. Each profile provides a different configuration for a given execution environment. When you do not specify a profile, Nextflow uses the `standard` profile by default.
You can enable configuration profiles at runtime as a comma-separated list:
```bash
nextflow run main.nf -profile standard,cloud
```
Nextflow applies config profiles in the order in which they were defined in the config, regardless of the order you specify them on the command line.
:::{versionadded} 25.04.0
When using the {ref}`strict syntax <strict-syntax-page>`, Nextflow applies profiles in the order you specify them on the command line.
:::
(config-workflow-handlers)=
## Workflow handlers
:::{deprecated} 25.10.0
Use a {ref}`trace observer <plugins-trace-observers>` in a plugin to add custom workflow handlers to a pipeline via configuration.
:::
You can define workflow event handlers in the config file:
```groovy
workflow.onComplete = {
println "Pipeline complete"
println "Command line: $workflow.commandLine"
}
workflow.onError = {
println "Error: something went wrong"
}
```
This approach is useful for handling workflow events without modifying the pipeline code. See {ref}`workflow-handlers` for more information.