(config-page)= # Configuration ## Configuration files When you launch a pipeline script, Nextflow detects configuration files from multiple sources and applies them in the following order (from lowest to highest priority): 1. `$NXF_HOME/config` (defaults to `$HOME/.nextflow/config`) 2. `nextflow.config` in the project directory 3. `nextflow.config` in the launch directory 4. Config files specified with `-c ` :::{tip} You can use the `-C ` option to specify a fixed set of configuration files and ignore all other files. ::: (config-syntax)= ## Syntax Nextflow configuration uses a similar syntax as Nextflow scripts. Configuration options must be set declaratively, but the value of a config option can be an arbitrary expression. A config file may contain any number of [assignments](#assignments), [blocks](#blocks), and [includes](#includes). You can add comments just like in scripts. ### Assignments A config assignment sets a config option to a value: ```groovy workDir = 'work' docker.enabled = true process.maxErrors = 10 ``` The config option consists of an *option name* prefixed by any number of *scopes*. Scopes group related config options. See {ref}`config-options` for the full set of config options. The value is typically a literal value, such as a number, boolean, or string. However, you can use any {ref}`expression `: ```groovy params.helper_file = "${projectDir}/assets/helper.txt" ``` ### Blocks You can specify config options in the same scope as a block: ```groovy // dot syntax docker.enabled = true docker.runOptions = '-u $(id -u):$(id -g)' // block syntax docker { enabled = true runOptions = '-u $(id -u):$(id -g)' } ``` As a result, deeply nested config options can be assigned in multiple ways. The following three assignments are equivalent: ```groovy executor.retry.maxAttempt = 5 executor { retry.maxAttempt = 5 } executor { retry { maxAttempt = 5 } } ``` ### Includes You can include configuration files in other configuration files with the `includeConfig` keyword: ```groovy process.executor = 'sge' process.queue = 'long' process.memory = '10G' includeConfig 'path/extra.config' ``` Relative paths are resolved against the location of the including file. :::{note} Config files should only be included at the top level or in a [profile](#config-profiles). This ensures the included config file is valid on its own and in the context in which it is included. ::: ## Constants and functions The following constants are globally available in a Nextflow configuration file: `baseDir: Path` : :::{deprecated} 20.04.0 ::: : Alias for `projectDir`. `launchDir: Path` : The directory where the workflow was launched. `projectDir: Path` : The directory where the main script is located. `secrets: Map` : Map of pipeline secrets. See {ref}`secrets-page` for more information. The following functions are globally available in a Nextflow configuration file: `env( name: String ) -> String` : :::{versionadded} 25.04.0 ::: : Get the value of the environment variable with the specified name in the Nextflow launch environment. (config-params)= ## Parameters You can define pipeline parameters in config files using the `params` scope: ```groovy // dot syntax params.max_cpus = 64 params.publish_mode = 'copy' // block syntax params { max_cpus = 64 publish_mode = 'copy' } ``` :::{note} When including a config file, the included config is evaluated with the parameters defined before the include. The included config cannot see parameters defined after the include. ::: You should declare parameters in the config file only when other config options use them. When you use a parameter in the script, you should declare it there and override it in config profiles as needed: ```nextflow // main.nf params.input = null ``` ```groovy // nextflow.config params { publish_mode = 'copy' } workflow.output.mode = params.publish_mode profiles { test { params.input = "${projectDir}/test/input.txt" } } ``` See {ref}`cli-params` for information about how Nextflow resolves pipeline parameters at runtime. (config-process)= ## Process configuration You can use the `process` scope to specify {ref}`process directives ` separately from the pipeline code. For example: ```groovy process { executor = 'sge' queue = 'long' clusterOptions = '-pe smp 10 -l virtual_free=64G,h_rt=30:00:00' } ``` This configuration executes all processes using the SGE executor with the given settings. (config-process-selectors)= ### Process selectors You can use the `withLabel` selector to configure all processes annotated with a {ref}`process-label` directive: ```groovy process { withLabel: big_mem { cpus = 16 memory = 64.GB queue = 'long' } } ``` This configuration assigns 16 CPUs, 64 GB of memory, and the `long` queue to all processes with the `big_mem` label. You can use the `withName` selector to configure a specific process by its name: ```groovy process { withName: hello { cpus = 4 memory = 8.GB queue = 'short' } } ``` The `withName` selector matches both: - Processes defined with that name - Processes included under that alias When you include a process with an alias, selectors for the alias take priority over selectors for the original name. For example, if you define a process as `hello` and include it as `sayHello`, both `withName: hello` and `withName: sayHello` apply, with `withName: sayHello` taking priority. :::{tip} You don't need to enclose label and process names in quotes unless they contain special characters (`-`, `!`, etc.) or are keywords or built-in type identifiers. ::: (config-selector-expressions)= ### Selector expressions You can use regular expressions in process selectors to apply the same configuration to all processes matching the pattern: ```groovy process { withLabel: 'hello|bye' { cpus = 2 memory = 4.GB } } ``` This configuration requests 2 CPUs and 4 GB of memory for processes labeled as `hello` or `bye`. You can negate a selector expression by prefixing it with the special character `!`: ```groovy process { withLabel: 'hello' { cpus = 2 } withLabel: '!hello' { cpus = 4 } withName: '!align.*' { queue = 'long' } } ``` This configuration sets 2 CPUs for processes labeled as `hello` and 4 CPUs for all processes *not* labeled as `hello`. It also specifies the `long` queue for processes whose name does *not* start with `align`. (config-selector-priority)= ### Selector priority Nextflow applies process configuration settings in the following order (from lowest to highest priority): 1. Process configuration settings (without a selector) 2. Process directives in the process definition 3. `withLabel` selectors matching any of the process labels 4. `withName` selectors matching the process name 5. `withName` selectors matching the process included alias 6. `withName` selectors matching the process fully qualified name For example: ```groovy process { cpus = 4 withLabel: hello { cpus = 8 } withName: bye { cpus = 16 } withName: 'aloha:bye' { cpus = 32 } } ``` This configuration: - Sets 4 CPUs for all processes (unless otherwise specified in their process definition) - Sets 8 CPUs for processes annotated with the `hello` label - Sets 16 CPUs for any process named `bye` (or imported as `bye`) - Sets 32 CPUs for any process named `bye` (or imported as `bye`) invoked by a workflow named `aloha` (config-profiles)= ## Config profiles Configuration files can define one or more *profiles*. A profile is a set of configuration settings that can be selected at runtime using the `-profile` command line option. Configuration profiles are defined in the `profiles` scope. For example: ```groovy profiles { standard { process.executor = 'local' } cluster { process.executor = 'sge' process.queue = 'long' process.memory = '10GB' } cloud { process.executor = 'cirrus' process.container = 'cbcrg/imagex' docker.enabled = true } } ``` This configuration defines three profiles: `standard`, `cluster`, and `cloud`. Each profile provides a different configuration for a given execution environment. When you do not specify a profile, Nextflow uses the `standard` profile by default. You can enable configuration profiles at runtime as a comma-separated list: ```bash nextflow run main.nf -profile standard,cloud ``` Nextflow applies config profiles in the order in which they were defined in the config, regardless of the order you specify them on the command line. :::{versionadded} 25.04.0 When using the {ref}`strict syntax `, Nextflow applies profiles in the order you specify them on the command line. ::: (config-workflow-handlers)= ## Workflow handlers :::{deprecated} 25.10.0 Use a {ref}`trace observer ` in a plugin to add custom workflow handlers to a pipeline via configuration. ::: You can define workflow event handlers in the config file: ```groovy workflow.onComplete = { println "Pipeline complete" println "Command line: $workflow.commandLine" } workflow.onError = { println "Error: something went wrong" } ``` This approach is useful for handling workflow events without modifying the pipeline code. See {ref}`workflow-handlers` for more information.