add nextflow d30e48d

This commit is contained in:
2026-04-29 23:01:54 +02:00
parent d0b12d668d
commit 97cc9058d3
2840 changed files with 730250 additions and 0 deletions

View File

@@ -0,0 +1,454 @@
(channel-factory)=
# Channel factories
This page describes the channel factories that are available in the `channel` namespace.
(channel-empty)=
## empty
**`empty() -> Channel`**
The `channel.empty` factory creates a channel that emits nothing:
```nextflow
channel.empty().view() // prints nothing
```
(channel-filepairs)=
## fromFilePairs
:::{note}
As a best practice, use a samplesheet instead of matching file pairs directly with glob patterns. See {ref}`static-types-samplesheet` for more information.
:::
The `channel.fromFilePairs` factory creates a channel that emits file pairs matching a [glob][glob] pattern:
```nextflow
ch = channel.fromFilePairs('/my/data/SRR*_{1,2}.fastq')
ch.view()
```
Each file pair is emitted as a 2-tuple containing the grouping key (from the `*` wildcard) and the list of files (sorted lexicographically):
```
[SRR493366, [/my/data/SRR493366_1.fastq, /my/data/SRR493366_2.fastq]]
[SRR493367, [/my/data/SRR493367_1.fastq, /my/data/SRR493367_2.fastq]]
[SRR493368, [/my/data/SRR493368_1.fastq, /my/data/SRR493368_2.fastq]]
[SRR493369, [/my/data/SRR493369_1.fastq, /my/data/SRR493369_2.fastq]]
[SRR493370, [/my/data/SRR493370_1.fastq, /my/data/SRR493370_2.fastq]]
[SRR493371, [/my/data/SRR493371_1.fastq, /my/data/SRR493371_2.fastq]]
```
The glob pattern must contain at least one `*` wildcard character.
Available options:
`checkIfExists`
: When `true`, throws an error if the file path does not exist in the file system (default: `false`).
`flat`
: When `true`, tuples are emitted with the matching files flattened instead of as a nested list (default: `false`).
`followLinks`
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`).
`hidden`
: When `true`, matches hidden files when using a glob pattern (default: `false`).
`maxDepth`
: Maximum number of directory levels to visit with the `**` wildcard (default: no limit).
`size`
: The number of expected files for each file pair (default: `2`). Set to `-1` to allow any size.
`type`
: Whether to return only files (`'file'`), only directories (`'dir'`), or both (`'any'`) when using a glob pattern. By default, only files are returned (`'file'`).
(channel-from-lineage)=
## fromLineage
:::{versionadded} 25.04.0
:::
:::{warning} *Experimental: may change in a future release.*
:::
**`fromLineage( [opts] ) -> Channel<Path>`**
The `channel.fromLineage` factory creates a channel that emits files from the {ref}`cli-lineage` store that match the given key-value params:
```nextflow
ch = channel.fromLineage(
workflowRun: 'lid://0d1d1622ced3e4edc690bec768919b45',
label: ['alpha', 'beta']
)
ch.view()
```
The above snippet emits files published by the given workflow run that are labeled as `alpha` and `beta`.
Available options:
`label`
: List of labels associated with the desired files.
`taskRun`
: LID of the task run that produced the desired files.
`workflowRun`
: LID of the workflow run that produced the desired files.
(channel-fromlist)=
## fromList
**`fromList( values: Iterable<E> ) -> Channel<E>`**
The `channel.fromList` factory creates a channel that emits each element in a collection:
```nextflow
ch = channel.fromList( ['a', 'b', 'c', 'd'] )
ch.view { v -> "value: $v" }
```
Prints:
```
value: a
value: b
value: c
value: d
```
See also: [channel.of](#of)
(channel-path)=
## fromPath
**`fromPath( pattern: String, [opts] ) -> Channel<Path>`**
The `channel.fromPath` factory creates a channel that emits file paths matching a name or [glob][glob] pattern:
```nextflow
// match single file
channel.fromPath('data/some/bigfile.txt')
// match `txt` files in `data/bag`
channel.fromPath('data/big/*.txt')
// match `fa` files in `data` and its subdirectories
channel.fromPath('data/**.fa')
// match `fa` files with same suffix in any subdirectory of `data`
channel.fromPath('data/**/*.fa')
// match file pair (`file_1.fq` and `file_2.fq`)
channel.fromPath('data/file_{1,2}.fq')
```
By default, glob patterns do not match hidden files (i.e. files with names that start with `.`). Use a glob pattern that explicitly starts with `.` or set `hidden: true` to match hidden files:
```nextflow
// match hidden files in `data`
channel.fromPath('data/.*')
channel.fromPath('data/*', hidden: true)
// match hidden files in `data` with `fa` extension
channel.fromPath('data/.*.fa')
```
By default, glob patterns only match regular files, not directories. Use the `type` option to control whether to match files, directories, or both:
```nextflow
// match only directories
channel.fromPath('data/*', type: 'dir')
// match files and directories
channel.fromPath('data/*', type: 'any')
```
Available options:
`checkIfExists`
: When `true`, throws an error if the file path does not exist in the file system (default: `false`).
`followLinks`
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`).
`glob`
: When `true`, interprets the characters `*`, `?`, `[]`, and `{}` as glob wildcards, otherwise treats them as normal characters (default: `true`).
`hidden`
: When `true`, matches hidden files when using a glob pattern (default: `false`).
`maxDepth`
: Maximum number of directory levels to visit with the `**` wildcard (default: no limit).
`relative`
: When `true`, returns file paths as relative to the top-most common directory (default: `false`).
`type`
: Whether to return only files (`'file'`), only directories (`'dir'`), or both (`'any'`) when using a glob pattern. By default, only files are returned (`'file'`).
(channel-fromsra)=
## fromSRA
:::{deprecated} 26.04.0
Use the [Entrez Direct](https://www.ncbi.nlm.nih.gov/books/NBK179288/) command-line tool to query the SRA database.
:::
The `channel.fromSRA` factory queries the [NCBI SRA](https://www.ncbi.nlm.nih.gov/sra) database and returns a channel emitting the FASTQ files matching the specified criteria i.e project or accession number(s). For example:
```nextflow
channel.fromSRA('SRP043510').view()
```
It returns:
```
[SRR1448794, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/004/SRR1448794/SRR1448794.fastq.gz]
[SRR1448795, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/005/SRR1448795/SRR1448795.fastq.gz]
[SRR1448792, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/002/SRR1448792/SRR1448792.fastq.gz]
[SRR1448793, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/003/SRR1448793/SRR1448793.fastq.gz]
[SRR1910483, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR191/003/SRR1910483/SRR1910483.fastq.gz]
[SRR1910482, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR191/002/SRR1910482/SRR1910482.fastq.gz]
(remaining omitted)
```
Multiple accession IDs can be specified as a list:
```nextflow
ids = ['ERR908507', 'ERR908506', 'ERR908505']
channel.fromSRA(ids).view()
```
```
[ERR908507, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908507/ERR908507_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908507/ERR908507_2.fastq.gz]]
[ERR908506, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908506/ERR908506_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908506/ERR908506_2.fastq.gz]]
[ERR908505, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908505/ERR908505_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908505/ERR908505_2.fastq.gz]]
```
Each read pair is implicitly managed and returned as a list of files.
This method uses the NCBI [ESearch](https://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch) API behind the scenes, therefore it allows the use of any query term supported by this API.
To access the ESearch API, you must provide your [NCBI API keys](https://ncbiinsights.ncbi.nlm.nih.gov/2017/11/02/new-api-keys-for-the-e-utilities) through one of the following ways:
- The `apiKey` option:
```nextflow
channel.fromSRA(ids, apiKey:'0123456789abcdef')
```
- The `NCBI_API_KEY` variable in your environment:
```bash
export NCBI_API_KEY=0123456789abcdef
```
Available options:
`apiKey`
: NCBI user API key.
`cache`
: Enable/disable the caching API requests (default: `true`).
`max`
: Maximum number of entries that can be retried (default: unlimited) .
`protocol`
: Allow choosing the protocol for the resulting remote URLs. Available choices: `ftp`, `http`, `https` (default: `ftp`).
`retryPolicy`
: Set a retry policy in case the SRA request fails with a retriable error.
: Available properties:
- `delay`: Delay between attempts (default: `500ms`)
- `jitter`: Jitter value (default: `0.25`)
- `maxAttempts`: Max attempts (default: `3`)
- `maxDelay`: Max delay (default: `30s`)
: For example:
```nextflow
channel.fromSRA(ids, retryPolicy: [delay: '250ms', maxAttempts: 5])
```
(channel-interval)=
## interval
**`interval( interval: String ) -> Channel<Integer>`**
The `channel.interval` factory emits an incrementing index (starting from zero) at a periodic interval. For example:
```nextflow
channel.interval('1s').view()
```
The above snippet will emit 0, 1, 2, and so on, every second, forever. You can use an operator such as {ref}`operator-take` or {ref}`operator-until` to close the channel based on a stopping condition.
(channel-of)=
## of
**`of( values... ) -> Channel`**
The `channel.of` factory allows you to create a channel that emits each argument:
```nextflow
ch = channel.of( 1, 3, 5, 7 ).view()
```
Prints:
```
1
3
5
7
```
Ranges of values are expanded accordingly:
```nextflow
channel.of(1..23, 'X', 'Y').view()
```
Prints:
```
1
2
3
4
:
23
X
Y
```
See also: [channel.fromList](#fromlist)
(channel-topic)=
## topic
:::{versionadded} 25.04.0
:::
:::{note}
This feature was previewed in versions 24.04 and 24.10 with the `nextflow.preview.topic` feature flag.
:::
**`topic( name: String ) -> Channel`**
A *topic channel* is a channel that can receive values from many sources *implicitly* based on a matching *topic name*.
A typed process can emit values to a topic using the `topic:` section:
```nextflow
nextflow.enable.types = true
process hello {
topic:
file('hello.txt') >> 'my-topic'
// ...
}
process bye {
topic:
file('bye.txt') >> 'my-topic'
// ...
}
```
A legacy process can assign outputs in the `output:` section to a topic using the `topic` option:
```nextflow
process hello {
output:
path('hello.txt'), topic: 'my-topic'
// ...
}
```
The `channel.topic` factory returns the topic channel for the given name:
```nextflow
channel.topic('my-topic').view()
```
The above example emits all values sent to the `my-topic` topic from processes such as `hello` and `bye`.
This approach is a convenient way to collect related items from many different sources without explicitly connecting them (e.g. using the `mix` operator).
:::{warning}
Any process that consumes a topic channel (directly or indirectly) should not send any outputs to that topic, or else the pipeline will hang forever.
:::
See also: {ref}`process-typed-topics` for process outputs
(channel-value)=
## value
**`value( value: V ) -> Value<V>`**
The `channel.value` factory creates a dataflow value bound to the given argument:
```nextflow
channel.value( 'Hello there' )
channel.value( [1,2,3,4,5] )
```
The first line creates a dataflow value bound to the string `'Hello there'`. The second line creates a dataflow value bound to the list `[1,2,3,4,5]`, which is treated as a single value in dataflow logic.
(channel-watchpath)=
## watchPath
**`watchPath( pattern: String, events: String = 'create' ) -> Channel<Path>`**
The `channel.watchPath` factory creates a channel that watches a [glob][glob] pattern and emits matching files as they appear.
For example:
```nextflow
ch = channel.watchPath('/path/*.fa')
ch.view { fa -> "Fasta file: $fa" }
```
The second argument specifies which filesystem events to watch as a comma-separated string:
```nextflow
ch = channel.watchPath('/path/*.fa', 'create,modify')
ch.view { fa -> "File created or modified: $fa" }
```
By default, only new files are watched. The following events are supported:
- `create`: A new file is created
- `modify`: A file is modified
- `delete`: A file is deleted
:::{warning}
The `channel.watchPath` factory waits endlessly for matching files, which means that it will cause your pipeline to run forever. Consider using the `take` or `until` operator to apply a stopping condition (e.g. receiving 10 files, receiving a file named `DONE`).
:::
:::{note}
The `channel.watchPath` factory only works with local and shared filesystems. It does not support object storage such as S3.
:::
See also: [channel.fromPath](#frompath)
[glob]: http://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,317 @@
(config-env-vars)=
# Environment variables
The following environment variables control the configuration of the Nextflow runtime and the underlying Java virtual machine.
## Java settings
`JAVA_CMD`
: Defines the path location of the Java binary command used to launch Nextflow.
`JAVA_HOME`
: Defines the path location of the Java VM installation used to run Nextflow.
(nxf-env-vars)=
## Nextflow settings
`NXF_AGENT_MODE`
: :::{versionadded} 26.04.0
:::
: When `true`, enables agent output mode. In this mode, Nextflow replaces the interactive ANSI log with minimal, structured output optimized for AI agents and non-interactive environments. The output uses tagged lines such as `[PIPELINE]`, `[PROCESS]`, `[WARN]`, `[ERROR]`, and `[SUCCESS]`/`[FAILED]` written to standard output (default: `false`).
`NXF_ANSI_LOG`
: Enables/disables ANSI console output (default `true` when ANSI terminal is detected).
`NXF_ANSI_SUMMARY`
: Enables/disables ANSI completion summary: `true\|false` (default: print summary if execution last more than 1 minute).
`NXF_ASSETS`
: Defines the directory where downloaded pipeline repositories are stored (default: `$NXF_HOME/assets`)
`NXF_CACHE_DIR`
: :::{versionadded} 24.02.0-edge
:::
: Defines the base cache directory when using the default cache store (default: `"$launchDir/.nextflow"`).
`NXF_CHARLIECLOUD_CACHEDIR`
: Directory where remote Charliecloud images are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
`NXF_CLOUDCACHE_PATH`
: :::{versionadded} 23.07.0-edge
:::
: Defines the base cache path when using the cloud cache store.
`NXF_CLOUD_DRIVER`
: Defines the default cloud driver to be used if not specified in the config file or as command line option, either `aws` or `google`.
`NXF_CONDA_CACHEDIR`
: Directory where Conda environments are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
`NXF_CONDA_ENABLED`
: :::{versionadded} 22.08.0-edge
:::
: Enable the use of Conda recipes defined by using the {ref}`process-conda` directive. (default: `false`).
`NXF_CONTAINER_ENTRYPOINT_OVERRIDE`
: :::{deprecated} 22.10.0
:::
: When `true`, override the container entrypoint with `/bin/bash` (default: `false`).
`NXF_DATE_FORMAT`
: :::{versionadded} 25.07.0-edge
:::
: Defines the format for date and time representations in notifications and reports. Supports custom formats (e.g., `yyyy-MM-dd HH:mm:ss`) or `iso` for ISO 8601 format with timezone (default: `dd-MMM-yyyy HH:mm:ss`).
`NXF_DEFAULT_DSL`
: :::{versionadded} 22.03.0-edge
:::
: Defines the DSL version that should be used in not specified otherwise in the script of config file (default: `2`)
`NXF_DISABLE_CHECK_LATEST`
: :::{versionadded} 23.09.0-edge
:::
: Nextflow automatically checks for a newer version of itself unless this option is enabled (default: `false`).
`NXF_DISABLE_JOBS_CANCELLATION`
: :::{versionadded} 21.12.0-edge
:::
: Disables the cancellation of child jobs on workflow execution termination.
`NXF_DISABLE_PARAMS_TYPE_DETECTION`
: :::{versionadded} 23.07.0-edge
:::
: Disables the automatic type detection of command line parameters.
`NXF_DISABLE_WAVE_SERVICE`
: :::{versionadded} 23.08.0-edge
:::
: Disables the requirement for Wave service when enabling the Fusion file system.
`NXF_ENABLE_AWS_SES`
: :::{versionadded} 23.06.0-edge
:::
: Enable to use of AWS SES native API for sending emails in place of legacy SMTP settings (default: `false`)
`NXF_ENABLE_FS_SYNC`
: :::{versionadded} 23.10.0
:::
: When enabled the job script will execute Linux `sync` command on job completion. This may be useful to synchronize the job state over shared file systems (default: `false`)
`NXF_ENABLE_SECRETS`
: :::{versionadded} 21.09.0-edge
:::
: Enable Nextflow secrets features (default: `true`)
`NXF_ENABLE_STRICT`
: :::{versionadded} 22.05.0-edge
:::
: Enable Nextflow *strict* execution mode (default: `false`)
`NXF_ENABLE_VIRTUAL_THREADS`
: :::{versionadded} 23.05.0-edge
:::
: Enable the use of virtual threads in the Nextflow runtime (default: `false`)
`NXF_EXECUTOR`
: Defines the default process executor, e.g. `sge`
`NXF_FILE_ROOT`
: :::{versionadded} 23.05.0-edge
:::
: The file storage path against which relative file paths are resolved.
: For example, with `NXF_FILE_ROOT=/some/root/path`, the use of `file('hello')` will be resolved to the absolute path `/some/root/path/hello`. A remote root path can be specified using the usual protocol prefix, e.g. `NXF_FILE_ROOT=s3://my-bucket/data`. Files defined using an absolute path are not affected by this setting.
`NXF_FUSION_TRACE`
: :::{versionadded} 26.04.0
:::
: When set to `true`, collect task resource metrics (CPU, memory, I/O) from the Fusion trace file (`.fusion/trace.json`) produced in the task work directory, replacing the metrics collected by the default bash command-trace wrapper. Requires Fusion to be enabled. GPU metrics from Fusion are always collected regardless of this setting. Defaults to `false`.
`NXF_HOME`
: Nextflow home directory (default: `$HOME/.nextflow`).
`NXF_JAVA_HOME`
: Defines the path location of the Java VM installation used to run Nextflow. This variable overrides the `JAVA_HOME` variable if defined.
`NXF_JVM_ARGS`
: :::{versionadded} 21.12.1-edge
:::
: Allows the setting Java VM options. This is similar to `NXF_OPTS` however it's only applied the JVM running Nextflow and not to any java pre-launching commands.
`NXF_LOG_FILE`
: The filename of the Nextflow log (default: `.nextflow.log`)
`NXF_OFFLINE`
: When `true` prevents Nextflow from automatically downloading and updating remote project repositories (default: `false`).
: :::{versionchanged} 23.09.0-edge
This option also disables the automatic version check (see `NXF_DISABLE_CHECK_LATEST`).
:::
: :::{versionchanged} 23.11.0-edge
This option also prevents plugins from being downloaded. Plugin versions must be specified in offline mode, or else Nextflow will fail.
:::
`NXF_OPTS`
: Provides extra options for the Java and Nextflow runtime. It must be a blank separated list of `-Dkey[=value]` properties.
`NXF_ORG`
: Default `organization` prefix when looking for a hosted repository (default: `nextflow-io`).
`NXF_PARAMS_FILE`
: :::{versionadded} 20.10.0
:::
: Defines the path location of the pipeline parameters file .
`NXF_PID_FILE`
: Name of the file where the process PID is saved when Nextflow is launched in background.
`NXF_PLUGINS_ALLOWED`
: :::{versionadded} 25.04.0
:::
: Comma separated list of plugin IDs that can be used in a workflow executions e.g. `NXF_PLUGINS_ALLOWED=nf-amazon,nf-tower,nf-wave`. Use empty string to disallow all plugins.
`NXF_PLUGINS_DEFAULT`
: Whether to use the default plugins when no plugins are specified in the Nextflow configuration (default: `true`).
`NXF_PLUGINS_DIR`
: The path where the plugin archives are loaded and stored (default: `$NXF_HOME/plugins`).
`NXF_PLUGINS_REGISTRY_URL`
: :::{versionadded} 25.08.0-edge
:::
: Specifies the URL of the plugin registry used to download and resolve plugins. This allows using custom or private plugin registries instead of the default public registry.
`NXF_PLUGINS_TEST_REPOSITORY`
: :::{versionadded} 23.04.0
:::
: Defines a custom plugin registry or plugin release URL for testing plugins outside of the main registry.
`NXF_PUBLISH_FAIL_ON_ERROR`
: :::{versionadded} 24.04.3
:::
: Defines the default behavior of `publishDir.failOnError` setting. See {ref}`publishDir<process-publishdir>` directive for more information.
`NXF_RETRY_POLICY_DELAY`
: :::{versionadded} 25.06.0-edge
:::
: Delay used for HTTP retryable operations (default: `350ms`).
`NXF_RETRY_POLICY_JITTER`
: :::{versionadded} 25.06.0-edge
:::
: Jitter value used for HTTP retryable operations (default: `0.25`).
`NXF_RETRY_POLICY_MAX_ATTEMPTS`
: :::{versionadded} 25.06.0-edge
:::
: Max number of attempts used for HTTP retryable operations (default: `5`).
`NXF_RETRY_POLICY_MAX_DELAY`
: :::{versionadded} 25.06.0-edge
:::
: Max delay used for HTTP retryable operations (default: `90s`).
`NXF_RETRY_POLICY_MULTIPLIER`
: :::{versionadded} 25.08.0-edge
:::
: Delay multiplier used for HTTP retryable operations (default: `2.0`).
`NXF_SCM_FILE`
: :::{versionadded} 20.10.0
:::
: Defines the path location of the SCM config file .
`NXF_SINGULARITY_CACHEDIR`
: Directory where remote Singularity images are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
`NXF_SINGULARITY_LIBRARYDIR`
: :::{versionadded} 21.09.0-edge
:::
: Directory where remote Singularity images are retrieved. It should be a directory accessible to all compute nodes.
`NXF_SPACK_CACHEDIR`
: Directory where Spack environments are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
`NXF_SPACK_ENABLED`
: :::{versionadded} 23.02.0-edge
:::
: Enable the use of Spack recipes defined by using the {ref}`process-spack` directive. (default: `false`).
`NXF_SYNTAX_PARSER`
: :::{versionadded} 25.02.0-edge
:::
: Set to `'v2'` to use the {ref}`strict syntax <strict-syntax-page>` for Nextflow scripts and config files (default: `'v1'`).
`NXF_TEMP`
: Directory where temporary files are stored
`NXF_TRACE`
: Enable trace level logging for the specified packages. Equivalent to the `-trace` command-line option.
`NXF_VER`
: Defines which version of Nextflow to use.
`NXF_WORK`
: Directory where working files are stored (usually your *scratch* directory)
`NXF_WRAPPER_STAGE_FILE_THRESHOLD`
: :::{versionadded} 23.05.0-edge
:::
: Enables writing large staging scripts to a separate `.command.stage` file. The value defines the minimum size of the `.command.run` staging script for it to be written to the separate file (default when enabled: `'1 MB'`).
: This setting is useful for executors that impose a size limit on job scripts.
## Seqera Platform settings
`TOWER_ACCESS_TOKEN`
: Specifies the access token for authenticating with Seqera Platform. Can also be configured using the `tower.accessToken` config option.
`TOWER_API_ENDPOINT`
: Specifies the Seqera Platform API endpoint (default: `https://api.cloud.seqera.io`). Can also be configured using the `tower.endpoint` config option.
`TOWER_AUTH_DOMAIN`
: :::{versionadded} 25.10.0
:::
: Specifies the Auth0 domain for authenticating with Seqera Platform when connecting to a custom endpoint. When specified, this value takes precedence over the built-in mappings for known Seqera endpoints. Must be used in conjunction with `TOWER_AUTH_CLIENT_ID`.
`TOWER_AUTH_CLIENT_ID`
: :::{versionadded} 25.10.0
:::
: Specifies the Auth0 client ID for authenticating with a custom Seqera Platform endpoint. Must be used in conjunction with `TOWER_AUTH_DOMAIN`.
`TOWER_REFRESH_TOKEN`
: Specifies the refresh token for maintaining authentication with Seqera Platform. Can also be configured using the `tower.refreshToken` config option.
`TOWER_COMPUTE_ENV_ID`
: Specifies the Seqera Platform compute environment ID. When specified, the scheduler resolves the compute environment directly by this ID instead of inferring a suitable compute environment. Can also be configured using the `tower.computeEnvId` config option.
`TOWER_WORKSPACE_ID`
: Specifies the Seqera Platform workspace ID. Can also be configured using the `tower.workspaceId` config option.
## Other settings
`FTP_PROXY`
: :::{versionadded} 21.06.0-edge
:::
: Defines the FTP proxy server. Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `ftp://user:password@proxy-host.com:port`.
`HTTP_PROXY`
: Defines the HTTP proxy server.
: :::{versionadded} 21.06.0-edge
Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `http://user:password@proxy-host.com:port`.
:::
`HTTPS_PROXY`
: Defines the HTTPS proxy server.
: :::{versionadded} 21.06.0-edge
Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `https://user:password@proxy-host.com:port`.
:::
`NO_COLOR`
: Disables ANSI color codes in Nextflow log output. When this variable is set, Nextflow prints plain text logs following the [NO_COLOR standard](https://no-color.org/).
: If both `NO_COLOR` and `NXF_ANSI_LOG` are set, `NXF_ANSI_LOG` takes precedence.
`NO_PROXY`
: Defines one or more host names that should not use the proxy server. Separate multiple names using a comma character.
`TERMINAL_WIDTH`
: Forces the terminal width of ANSI-formatted log output. Overrides automatic terminal width detection and uses the specified width for line wrapping when set to an integer value.

View File

@@ -0,0 +1,56 @@
(config-feature-flags)=
# Feature flags
Feature flags enable opt-in features. They must be specified in the pipeline script.
:::{warning}
Feature flags marked as *preview* can cause pipelines run with newer versions of Nextflow to fail due to breaking changes. Always consult the {ref}`migration notes <migrations-page>` before updating to a new Nextflow version.
:::
`nextflow.enable.configProcessNamesValidation`
: :::{deprecated} 25.10.0
Use the {ref}`strict syntax <strict-syntax-page>` instead. It validates process selectors without producing false warnings.
:::
: When `true`, prints a warning for every `withName:` process selector that doesn't match a process in the pipeline (default: `true`).
`nextflow.enable.dsl`
: :::{deprecated} 25.04.0
:::
: Defines the DSL version to use (`1` or `2`).
`nextflow.enable.moduleBinaries`
: When `true`, enables the use of module-scoped executable scripts via {ref}`module resources <module-resources>`.
`nextflow.enable.strict`
: :::{deprecated} 26.04.0
:::
: When `true`, executes the pipeline in "strict" mode, which introduces the following rules:
- When reading a params file, Nextflow will fail if a dynamic param value references an undefined variable
- When merging params from a config file with params from the command line, Nextflow will fail if a param is specified from both sources but with different types
- When using the `join` operator, the `failOnDuplicate` option is `true` regardless of any user setting
- When using the `join` operator, the `failOnMismatch` option is `true` (unless `remainder` is also `true`) regardless of any user setting
- When using the `publishDir` process directive, the `failOnError` option is `true` regardless of any user setting
- In a process definition, Nextflow will fail if an input or output tuple has only one element
- In a process definition, Nextflow will fail if an output emit name is not a valid identifier (i.e. it should match the pattern `/[A-Za-z_][A-Za-z0-9_]*/`)
- During a process execution, Nextflow will fail if a received input tuple does not have the same number of elements as was declared
- During a process execution, Nextflow will fail if the `storeDir` directive is used with non-file outputs
- Nextflow will fail if a pipeline param is referenced before it is defined
- Nextflow will fail if multiple functions and/or processes with the same name are defined in a module script
`nextflow.enable.types`
: :::{versionadded} 26.04.0
:::
: *Preview feature: the syntax and behavior may change in future releases.*
: When `true`, enables the use of {ref}`typed processes <process-typed-page>` and {ref}`typed workflows <workflow-typed-page>`. Must be enabled in every script that uses typed processes/workflows. Legacy processes/workflows cannot be defined in scripts with this flag enabled.

View File

@@ -0,0 +1,350 @@
(operator-typed-page)=
# Operators (typed)
:::{versionadded} 26.04.0
:::
This page describes the core operators that are recommended for use with static typing.
## collect
**`Channel<E> collect() -> Value<Bag<E>>`**
The `collect` operator collects all values from a source channel into a collection and emits it as a dataflow value:
```{literalinclude} ../snippets/collect.nf
:language: nextflow
```
```{literalinclude} ../snippets/collect.out
:language: console
```
## combine
**`Channel<L> combine( other: Channel<R> ) -> Channel<Tuple>`**
**`Channel<L> combine( other: Value<R> ) -> Channel<Tuple>`**
The `combine` operator emits every pairwise combination of a source channel with another channel or dataflow value:
```{literalinclude} ../snippets/combine.nf
:language: nextflow
```
```{literalinclude} ../snippets/combine.out
:language: console
```
Tuples in both the left- and right-hand sources are flattened in the combined tuple. For example, `tuple(1, 2)` and `tuple('red', 'blue')` are combined as `tuple(1, 2, 'red', 'blue')`.
**`Channel<Record> combine( [opts] ) -> Channel<Record>`**
When the `combine` operator is called on a channel of records with named arguments, the named arguments are appended to each record in the source channel. Each named argument can be a value or dataflow value.
## filter
**`Channel<E> filter( condition: (E) -> Boolean ) -> Channel<E>`**
The `filter` operator emits the values from a source channel that satisfy a condition, discarding all other values:
```{literalinclude} ../snippets/filter-closure.nf
:language: nextflow
```
```{literalinclude} ../snippets/filter-closure.out
:language: console
```
## flatMap
**`Channel<E> flatMap( transform: (E) -> Iterable<R> ) -> Channel<R>`**
The `flatMap` operator applies a *mapping function* to each value from a source channel. The mapping function should return a collection, and each element in the collection is emitted separately.
For example:
```{literalinclude} ../snippets/flatmap-list.nf
:language: nextflow
```
```{literalinclude} ../snippets/flatmap-list.out
:language: console
```
(operator-groupby)=
## groupBy
**`Channel<Tuple<K, V>> groupBy() -> Channel<Tuple<K, Bag<V>>>`**
**`Channel<Tuple<K, Integer, V>> groupBy() -> Channel<Tuple<K, Bag<V>>>`**
The `groupBy` operator collects values from a source channel into groups based on a grouping key. A tuple is emitted for each group, containing the grouping key and collection of values.
The source channel should supply either 2-tuples of the form `(<key>, <value>)` or 3-tuples of the form `(<key>, <size>, <value>)`.
If the source tuples do not specify a size, `groupBy` will not emit any groups until *all* inputs have been received:
```{literalinclude} ../snippets/groupby.nf
:language: nextflow
```
```{literalinclude} ../snippets/groupby.out
:language: console
```
If the source tuples do specify a size, then `groupBy` will emit each group as soon as it is ready:
```{literalinclude} ../snippets/groupby-size.nf
:language: nextflow
```
```{literalinclude} ../snippets/groupby-size.out
:language: console
```
:::{note}
When specifying the group size, make sure that the number of inputs for a given group matches the specified size for that group. Otherwise, the run will fail.
:::
(operator-join-record)=
## join
**`Channel<Record> join( other: Channel<Record>, [opts] ) -> Channel<Record>`**
The `join` operator emits the relational join of two channels of records, using a matching key given by the `by` option:
```{literalinclude} ../snippets/join-record.nf
:language: nextflow
```
```{literalinclude} ../snippets/join-record.out
:language: console
```
Duplicate matching keys are handled by emitting each matching combination (like a relational join):
```{literalinclude} ../snippets/join-record-duplicates.nf
:language: nextflow
```
```{literalinclude} ../snippets/join-record-duplicates.out
:language: console
```
By default, unmatched values are discarded. The `remainder` option can be used to emit them at the end:
```{literalinclude} ../snippets/join-record-remainder.nf
:language: nextflow
```
```{literalinclude} ../snippets/join-record-remainder.out
:language: console
```
Available options:
`by: String`
: *This option is required.*
: The record field to use as the matching key.
`remainder: Boolean`
: When `true`, unmatched values are emitted at the end, otherwise they are discarded (default: `false`).
## map
**`Channel<E> map( transform: (E) -> R ) -> Channel<R>`**
The `map` operator applies a *mapping function* to each value from a source channel:
```{literalinclude} ../snippets/map.nf
:language: nextflow
```
```{literalinclude} ../snippets/map.out
:language: console
```
## mix
**`Channel<E> mix( other: Channel<E> ) -> Channel<E>`**
**`Channel<E> mix( other: Value<E> ) -> Channel<E>`**
The `mix` operator emits the values from a channel and another channel or dataflow value into a single output channel:
```{literalinclude} ../snippets/mix.nf
:language: nextflow
```
```{literalinclude} ../snippets/mix.out
:language: console
```
The values in the output channel may be emitted in any order, for example:
```console
z
1
a
2
b
3
```
## reduce
**`Channel<E> reduce( seed: R, accumulator: (R, E) -> R ) -> Value<R>`**
**`Channel<E> reduce( accumulator: (E, E) -> E ) -> Value<E>`**
The `reduce` operator applies an *accumulator function* sequentially to each value in a source channel, and emits the final accumulated value.
The accumulator function takes two parameters -- the accumulated value and the *i*-th emitted value -- and it should return the accumulated result, which is passed to the next invocation with the *i+1*-th value. This process is repeated for each value in the source channel.
For example:
```{literalinclude} ../snippets/reduce.nf
:language: nextflow
```
```{literalinclude} ../snippets/reduce.out
:language: console
```
By default, the first value is used as the initial accumulated value (the *seed*). You can optionally specify a different initial value as shown below:
```{literalinclude} ../snippets/reduce-with-initial-value.nf
:language: nextflow
```
```{literalinclude} ../snippets/reduce-with-initial-value.out
:language: console
```
## subscribe
**`Channel<E> subscribe( action: (E) -> () )`**
**`Channel<E> subscribe( [opts] )`**
The `subscribe` operator invokes a custom function for each value from a source channel:
```{literalinclude} ../snippets/subscribe.nf
:language: nextflow
```
```{literalinclude} ../snippets/subscribe.out
:language: console
```
The `subscribe` operator supports multiple types of event handlers:
```{literalinclude} ../snippets/subscribe-with-on-complete.nf
:language: nextflow
```
```{literalinclude} ../snippets/subscribe-with-on-complete.out
:language: console
```
:::{note}
Unlike most operators, `subscribe` does not return anything. It should only be used for *side effects*, such as printing to the console, writing to a file, or making HTTP requests.
:::
Available options:
`onNext: (E) -> ()`
: Closure that is invoked when an value is emitted. Equivalent to providing a single closure argument.
`onComplete: () -> ()`
: Closure that is invoked after the last value is emitted by the channel.
`onError: (T) -> ()`
: Closure that is invoked when an exception is raised while handling the `onNext` event. It will not make further calls to `onNext` or `onComplete`. The `onError` method takes as its parameter the `Throwable` that caused the error.
## unique
**`Channel<E> unique( transform: (E) -> ? ) -> Channel<E>`**
**`Channel<E> unique() -> Channel<E>`**
The `unique` operator emits the unique values from a source channel:
```{literalinclude} ../snippets/unique.nf
:language: nextflow
```
```{literalinclude} ../snippets/unique.out
:language: console
```
An optional closure can be used to transform each value before it is evaluated for uniqueness:
```{literalinclude} ../snippets/unique-with-mapper.nf
:language: nextflow
```
```{literalinclude} ../snippets/unique-with-mapper.out
:language: console
```
## until
**`Channel<E> until( condition: (E) -> Boolean ) -> Channel<E>`**
The `until` operator emits each value from a source channel until a stopping condition is satisfied:
```{literalinclude} ../snippets/until.nf
:language: nextflow
```
```{literalinclude} ../snippets/until.out
:language: console
```
## view
**`Channel<E> view( transform: (E) -> String, [opts] ) -> Channel<E>`**
**`Channel<E> view( [opts] ) -> Channel<E>`**
The `view` operator prints each value from a source channel to standard output:
```{literalinclude} ../snippets/view.nf
:language: nextflow
```
```{literalinclude} ../snippets/view.out
:language: console
```
An optional closure can be used to transform each value before it is printed:
```{literalinclude} ../snippets/view-with-mapper.nf
:language: nextflow
```
```{literalinclude} ../snippets/view-with-mapper.out
:language: console
```
The `tag` option can be used to print the channel only when the `-dump-channels` command-line option is specified with the given tag:
```{literalinclude} ../snippets/view-tag.nf
:language: nextflow
```
You can run this script with `-dump-channels plus1` or `-dump-channels exp2` to print either channel, or `-dump-channels plus1,exp2` to print both.
The `view` operator also emits every value that it receives, allowing it to be chained with other operators.
Available options:
`newLine: Boolean`
: Print each value to a separate line (default: `true`).
`tag: String`
: Print the channel values only when `-dump-channels` is specified on the command line with the given tag.

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,25 @@
(stdlib-groovy)=
# Groovy and Java classes
Any Groovy or Java class that is available to Nextflow at runtime can be used in a Nextflow script. The following classes are imported by default:
- `groovy.lang.*`
- `groovy.util.*`
- `java.io.*`
- `java.lang.*`
- `java.math.BigDecimal`
- `java.math.BigInteger`
- `java.net.*`
- `java.util.*`
All other classes must be referenced by their fully-qualified name:
```nextflow
def vals = [1, 2, 3]
println groovy.json.JsonOutput.toJson(vals)
```
:::{note}
The set of classes in Nextflow's runtime classpath can change between different Nextflow versions. As a best practice, any code that uses classes outside the Nextflow standard library should either be refactored to only use the Nextflow standard library or be refactored as a {ref}`plugin <dev-plugins-page>` with explicit dependencies.
:::

View File

@@ -0,0 +1,289 @@
(stdlib-namespaces)=
# Namespaces
This page lists all of the available namespaces in the Nextflow standard library.
(stdlib-namespaces-global)=
## Global namespace
The global namespace contains globally available constants and functions.
**Constants**
`baseDir: Path`
: :::{deprecated} 20.04.0
:::
: Alias of `workflow.projectDir`.
`launchDir: Path`
: Alias of `workflow.launchDir`.
`moduleDir: Path`
: Directory where a module script is located (equivalent to `projectDir` if used in the main script).
`params`
: Map of workflow parameters specified in the config file or as command line options.
`projectDir: Path`
: Alias of `workflow.projectDir`.
`secrets: Map<String,String>`
: :::{versionadded} 24.02.0-edge
:::
: Map of pipeline secrets. See {ref}`secrets-page` for more information.
`workDir: Path`
: Alias of `workflow.workDir`.
**Functions**
`branchCriteria( criteria: Closure ) -> Closure`
: Create a branch criteria to use with the {ref}`operator-branch` operator.
`env( name: String ) -> String`
: :::{versionadded} 25.04.0
:::
: Get the value of the environment variable with the specified name in the Nextflow launch environment.
`error( message: String = null )`
: Throw a script runtime error with an optional error message.
`exit( exitCode: int = 0, message: String = null )`
: :::{deprecated} 22.06.0-edge
Use `error()` instead
:::
: Stop the pipeline execution and return an exit code and optional error message.
`file( filePattern: String, [options] ) -> Path`
: Get a file from a file name or glob pattern.
: The following options are available:
`checkIfExists: boolean`
: When `true`, throws an exception if the specified path does not exist in the file system (default: `false`)
`followLinks: boolean`
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`)
`glob: boolean`
: When `true`, interprets characters `*`, `?`, `[]` and `{}` as glob wildcards, otherwise handles them as normal characters (default: `true`)
`hidden: boolean`
: When `true`, includes hidden files in the resulting paths (default: `false`)
`maxDepth: int`
: Maximum number of directory levels to visit (default: *no limit*)
`type: String`
: Type of paths returned, can be `'file'`, `'dir'` or `'any'` (default: `'file'`)
: :::{note}
This function returns a collection if the glob pattern yields zero or multiple files. Use `files()` to get a collection of files.
:::
`files( filePattern: String, [options] ) -> Iterable<Path>`
: Get a collection of files from a file name or glob pattern. Supports the same options as `file()`.
: See also: {ref}`channel.fromPath <channel-path>`.
`groupKey( key, size: int ) -> GroupKey`
: Create a grouping key to use with the {ref}`operator-grouptuple` operator.
`multiMapCriteria( criteria: Closure ) -> Closure`
: Create a multi-map criteria to use with the {ref}`operator-multiMap` operator.
`print( value )`
: Print a value to standard output.
`printf( format: String, values... )`
: Print a formatted string with the given values to standard output.
`println( value )`
: Print a value to standard output with a newline.
`sendMail( [options] )`
: Send an email. See {ref}`mail-page` for more information.
`sleep( milliseconds: long )`
: Sleep for the given number of milliseconds.
`record( [options] ) -> Record`
: Create a record from the given named arguments.
`tuple( args... ) -> Tuple`
: Create a tuple from the given arguments.
(stdlib-namespaces-channel)=
## `channel`
The `channel` namespace contains the built-in channel factories. See {ref}`channel-factory` for details.
(stdlib-namespaces-nextflow)=
## `log`
The `log` namepsace contains functions for logging messages to the console.
`error( message: String )`
: Log an error message to the console.
: This function does not terminate the pipeline -- use the global `error()` function instead.
`info( message: String )`
: Log an info message to the console.
`warn( message: String )`
: Log a warning message to the console.
## `nextflow`
The `nextflow` namespace contains information about the current Nextflow runtime.
`build: int`
: Nextflow runtime build number.
`timestamp: String`
: Nextflow runtime compile timestamp.
`version: VersionNumber`
: Nextflow runtime version number. See {ref}`VersionNumber <stdlib-types-versionnumber>` for more information.
(stdlib-namespaces-workflow)=
## `workflow`
The `workflow` namespace contains information about the current workflow run.
**Properties**
`commandLine: String`
: Command line as entered by the user to launch the workflow execution.
`commitId: String`
: Git commit ID of the executed workflow repository.
: When providing a Git tag, branch name, or commit hash using the `-r` CLI option, the associated `workflow.commitId` is also populated.
`complete: OffsetDateTime`
: *Available only in the `workflow.onComplete` handler*
: Timestamp of workflow when execution is completed.
`configFiles: List<Path>`
: Configuration files used for the workflow execution.
`container: String | Map<String,String>`
: Docker image used to run workflow tasks, or a map of process names to process containers when multiple images are used.
`containerEngine: String`
: Returns the name of the container engine (e.g. docker or singularity) or null if no container engine is enabled.
`duration: Duration`
: *Available only in the `workflow.onComplete` handler*
: Time elapsed to complete workflow execution.
`errorMessage: String`
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
: Error message of the task that caused the workflow execution to fail.
`errorReport: String`
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
: Detailed error of the task that caused the workflow execution to fail.
`exitStatus: int`
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
: Exit status of the task that caused the workflow execution to fail.
`failOnIgnore: boolean`
: :::{versionadded} 24.05.0-edge
:::
: Whether the `workflow.failOnIgnore` config option was enabled.
: See also: {ref}`process-error-strategy`
`fusion`
: Namespace containing information about the current Fusion runtime. The following properties are available:
: `enabled: boolean`
: Whether Fusion is enabled.
: `version: String`
: The Fusion version being used.
`homeDir: Path`
: User system home directory.
`launchDir: Path`
: Directory where the workflow was launched.
`manifest`
: Namespace corresponding to the {ref}`config-manifest` config scope.
`outputDir: Path`
: :::{versionadded} 24.10.0
:::
: Workflow output directory.
`preview: boolean`
: :::{versionadded} 24.04.0
:::
: Whether the current workflow run is a preview run.
`profile: String`
: Comma-separated list of active configuration profiles.
`projectDir: Path`
: Directory where the workflow project is located.
`repository: String`
: Project repository Git remote URL.
`resume: boolean`
: Returns `true` whenever the current instance is resumed from a previous execution.
`revision: String`
: Git branch/tag of the executed workflow repository.
: When providing a Git tag or branch name using the `-r` CLI option, the `workflow.revision` is also populated.
`runName: String`
: Mnemonic name assigned to this execution instance.
`scriptFile: Path`
: Project main script file path.
`scriptId: String`
: Project main script unique hash ID.
`scriptName: String`
: Project main script file name.
`sessionId: UUID`
: Unique identifier (UUID) associated to current execution.
`start: OffsetDateTime`
: Timestamp of workflow at execution start.
`stubRun: boolean`
: Returns `true` whenever the current instance is a stub-run execution .
`success: boolean`
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
: Reports if the execution completed successfully.
`userName: String`
: User system account name.
`wave`
: Namespace containing Wave runtime information. The following properties are available:
: `enabled: boolean`
: Whether Wave is enabled.
`workDir: Path`
: The directory where task temporary files are stored.
**Functions**
`onComplete( action: Closure )`
: Define an action to take when the workflow completes (whether successful or not).
`onError( action: Closure )`
: Define an action to take if the workflow is terminated due to a runtime error or task failure.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,12 @@
(stdlib-page)=
# Standard library
This section describes the Nextflow standard library, which consists of built-in namespaces and types.
```{toctree}
:maxdepth: 1
stdlib-namespaces
stdlib-types
stdlib-groovy
```

File diff suppressed because it is too large Load Diff