add nextflow d30e48d
This commit is contained in:
454
nextflow/docs/reference/channel.md
Normal file
454
nextflow/docs/reference/channel.md
Normal file
@@ -0,0 +1,454 @@
|
||||
(channel-factory)=
|
||||
|
||||
# Channel factories
|
||||
|
||||
This page describes the channel factories that are available in the `channel` namespace.
|
||||
|
||||
(channel-empty)=
|
||||
|
||||
## empty
|
||||
|
||||
**`empty() -> Channel`**
|
||||
|
||||
The `channel.empty` factory creates a channel that emits nothing:
|
||||
|
||||
```nextflow
|
||||
channel.empty().view() // prints nothing
|
||||
```
|
||||
|
||||
(channel-filepairs)=
|
||||
|
||||
## fromFilePairs
|
||||
|
||||
:::{note}
|
||||
As a best practice, use a samplesheet instead of matching file pairs directly with glob patterns. See {ref}`static-types-samplesheet` for more information.
|
||||
:::
|
||||
|
||||
The `channel.fromFilePairs` factory creates a channel that emits file pairs matching a [glob][glob] pattern:
|
||||
|
||||
```nextflow
|
||||
ch = channel.fromFilePairs('/my/data/SRR*_{1,2}.fastq')
|
||||
ch.view()
|
||||
```
|
||||
|
||||
Each file pair is emitted as a 2-tuple containing the grouping key (from the `*` wildcard) and the list of files (sorted lexicographically):
|
||||
|
||||
```
|
||||
[SRR493366, [/my/data/SRR493366_1.fastq, /my/data/SRR493366_2.fastq]]
|
||||
[SRR493367, [/my/data/SRR493367_1.fastq, /my/data/SRR493367_2.fastq]]
|
||||
[SRR493368, [/my/data/SRR493368_1.fastq, /my/data/SRR493368_2.fastq]]
|
||||
[SRR493369, [/my/data/SRR493369_1.fastq, /my/data/SRR493369_2.fastq]]
|
||||
[SRR493370, [/my/data/SRR493370_1.fastq, /my/data/SRR493370_2.fastq]]
|
||||
[SRR493371, [/my/data/SRR493371_1.fastq, /my/data/SRR493371_2.fastq]]
|
||||
```
|
||||
|
||||
The glob pattern must contain at least one `*` wildcard character.
|
||||
|
||||
Available options:
|
||||
|
||||
`checkIfExists`
|
||||
: When `true`, throws an error if the file path does not exist in the file system (default: `false`).
|
||||
|
||||
`flat`
|
||||
: When `true`, tuples are emitted with the matching files flattened instead of as a nested list (default: `false`).
|
||||
|
||||
`followLinks`
|
||||
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`).
|
||||
|
||||
`hidden`
|
||||
: When `true`, matches hidden files when using a glob pattern (default: `false`).
|
||||
|
||||
`maxDepth`
|
||||
: Maximum number of directory levels to visit with the `**` wildcard (default: no limit).
|
||||
|
||||
`size`
|
||||
: The number of expected files for each file pair (default: `2`). Set to `-1` to allow any size.
|
||||
|
||||
`type`
|
||||
: Whether to return only files (`'file'`), only directories (`'dir'`), or both (`'any'`) when using a glob pattern. By default, only files are returned (`'file'`).
|
||||
|
||||
(channel-from-lineage)=
|
||||
|
||||
## fromLineage
|
||||
|
||||
:::{versionadded} 25.04.0
|
||||
:::
|
||||
|
||||
:::{warning} *Experimental: may change in a future release.*
|
||||
:::
|
||||
|
||||
**`fromLineage( [opts] ) -> Channel<Path>`**
|
||||
|
||||
The `channel.fromLineage` factory creates a channel that emits files from the {ref}`cli-lineage` store that match the given key-value params:
|
||||
|
||||
```nextflow
|
||||
ch = channel.fromLineage(
|
||||
workflowRun: 'lid://0d1d1622ced3e4edc690bec768919b45',
|
||||
label: ['alpha', 'beta']
|
||||
)
|
||||
ch.view()
|
||||
```
|
||||
|
||||
The above snippet emits files published by the given workflow run that are labeled as `alpha` and `beta`.
|
||||
|
||||
Available options:
|
||||
|
||||
`label`
|
||||
: List of labels associated with the desired files.
|
||||
|
||||
`taskRun`
|
||||
: LID of the task run that produced the desired files.
|
||||
|
||||
`workflowRun`
|
||||
: LID of the workflow run that produced the desired files.
|
||||
|
||||
(channel-fromlist)=
|
||||
|
||||
## fromList
|
||||
|
||||
**`fromList( values: Iterable<E> ) -> Channel<E>`**
|
||||
|
||||
The `channel.fromList` factory creates a channel that emits each element in a collection:
|
||||
|
||||
```nextflow
|
||||
ch = channel.fromList( ['a', 'b', 'c', 'd'] )
|
||||
ch.view { v -> "value: $v" }
|
||||
```
|
||||
|
||||
Prints:
|
||||
|
||||
```
|
||||
value: a
|
||||
value: b
|
||||
value: c
|
||||
value: d
|
||||
```
|
||||
|
||||
See also: [channel.of](#of)
|
||||
|
||||
(channel-path)=
|
||||
|
||||
## fromPath
|
||||
|
||||
**`fromPath( pattern: String, [opts] ) -> Channel<Path>`**
|
||||
|
||||
The `channel.fromPath` factory creates a channel that emits file paths matching a name or [glob][glob] pattern:
|
||||
|
||||
```nextflow
|
||||
// match single file
|
||||
channel.fromPath('data/some/bigfile.txt')
|
||||
|
||||
// match `txt` files in `data/bag`
|
||||
channel.fromPath('data/big/*.txt')
|
||||
|
||||
// match `fa` files in `data` and its subdirectories
|
||||
channel.fromPath('data/**.fa')
|
||||
|
||||
// match `fa` files with same suffix in any subdirectory of `data`
|
||||
channel.fromPath('data/**/*.fa')
|
||||
|
||||
// match file pair (`file_1.fq` and `file_2.fq`)
|
||||
channel.fromPath('data/file_{1,2}.fq')
|
||||
```
|
||||
|
||||
By default, glob patterns do not match hidden files (i.e. files with names that start with `.`). Use a glob pattern that explicitly starts with `.` or set `hidden: true` to match hidden files:
|
||||
|
||||
```nextflow
|
||||
// match hidden files in `data`
|
||||
channel.fromPath('data/.*')
|
||||
channel.fromPath('data/*', hidden: true)
|
||||
|
||||
// match hidden files in `data` with `fa` extension
|
||||
channel.fromPath('data/.*.fa')
|
||||
```
|
||||
|
||||
By default, glob patterns only match regular files, not directories. Use the `type` option to control whether to match files, directories, or both:
|
||||
|
||||
```nextflow
|
||||
// match only directories
|
||||
channel.fromPath('data/*', type: 'dir')
|
||||
|
||||
// match files and directories
|
||||
channel.fromPath('data/*', type: 'any')
|
||||
```
|
||||
|
||||
Available options:
|
||||
|
||||
`checkIfExists`
|
||||
: When `true`, throws an error if the file path does not exist in the file system (default: `false`).
|
||||
|
||||
`followLinks`
|
||||
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`).
|
||||
|
||||
`glob`
|
||||
: When `true`, interprets the characters `*`, `?`, `[]`, and `{}` as glob wildcards, otherwise treats them as normal characters (default: `true`).
|
||||
|
||||
`hidden`
|
||||
: When `true`, matches hidden files when using a glob pattern (default: `false`).
|
||||
|
||||
`maxDepth`
|
||||
: Maximum number of directory levels to visit with the `**` wildcard (default: no limit).
|
||||
|
||||
`relative`
|
||||
: When `true`, returns file paths as relative to the top-most common directory (default: `false`).
|
||||
|
||||
`type`
|
||||
: Whether to return only files (`'file'`), only directories (`'dir'`), or both (`'any'`) when using a glob pattern. By default, only files are returned (`'file'`).
|
||||
|
||||
(channel-fromsra)=
|
||||
|
||||
## fromSRA
|
||||
|
||||
:::{deprecated} 26.04.0
|
||||
Use the [Entrez Direct](https://www.ncbi.nlm.nih.gov/books/NBK179288/) command-line tool to query the SRA database.
|
||||
:::
|
||||
|
||||
The `channel.fromSRA` factory queries the [NCBI SRA](https://www.ncbi.nlm.nih.gov/sra) database and returns a channel emitting the FASTQ files matching the specified criteria i.e project or accession number(s). For example:
|
||||
|
||||
```nextflow
|
||||
channel.fromSRA('SRP043510').view()
|
||||
```
|
||||
|
||||
It returns:
|
||||
|
||||
```
|
||||
[SRR1448794, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/004/SRR1448794/SRR1448794.fastq.gz]
|
||||
[SRR1448795, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/005/SRR1448795/SRR1448795.fastq.gz]
|
||||
[SRR1448792, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/002/SRR1448792/SRR1448792.fastq.gz]
|
||||
[SRR1448793, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR144/003/SRR1448793/SRR1448793.fastq.gz]
|
||||
[SRR1910483, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR191/003/SRR1910483/SRR1910483.fastq.gz]
|
||||
[SRR1910482, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR191/002/SRR1910482/SRR1910482.fastq.gz]
|
||||
(remaining omitted)
|
||||
```
|
||||
|
||||
Multiple accession IDs can be specified as a list:
|
||||
|
||||
```nextflow
|
||||
ids = ['ERR908507', 'ERR908506', 'ERR908505']
|
||||
channel.fromSRA(ids).view()
|
||||
```
|
||||
|
||||
```
|
||||
[ERR908507, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908507/ERR908507_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908507/ERR908507_2.fastq.gz]]
|
||||
[ERR908506, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908506/ERR908506_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908506/ERR908506_2.fastq.gz]]
|
||||
[ERR908505, [ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908505/ERR908505_1.fastq.gz, ftp://ftp.sra.ebi.ac.uk/vol1/fastq/ERR908/ERR908505/ERR908505_2.fastq.gz]]
|
||||
```
|
||||
|
||||
Each read pair is implicitly managed and returned as a list of files.
|
||||
|
||||
This method uses the NCBI [ESearch](https://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch) API behind the scenes, therefore it allows the use of any query term supported by this API.
|
||||
|
||||
To access the ESearch API, you must provide your [NCBI API keys](https://ncbiinsights.ncbi.nlm.nih.gov/2017/11/02/new-api-keys-for-the-e-utilities) through one of the following ways:
|
||||
|
||||
- The `apiKey` option:
|
||||
```nextflow
|
||||
channel.fromSRA(ids, apiKey:'0123456789abcdef')
|
||||
```
|
||||
|
||||
- The `NCBI_API_KEY` variable in your environment:
|
||||
```bash
|
||||
export NCBI_API_KEY=0123456789abcdef
|
||||
```
|
||||
|
||||
Available options:
|
||||
|
||||
`apiKey`
|
||||
: NCBI user API key.
|
||||
|
||||
`cache`
|
||||
: Enable/disable the caching API requests (default: `true`).
|
||||
|
||||
`max`
|
||||
: Maximum number of entries that can be retried (default: unlimited) .
|
||||
|
||||
`protocol`
|
||||
: Allow choosing the protocol for the resulting remote URLs. Available choices: `ftp`, `http`, `https` (default: `ftp`).
|
||||
|
||||
`retryPolicy`
|
||||
: Set a retry policy in case the SRA request fails with a retriable error.
|
||||
|
||||
: Available properties:
|
||||
|
||||
- `delay`: Delay between attempts (default: `500ms`)
|
||||
- `jitter`: Jitter value (default: `0.25`)
|
||||
- `maxAttempts`: Max attempts (default: `3`)
|
||||
- `maxDelay`: Max delay (default: `30s`)
|
||||
|
||||
: For example:
|
||||
|
||||
```nextflow
|
||||
channel.fromSRA(ids, retryPolicy: [delay: '250ms', maxAttempts: 5])
|
||||
```
|
||||
|
||||
(channel-interval)=
|
||||
|
||||
## interval
|
||||
|
||||
**`interval( interval: String ) -> Channel<Integer>`**
|
||||
|
||||
The `channel.interval` factory emits an incrementing index (starting from zero) at a periodic interval. For example:
|
||||
|
||||
```nextflow
|
||||
channel.interval('1s').view()
|
||||
```
|
||||
|
||||
The above snippet will emit 0, 1, 2, and so on, every second, forever. You can use an operator such as {ref}`operator-take` or {ref}`operator-until` to close the channel based on a stopping condition.
|
||||
|
||||
(channel-of)=
|
||||
|
||||
## of
|
||||
|
||||
**`of( values... ) -> Channel`**
|
||||
|
||||
The `channel.of` factory allows you to create a channel that emits each argument:
|
||||
|
||||
```nextflow
|
||||
ch = channel.of( 1, 3, 5, 7 ).view()
|
||||
```
|
||||
|
||||
Prints:
|
||||
|
||||
```
|
||||
1
|
||||
3
|
||||
5
|
||||
7
|
||||
```
|
||||
|
||||
Ranges of values are expanded accordingly:
|
||||
|
||||
```nextflow
|
||||
channel.of(1..23, 'X', 'Y').view()
|
||||
```
|
||||
|
||||
Prints:
|
||||
|
||||
```
|
||||
1
|
||||
2
|
||||
3
|
||||
4
|
||||
:
|
||||
23
|
||||
X
|
||||
Y
|
||||
```
|
||||
|
||||
See also: [channel.fromList](#fromlist)
|
||||
|
||||
(channel-topic)=
|
||||
|
||||
## topic
|
||||
|
||||
:::{versionadded} 25.04.0
|
||||
:::
|
||||
|
||||
:::{note}
|
||||
This feature was previewed in versions 24.04 and 24.10 with the `nextflow.preview.topic` feature flag.
|
||||
:::
|
||||
|
||||
**`topic( name: String ) -> Channel`**
|
||||
|
||||
A *topic channel* is a channel that can receive values from many sources *implicitly* based on a matching *topic name*.
|
||||
|
||||
A typed process can emit values to a topic using the `topic:` section:
|
||||
|
||||
```nextflow
|
||||
nextflow.enable.types = true
|
||||
|
||||
process hello {
|
||||
topic:
|
||||
file('hello.txt') >> 'my-topic'
|
||||
|
||||
// ...
|
||||
}
|
||||
|
||||
process bye {
|
||||
topic:
|
||||
file('bye.txt') >> 'my-topic'
|
||||
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
A legacy process can assign outputs in the `output:` section to a topic using the `topic` option:
|
||||
|
||||
```nextflow
|
||||
process hello {
|
||||
output:
|
||||
path('hello.txt'), topic: 'my-topic'
|
||||
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
The `channel.topic` factory returns the topic channel for the given name:
|
||||
|
||||
```nextflow
|
||||
channel.topic('my-topic').view()
|
||||
```
|
||||
|
||||
The above example emits all values sent to the `my-topic` topic from processes such as `hello` and `bye`.
|
||||
|
||||
This approach is a convenient way to collect related items from many different sources without explicitly connecting them (e.g. using the `mix` operator).
|
||||
|
||||
:::{warning}
|
||||
Any process that consumes a topic channel (directly or indirectly) should not send any outputs to that topic, or else the pipeline will hang forever.
|
||||
:::
|
||||
|
||||
See also: {ref}`process-typed-topics` for process outputs
|
||||
|
||||
(channel-value)=
|
||||
|
||||
## value
|
||||
|
||||
**`value( value: V ) -> Value<V>`**
|
||||
|
||||
The `channel.value` factory creates a dataflow value bound to the given argument:
|
||||
|
||||
```nextflow
|
||||
channel.value( 'Hello there' )
|
||||
channel.value( [1,2,3,4,5] )
|
||||
```
|
||||
|
||||
The first line creates a dataflow value bound to the string `'Hello there'`. The second line creates a dataflow value bound to the list `[1,2,3,4,5]`, which is treated as a single value in dataflow logic.
|
||||
|
||||
(channel-watchpath)=
|
||||
|
||||
## watchPath
|
||||
|
||||
**`watchPath( pattern: String, events: String = 'create' ) -> Channel<Path>`**
|
||||
|
||||
The `channel.watchPath` factory creates a channel that watches a [glob][glob] pattern and emits matching files as they appear.
|
||||
|
||||
For example:
|
||||
|
||||
```nextflow
|
||||
ch = channel.watchPath('/path/*.fa')
|
||||
ch.view { fa -> "Fasta file: $fa" }
|
||||
```
|
||||
|
||||
The second argument specifies which filesystem events to watch as a comma-separated string:
|
||||
|
||||
```nextflow
|
||||
ch = channel.watchPath('/path/*.fa', 'create,modify')
|
||||
ch.view { fa -> "File created or modified: $fa" }
|
||||
```
|
||||
|
||||
By default, only new files are watched. The following events are supported:
|
||||
|
||||
- `create`: A new file is created
|
||||
- `modify`: A file is modified
|
||||
- `delete`: A file is deleted
|
||||
|
||||
:::{warning}
|
||||
The `channel.watchPath` factory waits endlessly for matching files, which means that it will cause your pipeline to run forever. Consider using the `take` or `until` operator to apply a stopping condition (e.g. receiving 10 files, receiving a file named `DONE`).
|
||||
:::
|
||||
|
||||
:::{note}
|
||||
The `channel.watchPath` factory only works with local and shared filesystems. It does not support object storage such as S3.
|
||||
:::
|
||||
|
||||
See also: [channel.fromPath](#frompath)
|
||||
|
||||
[glob]: http://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob
|
||||
1915
nextflow/docs/reference/cli.md
Normal file
1915
nextflow/docs/reference/cli.md
Normal file
File diff suppressed because it is too large
Load Diff
2032
nextflow/docs/reference/config.md
Normal file
2032
nextflow/docs/reference/config.md
Normal file
File diff suppressed because it is too large
Load Diff
317
nextflow/docs/reference/env-vars.md
Normal file
317
nextflow/docs/reference/env-vars.md
Normal file
@@ -0,0 +1,317 @@
|
||||
(config-env-vars)=
|
||||
|
||||
# Environment variables
|
||||
|
||||
The following environment variables control the configuration of the Nextflow runtime and the underlying Java virtual machine.
|
||||
|
||||
## Java settings
|
||||
|
||||
`JAVA_CMD`
|
||||
: Defines the path location of the Java binary command used to launch Nextflow.
|
||||
|
||||
`JAVA_HOME`
|
||||
: Defines the path location of the Java VM installation used to run Nextflow.
|
||||
|
||||
(nxf-env-vars)=
|
||||
|
||||
## Nextflow settings
|
||||
|
||||
`NXF_AGENT_MODE`
|
||||
: :::{versionadded} 26.04.0
|
||||
:::
|
||||
: When `true`, enables agent output mode. In this mode, Nextflow replaces the interactive ANSI log with minimal, structured output optimized for AI agents and non-interactive environments. The output uses tagged lines such as `[PIPELINE]`, `[PROCESS]`, `[WARN]`, `[ERROR]`, and `[SUCCESS]`/`[FAILED]` written to standard output (default: `false`).
|
||||
|
||||
`NXF_ANSI_LOG`
|
||||
: Enables/disables ANSI console output (default `true` when ANSI terminal is detected).
|
||||
|
||||
`NXF_ANSI_SUMMARY`
|
||||
: Enables/disables ANSI completion summary: `true\|false` (default: print summary if execution last more than 1 minute).
|
||||
|
||||
`NXF_ASSETS`
|
||||
: Defines the directory where downloaded pipeline repositories are stored (default: `$NXF_HOME/assets`)
|
||||
|
||||
`NXF_CACHE_DIR`
|
||||
: :::{versionadded} 24.02.0-edge
|
||||
:::
|
||||
: Defines the base cache directory when using the default cache store (default: `"$launchDir/.nextflow"`).
|
||||
|
||||
`NXF_CHARLIECLOUD_CACHEDIR`
|
||||
: Directory where remote Charliecloud images are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
|
||||
|
||||
`NXF_CLOUDCACHE_PATH`
|
||||
: :::{versionadded} 23.07.0-edge
|
||||
:::
|
||||
: Defines the base cache path when using the cloud cache store.
|
||||
|
||||
`NXF_CLOUD_DRIVER`
|
||||
: Defines the default cloud driver to be used if not specified in the config file or as command line option, either `aws` or `google`.
|
||||
|
||||
`NXF_CONDA_CACHEDIR`
|
||||
: Directory where Conda environments are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
|
||||
|
||||
`NXF_CONDA_ENABLED`
|
||||
: :::{versionadded} 22.08.0-edge
|
||||
:::
|
||||
: Enable the use of Conda recipes defined by using the {ref}`process-conda` directive. (default: `false`).
|
||||
|
||||
`NXF_CONTAINER_ENTRYPOINT_OVERRIDE`
|
||||
: :::{deprecated} 22.10.0
|
||||
:::
|
||||
: When `true`, override the container entrypoint with `/bin/bash` (default: `false`).
|
||||
|
||||
`NXF_DATE_FORMAT`
|
||||
: :::{versionadded} 25.07.0-edge
|
||||
:::
|
||||
: Defines the format for date and time representations in notifications and reports. Supports custom formats (e.g., `yyyy-MM-dd HH:mm:ss`) or `iso` for ISO 8601 format with timezone (default: `dd-MMM-yyyy HH:mm:ss`).
|
||||
|
||||
`NXF_DEFAULT_DSL`
|
||||
: :::{versionadded} 22.03.0-edge
|
||||
:::
|
||||
: Defines the DSL version that should be used in not specified otherwise in the script of config file (default: `2`)
|
||||
|
||||
`NXF_DISABLE_CHECK_LATEST`
|
||||
: :::{versionadded} 23.09.0-edge
|
||||
:::
|
||||
: Nextflow automatically checks for a newer version of itself unless this option is enabled (default: `false`).
|
||||
|
||||
`NXF_DISABLE_JOBS_CANCELLATION`
|
||||
: :::{versionadded} 21.12.0-edge
|
||||
:::
|
||||
: Disables the cancellation of child jobs on workflow execution termination.
|
||||
|
||||
`NXF_DISABLE_PARAMS_TYPE_DETECTION`
|
||||
: :::{versionadded} 23.07.0-edge
|
||||
:::
|
||||
: Disables the automatic type detection of command line parameters.
|
||||
|
||||
`NXF_DISABLE_WAVE_SERVICE`
|
||||
: :::{versionadded} 23.08.0-edge
|
||||
:::
|
||||
: Disables the requirement for Wave service when enabling the Fusion file system.
|
||||
|
||||
`NXF_ENABLE_AWS_SES`
|
||||
: :::{versionadded} 23.06.0-edge
|
||||
:::
|
||||
: Enable to use of AWS SES native API for sending emails in place of legacy SMTP settings (default: `false`)
|
||||
|
||||
`NXF_ENABLE_FS_SYNC`
|
||||
: :::{versionadded} 23.10.0
|
||||
:::
|
||||
: When enabled the job script will execute Linux `sync` command on job completion. This may be useful to synchronize the job state over shared file systems (default: `false`)
|
||||
|
||||
`NXF_ENABLE_SECRETS`
|
||||
: :::{versionadded} 21.09.0-edge
|
||||
:::
|
||||
: Enable Nextflow secrets features (default: `true`)
|
||||
|
||||
`NXF_ENABLE_STRICT`
|
||||
: :::{versionadded} 22.05.0-edge
|
||||
:::
|
||||
: Enable Nextflow *strict* execution mode (default: `false`)
|
||||
|
||||
`NXF_ENABLE_VIRTUAL_THREADS`
|
||||
: :::{versionadded} 23.05.0-edge
|
||||
:::
|
||||
: Enable the use of virtual threads in the Nextflow runtime (default: `false`)
|
||||
|
||||
`NXF_EXECUTOR`
|
||||
: Defines the default process executor, e.g. `sge`
|
||||
|
||||
`NXF_FILE_ROOT`
|
||||
: :::{versionadded} 23.05.0-edge
|
||||
:::
|
||||
: The file storage path against which relative file paths are resolved.
|
||||
: For example, with `NXF_FILE_ROOT=/some/root/path`, the use of `file('hello')` will be resolved to the absolute path `/some/root/path/hello`. A remote root path can be specified using the usual protocol prefix, e.g. `NXF_FILE_ROOT=s3://my-bucket/data`. Files defined using an absolute path are not affected by this setting.
|
||||
|
||||
`NXF_FUSION_TRACE`
|
||||
: :::{versionadded} 26.04.0
|
||||
:::
|
||||
: When set to `true`, collect task resource metrics (CPU, memory, I/O) from the Fusion trace file (`.fusion/trace.json`) produced in the task work directory, replacing the metrics collected by the default bash command-trace wrapper. Requires Fusion to be enabled. GPU metrics from Fusion are always collected regardless of this setting. Defaults to `false`.
|
||||
|
||||
`NXF_HOME`
|
||||
: Nextflow home directory (default: `$HOME/.nextflow`).
|
||||
|
||||
`NXF_JAVA_HOME`
|
||||
: Defines the path location of the Java VM installation used to run Nextflow. This variable overrides the `JAVA_HOME` variable if defined.
|
||||
|
||||
`NXF_JVM_ARGS`
|
||||
: :::{versionadded} 21.12.1-edge
|
||||
:::
|
||||
: Allows the setting Java VM options. This is similar to `NXF_OPTS` however it's only applied the JVM running Nextflow and not to any java pre-launching commands.
|
||||
|
||||
`NXF_LOG_FILE`
|
||||
: The filename of the Nextflow log (default: `.nextflow.log`)
|
||||
|
||||
`NXF_OFFLINE`
|
||||
: When `true` prevents Nextflow from automatically downloading and updating remote project repositories (default: `false`).
|
||||
: :::{versionchanged} 23.09.0-edge
|
||||
This option also disables the automatic version check (see `NXF_DISABLE_CHECK_LATEST`).
|
||||
:::
|
||||
: :::{versionchanged} 23.11.0-edge
|
||||
This option also prevents plugins from being downloaded. Plugin versions must be specified in offline mode, or else Nextflow will fail.
|
||||
:::
|
||||
|
||||
`NXF_OPTS`
|
||||
: Provides extra options for the Java and Nextflow runtime. It must be a blank separated list of `-Dkey[=value]` properties.
|
||||
|
||||
`NXF_ORG`
|
||||
: Default `organization` prefix when looking for a hosted repository (default: `nextflow-io`).
|
||||
|
||||
`NXF_PARAMS_FILE`
|
||||
: :::{versionadded} 20.10.0
|
||||
:::
|
||||
: Defines the path location of the pipeline parameters file .
|
||||
|
||||
`NXF_PID_FILE`
|
||||
: Name of the file where the process PID is saved when Nextflow is launched in background.
|
||||
|
||||
`NXF_PLUGINS_ALLOWED`
|
||||
: :::{versionadded} 25.04.0
|
||||
:::
|
||||
: Comma separated list of plugin IDs that can be used in a workflow executions e.g. `NXF_PLUGINS_ALLOWED=nf-amazon,nf-tower,nf-wave`. Use empty string to disallow all plugins.
|
||||
|
||||
`NXF_PLUGINS_DEFAULT`
|
||||
: Whether to use the default plugins when no plugins are specified in the Nextflow configuration (default: `true`).
|
||||
|
||||
`NXF_PLUGINS_DIR`
|
||||
: The path where the plugin archives are loaded and stored (default: `$NXF_HOME/plugins`).
|
||||
|
||||
`NXF_PLUGINS_REGISTRY_URL`
|
||||
: :::{versionadded} 25.08.0-edge
|
||||
:::
|
||||
: Specifies the URL of the plugin registry used to download and resolve plugins. This allows using custom or private plugin registries instead of the default public registry.
|
||||
|
||||
`NXF_PLUGINS_TEST_REPOSITORY`
|
||||
: :::{versionadded} 23.04.0
|
||||
:::
|
||||
: Defines a custom plugin registry or plugin release URL for testing plugins outside of the main registry.
|
||||
|
||||
`NXF_PUBLISH_FAIL_ON_ERROR`
|
||||
: :::{versionadded} 24.04.3
|
||||
:::
|
||||
: Defines the default behavior of `publishDir.failOnError` setting. See {ref}`publishDir<process-publishdir>` directive for more information.
|
||||
|
||||
`NXF_RETRY_POLICY_DELAY`
|
||||
: :::{versionadded} 25.06.0-edge
|
||||
:::
|
||||
: Delay used for HTTP retryable operations (default: `350ms`).
|
||||
|
||||
`NXF_RETRY_POLICY_JITTER`
|
||||
: :::{versionadded} 25.06.0-edge
|
||||
:::
|
||||
: Jitter value used for HTTP retryable operations (default: `0.25`).
|
||||
|
||||
`NXF_RETRY_POLICY_MAX_ATTEMPTS`
|
||||
: :::{versionadded} 25.06.0-edge
|
||||
:::
|
||||
: Max number of attempts used for HTTP retryable operations (default: `5`).
|
||||
|
||||
`NXF_RETRY_POLICY_MAX_DELAY`
|
||||
: :::{versionadded} 25.06.0-edge
|
||||
:::
|
||||
: Max delay used for HTTP retryable operations (default: `90s`).
|
||||
|
||||
`NXF_RETRY_POLICY_MULTIPLIER`
|
||||
: :::{versionadded} 25.08.0-edge
|
||||
:::
|
||||
: Delay multiplier used for HTTP retryable operations (default: `2.0`).
|
||||
|
||||
`NXF_SCM_FILE`
|
||||
: :::{versionadded} 20.10.0
|
||||
:::
|
||||
: Defines the path location of the SCM config file .
|
||||
|
||||
`NXF_SINGULARITY_CACHEDIR`
|
||||
: Directory where remote Singularity images are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
|
||||
|
||||
`NXF_SINGULARITY_LIBRARYDIR`
|
||||
: :::{versionadded} 21.09.0-edge
|
||||
:::
|
||||
: Directory where remote Singularity images are retrieved. It should be a directory accessible to all compute nodes.
|
||||
|
||||
`NXF_SPACK_CACHEDIR`
|
||||
: Directory where Spack environments are stored. When using a computing cluster it must be a shared folder accessible from all compute nodes.
|
||||
|
||||
`NXF_SPACK_ENABLED`
|
||||
: :::{versionadded} 23.02.0-edge
|
||||
:::
|
||||
: Enable the use of Spack recipes defined by using the {ref}`process-spack` directive. (default: `false`).
|
||||
|
||||
`NXF_SYNTAX_PARSER`
|
||||
: :::{versionadded} 25.02.0-edge
|
||||
:::
|
||||
: Set to `'v2'` to use the {ref}`strict syntax <strict-syntax-page>` for Nextflow scripts and config files (default: `'v1'`).
|
||||
|
||||
`NXF_TEMP`
|
||||
: Directory where temporary files are stored
|
||||
|
||||
`NXF_TRACE`
|
||||
: Enable trace level logging for the specified packages. Equivalent to the `-trace` command-line option.
|
||||
|
||||
`NXF_VER`
|
||||
: Defines which version of Nextflow to use.
|
||||
|
||||
`NXF_WORK`
|
||||
: Directory where working files are stored (usually your *scratch* directory)
|
||||
|
||||
`NXF_WRAPPER_STAGE_FILE_THRESHOLD`
|
||||
: :::{versionadded} 23.05.0-edge
|
||||
:::
|
||||
: Enables writing large staging scripts to a separate `.command.stage` file. The value defines the minimum size of the `.command.run` staging script for it to be written to the separate file (default when enabled: `'1 MB'`).
|
||||
: This setting is useful for executors that impose a size limit on job scripts.
|
||||
|
||||
## Seqera Platform settings
|
||||
|
||||
`TOWER_ACCESS_TOKEN`
|
||||
: Specifies the access token for authenticating with Seqera Platform. Can also be configured using the `tower.accessToken` config option.
|
||||
|
||||
`TOWER_API_ENDPOINT`
|
||||
: Specifies the Seqera Platform API endpoint (default: `https://api.cloud.seqera.io`). Can also be configured using the `tower.endpoint` config option.
|
||||
|
||||
`TOWER_AUTH_DOMAIN`
|
||||
: :::{versionadded} 25.10.0
|
||||
:::
|
||||
: Specifies the Auth0 domain for authenticating with Seqera Platform when connecting to a custom endpoint. When specified, this value takes precedence over the built-in mappings for known Seqera endpoints. Must be used in conjunction with `TOWER_AUTH_CLIENT_ID`.
|
||||
|
||||
`TOWER_AUTH_CLIENT_ID`
|
||||
: :::{versionadded} 25.10.0
|
||||
:::
|
||||
: Specifies the Auth0 client ID for authenticating with a custom Seqera Platform endpoint. Must be used in conjunction with `TOWER_AUTH_DOMAIN`.
|
||||
|
||||
`TOWER_REFRESH_TOKEN`
|
||||
: Specifies the refresh token for maintaining authentication with Seqera Platform. Can also be configured using the `tower.refreshToken` config option.
|
||||
|
||||
`TOWER_COMPUTE_ENV_ID`
|
||||
: Specifies the Seqera Platform compute environment ID. When specified, the scheduler resolves the compute environment directly by this ID instead of inferring a suitable compute environment. Can also be configured using the `tower.computeEnvId` config option.
|
||||
|
||||
`TOWER_WORKSPACE_ID`
|
||||
: Specifies the Seqera Platform workspace ID. Can also be configured using the `tower.workspaceId` config option.
|
||||
|
||||
## Other settings
|
||||
|
||||
`FTP_PROXY`
|
||||
: :::{versionadded} 21.06.0-edge
|
||||
:::
|
||||
: Defines the FTP proxy server. Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `ftp://user:password@proxy-host.com:port`.
|
||||
|
||||
`HTTP_PROXY`
|
||||
: Defines the HTTP proxy server.
|
||||
: :::{versionadded} 21.06.0-edge
|
||||
Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `http://user:password@proxy-host.com:port`.
|
||||
:::
|
||||
|
||||
`HTTPS_PROXY`
|
||||
: Defines the HTTPS proxy server.
|
||||
: :::{versionadded} 21.06.0-edge
|
||||
Proxy authentication is supported by providing the credentials in the proxy URL, e.g. `https://user:password@proxy-host.com:port`.
|
||||
:::
|
||||
|
||||
`NO_COLOR`
|
||||
: Disables ANSI color codes in Nextflow log output. When this variable is set, Nextflow prints plain text logs following the [NO_COLOR standard](https://no-color.org/).
|
||||
: If both `NO_COLOR` and `NXF_ANSI_LOG` are set, `NXF_ANSI_LOG` takes precedence.
|
||||
|
||||
`NO_PROXY`
|
||||
: Defines one or more host names that should not use the proxy server. Separate multiple names using a comma character.
|
||||
|
||||
`TERMINAL_WIDTH`
|
||||
: Forces the terminal width of ANSI-formatted log output. Overrides automatic terminal width detection and uses the specified width for line wrapping when set to an integer value.
|
||||
56
nextflow/docs/reference/feature-flags.md
Normal file
56
nextflow/docs/reference/feature-flags.md
Normal file
@@ -0,0 +1,56 @@
|
||||
(config-feature-flags)=
|
||||
|
||||
# Feature flags
|
||||
|
||||
Feature flags enable opt-in features. They must be specified in the pipeline script.
|
||||
|
||||
:::{warning}
|
||||
Feature flags marked as *preview* can cause pipelines run with newer versions of Nextflow to fail due to breaking changes. Always consult the {ref}`migration notes <migrations-page>` before updating to a new Nextflow version.
|
||||
:::
|
||||
|
||||
`nextflow.enable.configProcessNamesValidation`
|
||||
: :::{deprecated} 25.10.0
|
||||
Use the {ref}`strict syntax <strict-syntax-page>` instead. It validates process selectors without producing false warnings.
|
||||
:::
|
||||
: When `true`, prints a warning for every `withName:` process selector that doesn't match a process in the pipeline (default: `true`).
|
||||
|
||||
`nextflow.enable.dsl`
|
||||
: :::{deprecated} 25.04.0
|
||||
:::
|
||||
: Defines the DSL version to use (`1` or `2`).
|
||||
|
||||
`nextflow.enable.moduleBinaries`
|
||||
: When `true`, enables the use of module-scoped executable scripts via {ref}`module resources <module-resources>`.
|
||||
|
||||
`nextflow.enable.strict`
|
||||
: :::{deprecated} 26.04.0
|
||||
:::
|
||||
: When `true`, executes the pipeline in "strict" mode, which introduces the following rules:
|
||||
|
||||
- When reading a params file, Nextflow will fail if a dynamic param value references an undefined variable
|
||||
|
||||
- When merging params from a config file with params from the command line, Nextflow will fail if a param is specified from both sources but with different types
|
||||
|
||||
- When using the `join` operator, the `failOnDuplicate` option is `true` regardless of any user setting
|
||||
|
||||
- When using the `join` operator, the `failOnMismatch` option is `true` (unless `remainder` is also `true`) regardless of any user setting
|
||||
|
||||
- When using the `publishDir` process directive, the `failOnError` option is `true` regardless of any user setting
|
||||
|
||||
- In a process definition, Nextflow will fail if an input or output tuple has only one element
|
||||
|
||||
- In a process definition, Nextflow will fail if an output emit name is not a valid identifier (i.e. it should match the pattern `/[A-Za-z_][A-Za-z0-9_]*/`)
|
||||
|
||||
- During a process execution, Nextflow will fail if a received input tuple does not have the same number of elements as was declared
|
||||
|
||||
- During a process execution, Nextflow will fail if the `storeDir` directive is used with non-file outputs
|
||||
|
||||
- Nextflow will fail if a pipeline param is referenced before it is defined
|
||||
|
||||
- Nextflow will fail if multiple functions and/or processes with the same name are defined in a module script
|
||||
|
||||
`nextflow.enable.types`
|
||||
: :::{versionadded} 26.04.0
|
||||
:::
|
||||
: *Preview feature: the syntax and behavior may change in future releases.*
|
||||
: When `true`, enables the use of {ref}`typed processes <process-typed-page>` and {ref}`typed workflows <workflow-typed-page>`. Must be enabled in every script that uses typed processes/workflows. Legacy processes/workflows cannot be defined in scripts with this flag enabled.
|
||||
350
nextflow/docs/reference/operator-typed.md
Normal file
350
nextflow/docs/reference/operator-typed.md
Normal file
@@ -0,0 +1,350 @@
|
||||
(operator-typed-page)=
|
||||
|
||||
# Operators (typed)
|
||||
|
||||
:::{versionadded} 26.04.0
|
||||
:::
|
||||
|
||||
This page describes the core operators that are recommended for use with static typing.
|
||||
|
||||
## collect
|
||||
|
||||
**`Channel<E> collect() -> Value<Bag<E>>`**
|
||||
|
||||
The `collect` operator collects all values from a source channel into a collection and emits it as a dataflow value:
|
||||
|
||||
```{literalinclude} ../snippets/collect.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/collect.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## combine
|
||||
|
||||
**`Channel<L> combine( other: Channel<R> ) -> Channel<Tuple>`**
|
||||
|
||||
**`Channel<L> combine( other: Value<R> ) -> Channel<Tuple>`**
|
||||
|
||||
The `combine` operator emits every pairwise combination of a source channel with another channel or dataflow value:
|
||||
|
||||
```{literalinclude} ../snippets/combine.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/combine.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
Tuples in both the left- and right-hand sources are flattened in the combined tuple. For example, `tuple(1, 2)` and `tuple('red', 'blue')` are combined as `tuple(1, 2, 'red', 'blue')`.
|
||||
|
||||
**`Channel<Record> combine( [opts] ) -> Channel<Record>`**
|
||||
|
||||
When the `combine` operator is called on a channel of records with named arguments, the named arguments are appended to each record in the source channel. Each named argument can be a value or dataflow value.
|
||||
|
||||
## filter
|
||||
|
||||
**`Channel<E> filter( condition: (E) -> Boolean ) -> Channel<E>`**
|
||||
|
||||
The `filter` operator emits the values from a source channel that satisfy a condition, discarding all other values:
|
||||
|
||||
```{literalinclude} ../snippets/filter-closure.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/filter-closure.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## flatMap
|
||||
|
||||
**`Channel<E> flatMap( transform: (E) -> Iterable<R> ) -> Channel<R>`**
|
||||
|
||||
The `flatMap` operator applies a *mapping function* to each value from a source channel. The mapping function should return a collection, and each element in the collection is emitted separately.
|
||||
|
||||
For example:
|
||||
|
||||
```{literalinclude} ../snippets/flatmap-list.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/flatmap-list.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
(operator-groupby)=
|
||||
|
||||
## groupBy
|
||||
|
||||
**`Channel<Tuple<K, V>> groupBy() -> Channel<Tuple<K, Bag<V>>>`**
|
||||
|
||||
**`Channel<Tuple<K, Integer, V>> groupBy() -> Channel<Tuple<K, Bag<V>>>`**
|
||||
|
||||
The `groupBy` operator collects values from a source channel into groups based on a grouping key. A tuple is emitted for each group, containing the grouping key and collection of values.
|
||||
|
||||
The source channel should supply either 2-tuples of the form `(<key>, <value>)` or 3-tuples of the form `(<key>, <size>, <value>)`.
|
||||
|
||||
If the source tuples do not specify a size, `groupBy` will not emit any groups until *all* inputs have been received:
|
||||
|
||||
```{literalinclude} ../snippets/groupby.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/groupby.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
If the source tuples do specify a size, then `groupBy` will emit each group as soon as it is ready:
|
||||
|
||||
```{literalinclude} ../snippets/groupby-size.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/groupby-size.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
:::{note}
|
||||
When specifying the group size, make sure that the number of inputs for a given group matches the specified size for that group. Otherwise, the run will fail.
|
||||
:::
|
||||
|
||||
(operator-join-record)=
|
||||
|
||||
## join
|
||||
|
||||
**`Channel<Record> join( other: Channel<Record>, [opts] ) -> Channel<Record>`**
|
||||
|
||||
The `join` operator emits the relational join of two channels of records, using a matching key given by the `by` option:
|
||||
|
||||
```{literalinclude} ../snippets/join-record.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/join-record.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
Duplicate matching keys are handled by emitting each matching combination (like a relational join):
|
||||
|
||||
```{literalinclude} ../snippets/join-record-duplicates.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/join-record-duplicates.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
By default, unmatched values are discarded. The `remainder` option can be used to emit them at the end:
|
||||
|
||||
```{literalinclude} ../snippets/join-record-remainder.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/join-record-remainder.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
Available options:
|
||||
|
||||
`by: String`
|
||||
: *This option is required.*
|
||||
: The record field to use as the matching key.
|
||||
|
||||
`remainder: Boolean`
|
||||
: When `true`, unmatched values are emitted at the end, otherwise they are discarded (default: `false`).
|
||||
|
||||
## map
|
||||
|
||||
**`Channel<E> map( transform: (E) -> R ) -> Channel<R>`**
|
||||
|
||||
The `map` operator applies a *mapping function* to each value from a source channel:
|
||||
|
||||
```{literalinclude} ../snippets/map.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/map.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## mix
|
||||
|
||||
**`Channel<E> mix( other: Channel<E> ) -> Channel<E>`**
|
||||
|
||||
**`Channel<E> mix( other: Value<E> ) -> Channel<E>`**
|
||||
|
||||
The `mix` operator emits the values from a channel and another channel or dataflow value into a single output channel:
|
||||
|
||||
```{literalinclude} ../snippets/mix.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/mix.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
The values in the output channel may be emitted in any order, for example:
|
||||
|
||||
```console
|
||||
z
|
||||
1
|
||||
a
|
||||
2
|
||||
b
|
||||
3
|
||||
```
|
||||
|
||||
## reduce
|
||||
|
||||
**`Channel<E> reduce( seed: R, accumulator: (R, E) -> R ) -> Value<R>`**
|
||||
|
||||
**`Channel<E> reduce( accumulator: (E, E) -> E ) -> Value<E>`**
|
||||
|
||||
The `reduce` operator applies an *accumulator function* sequentially to each value in a source channel, and emits the final accumulated value.
|
||||
|
||||
The accumulator function takes two parameters -- the accumulated value and the *i*-th emitted value -- and it should return the accumulated result, which is passed to the next invocation with the *i+1*-th value. This process is repeated for each value in the source channel.
|
||||
|
||||
For example:
|
||||
|
||||
```{literalinclude} ../snippets/reduce.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/reduce.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
By default, the first value is used as the initial accumulated value (the *seed*). You can optionally specify a different initial value as shown below:
|
||||
|
||||
```{literalinclude} ../snippets/reduce-with-initial-value.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/reduce-with-initial-value.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## subscribe
|
||||
|
||||
**`Channel<E> subscribe( action: (E) -> () )`**
|
||||
|
||||
**`Channel<E> subscribe( [opts] )`**
|
||||
|
||||
The `subscribe` operator invokes a custom function for each value from a source channel:
|
||||
|
||||
```{literalinclude} ../snippets/subscribe.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/subscribe.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
The `subscribe` operator supports multiple types of event handlers:
|
||||
|
||||
```{literalinclude} ../snippets/subscribe-with-on-complete.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/subscribe-with-on-complete.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
:::{note}
|
||||
Unlike most operators, `subscribe` does not return anything. It should only be used for *side effects*, such as printing to the console, writing to a file, or making HTTP requests.
|
||||
:::
|
||||
|
||||
Available options:
|
||||
|
||||
`onNext: (E) -> ()`
|
||||
: Closure that is invoked when an value is emitted. Equivalent to providing a single closure argument.
|
||||
|
||||
`onComplete: () -> ()`
|
||||
: Closure that is invoked after the last value is emitted by the channel.
|
||||
|
||||
`onError: (T) -> ()`
|
||||
: Closure that is invoked when an exception is raised while handling the `onNext` event. It will not make further calls to `onNext` or `onComplete`. The `onError` method takes as its parameter the `Throwable` that caused the error.
|
||||
|
||||
## unique
|
||||
|
||||
**`Channel<E> unique( transform: (E) -> ? ) -> Channel<E>`**
|
||||
**`Channel<E> unique() -> Channel<E>`**
|
||||
|
||||
The `unique` operator emits the unique values from a source channel:
|
||||
|
||||
```{literalinclude} ../snippets/unique.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/unique.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
An optional closure can be used to transform each value before it is evaluated for uniqueness:
|
||||
|
||||
```{literalinclude} ../snippets/unique-with-mapper.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/unique-with-mapper.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## until
|
||||
|
||||
**`Channel<E> until( condition: (E) -> Boolean ) -> Channel<E>`**
|
||||
|
||||
The `until` operator emits each value from a source channel until a stopping condition is satisfied:
|
||||
|
||||
```{literalinclude} ../snippets/until.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/until.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
## view
|
||||
|
||||
**`Channel<E> view( transform: (E) -> String, [opts] ) -> Channel<E>`**
|
||||
|
||||
**`Channel<E> view( [opts] ) -> Channel<E>`**
|
||||
|
||||
The `view` operator prints each value from a source channel to standard output:
|
||||
|
||||
```{literalinclude} ../snippets/view.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/view.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
An optional closure can be used to transform each value before it is printed:
|
||||
|
||||
```{literalinclude} ../snippets/view-with-mapper.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
```{literalinclude} ../snippets/view-with-mapper.out
|
||||
:language: console
|
||||
```
|
||||
|
||||
The `tag` option can be used to print the channel only when the `-dump-channels` command-line option is specified with the given tag:
|
||||
|
||||
```{literalinclude} ../snippets/view-tag.nf
|
||||
:language: nextflow
|
||||
```
|
||||
|
||||
You can run this script with `-dump-channels plus1` or `-dump-channels exp2` to print either channel, or `-dump-channels plus1,exp2` to print both.
|
||||
|
||||
The `view` operator also emits every value that it receives, allowing it to be chained with other operators.
|
||||
|
||||
Available options:
|
||||
|
||||
`newLine: Boolean`
|
||||
: Print each value to a separate line (default: `true`).
|
||||
|
||||
`tag: String`
|
||||
: Print the channel values only when `-dump-channels` is specified on the command line with the given tag.
|
||||
1591
nextflow/docs/reference/operator.md
Normal file
1591
nextflow/docs/reference/operator.md
Normal file
File diff suppressed because it is too large
Load Diff
1768
nextflow/docs/reference/process.md
Normal file
1768
nextflow/docs/reference/process.md
Normal file
File diff suppressed because it is too large
Load Diff
25
nextflow/docs/reference/stdlib-groovy.md
Normal file
25
nextflow/docs/reference/stdlib-groovy.md
Normal file
@@ -0,0 +1,25 @@
|
||||
(stdlib-groovy)=
|
||||
|
||||
# Groovy and Java classes
|
||||
|
||||
Any Groovy or Java class that is available to Nextflow at runtime can be used in a Nextflow script. The following classes are imported by default:
|
||||
|
||||
- `groovy.lang.*`
|
||||
- `groovy.util.*`
|
||||
- `java.io.*`
|
||||
- `java.lang.*`
|
||||
- `java.math.BigDecimal`
|
||||
- `java.math.BigInteger`
|
||||
- `java.net.*`
|
||||
- `java.util.*`
|
||||
|
||||
All other classes must be referenced by their fully-qualified name:
|
||||
|
||||
```nextflow
|
||||
def vals = [1, 2, 3]
|
||||
println groovy.json.JsonOutput.toJson(vals)
|
||||
```
|
||||
|
||||
:::{note}
|
||||
The set of classes in Nextflow's runtime classpath can change between different Nextflow versions. As a best practice, any code that uses classes outside the Nextflow standard library should either be refactored to only use the Nextflow standard library or be refactored as a {ref}`plugin <dev-plugins-page>` with explicit dependencies.
|
||||
:::
|
||||
289
nextflow/docs/reference/stdlib-namespaces.md
Normal file
289
nextflow/docs/reference/stdlib-namespaces.md
Normal file
@@ -0,0 +1,289 @@
|
||||
(stdlib-namespaces)=
|
||||
|
||||
# Namespaces
|
||||
|
||||
This page lists all of the available namespaces in the Nextflow standard library.
|
||||
|
||||
(stdlib-namespaces-global)=
|
||||
|
||||
## Global namespace
|
||||
|
||||
The global namespace contains globally available constants and functions.
|
||||
|
||||
**Constants**
|
||||
|
||||
`baseDir: Path`
|
||||
: :::{deprecated} 20.04.0
|
||||
:::
|
||||
: Alias of `workflow.projectDir`.
|
||||
|
||||
`launchDir: Path`
|
||||
: Alias of `workflow.launchDir`.
|
||||
|
||||
`moduleDir: Path`
|
||||
: Directory where a module script is located (equivalent to `projectDir` if used in the main script).
|
||||
|
||||
`params`
|
||||
: Map of workflow parameters specified in the config file or as command line options.
|
||||
|
||||
`projectDir: Path`
|
||||
: Alias of `workflow.projectDir`.
|
||||
|
||||
`secrets: Map<String,String>`
|
||||
: :::{versionadded} 24.02.0-edge
|
||||
:::
|
||||
: Map of pipeline secrets. See {ref}`secrets-page` for more information.
|
||||
|
||||
`workDir: Path`
|
||||
: Alias of `workflow.workDir`.
|
||||
|
||||
**Functions**
|
||||
|
||||
`branchCriteria( criteria: Closure ) -> Closure`
|
||||
: Create a branch criteria to use with the {ref}`operator-branch` operator.
|
||||
|
||||
`env( name: String ) -> String`
|
||||
: :::{versionadded} 25.04.0
|
||||
:::
|
||||
: Get the value of the environment variable with the specified name in the Nextflow launch environment.
|
||||
|
||||
`error( message: String = null )`
|
||||
: Throw a script runtime error with an optional error message.
|
||||
|
||||
`exit( exitCode: int = 0, message: String = null )`
|
||||
: :::{deprecated} 22.06.0-edge
|
||||
Use `error()` instead
|
||||
:::
|
||||
: Stop the pipeline execution and return an exit code and optional error message.
|
||||
|
||||
`file( filePattern: String, [options] ) -> Path`
|
||||
: Get a file from a file name or glob pattern.
|
||||
|
||||
: The following options are available:
|
||||
|
||||
`checkIfExists: boolean`
|
||||
: When `true`, throws an exception if the specified path does not exist in the file system (default: `false`)
|
||||
|
||||
`followLinks: boolean`
|
||||
: When `true`, follows symbolic links when traversing a directory tree, otherwise treats them as files (default: `true`)
|
||||
|
||||
`glob: boolean`
|
||||
: When `true`, interprets characters `*`, `?`, `[]` and `{}` as glob wildcards, otherwise handles them as normal characters (default: `true`)
|
||||
|
||||
`hidden: boolean`
|
||||
: When `true`, includes hidden files in the resulting paths (default: `false`)
|
||||
|
||||
`maxDepth: int`
|
||||
: Maximum number of directory levels to visit (default: *no limit*)
|
||||
|
||||
`type: String`
|
||||
: Type of paths returned, can be `'file'`, `'dir'` or `'any'` (default: `'file'`)
|
||||
|
||||
: :::{note}
|
||||
This function returns a collection if the glob pattern yields zero or multiple files. Use `files()` to get a collection of files.
|
||||
:::
|
||||
|
||||
`files( filePattern: String, [options] ) -> Iterable<Path>`
|
||||
: Get a collection of files from a file name or glob pattern. Supports the same options as `file()`.
|
||||
: See also: {ref}`channel.fromPath <channel-path>`.
|
||||
|
||||
`groupKey( key, size: int ) -> GroupKey`
|
||||
: Create a grouping key to use with the {ref}`operator-grouptuple` operator.
|
||||
|
||||
`multiMapCriteria( criteria: Closure ) -> Closure`
|
||||
: Create a multi-map criteria to use with the {ref}`operator-multiMap` operator.
|
||||
|
||||
`print( value )`
|
||||
: Print a value to standard output.
|
||||
|
||||
`printf( format: String, values... )`
|
||||
: Print a formatted string with the given values to standard output.
|
||||
|
||||
`println( value )`
|
||||
: Print a value to standard output with a newline.
|
||||
|
||||
`sendMail( [options] )`
|
||||
: Send an email. See {ref}`mail-page` for more information.
|
||||
|
||||
`sleep( milliseconds: long )`
|
||||
: Sleep for the given number of milliseconds.
|
||||
|
||||
`record( [options] ) -> Record`
|
||||
: Create a record from the given named arguments.
|
||||
|
||||
`tuple( args... ) -> Tuple`
|
||||
: Create a tuple from the given arguments.
|
||||
|
||||
(stdlib-namespaces-channel)=
|
||||
|
||||
## `channel`
|
||||
|
||||
The `channel` namespace contains the built-in channel factories. See {ref}`channel-factory` for details.
|
||||
|
||||
(stdlib-namespaces-nextflow)=
|
||||
|
||||
## `log`
|
||||
|
||||
The `log` namepsace contains functions for logging messages to the console.
|
||||
|
||||
`error( message: String )`
|
||||
: Log an error message to the console.
|
||||
: This function does not terminate the pipeline -- use the global `error()` function instead.
|
||||
|
||||
`info( message: String )`
|
||||
: Log an info message to the console.
|
||||
|
||||
`warn( message: String )`
|
||||
: Log a warning message to the console.
|
||||
|
||||
## `nextflow`
|
||||
|
||||
The `nextflow` namespace contains information about the current Nextflow runtime.
|
||||
|
||||
`build: int`
|
||||
: Nextflow runtime build number.
|
||||
|
||||
`timestamp: String`
|
||||
: Nextflow runtime compile timestamp.
|
||||
|
||||
`version: VersionNumber`
|
||||
: Nextflow runtime version number. See {ref}`VersionNumber <stdlib-types-versionnumber>` for more information.
|
||||
|
||||
(stdlib-namespaces-workflow)=
|
||||
|
||||
## `workflow`
|
||||
|
||||
The `workflow` namespace contains information about the current workflow run.
|
||||
|
||||
**Properties**
|
||||
|
||||
`commandLine: String`
|
||||
: Command line as entered by the user to launch the workflow execution.
|
||||
|
||||
`commitId: String`
|
||||
: Git commit ID of the executed workflow repository.
|
||||
: When providing a Git tag, branch name, or commit hash using the `-r` CLI option, the associated `workflow.commitId` is also populated.
|
||||
|
||||
`complete: OffsetDateTime`
|
||||
: *Available only in the `workflow.onComplete` handler*
|
||||
: Timestamp of workflow when execution is completed.
|
||||
|
||||
`configFiles: List<Path>`
|
||||
: Configuration files used for the workflow execution.
|
||||
|
||||
`container: String | Map<String,String>`
|
||||
: Docker image used to run workflow tasks, or a map of process names to process containers when multiple images are used.
|
||||
|
||||
`containerEngine: String`
|
||||
: Returns the name of the container engine (e.g. docker or singularity) or null if no container engine is enabled.
|
||||
|
||||
`duration: Duration`
|
||||
: *Available only in the `workflow.onComplete` handler*
|
||||
: Time elapsed to complete workflow execution.
|
||||
|
||||
`errorMessage: String`
|
||||
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
|
||||
: Error message of the task that caused the workflow execution to fail.
|
||||
|
||||
`errorReport: String`
|
||||
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
|
||||
: Detailed error of the task that caused the workflow execution to fail.
|
||||
|
||||
`exitStatus: int`
|
||||
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
|
||||
: Exit status of the task that caused the workflow execution to fail.
|
||||
|
||||
`failOnIgnore: boolean`
|
||||
: :::{versionadded} 24.05.0-edge
|
||||
:::
|
||||
: Whether the `workflow.failOnIgnore` config option was enabled.
|
||||
: See also: {ref}`process-error-strategy`
|
||||
|
||||
`fusion`
|
||||
: Namespace containing information about the current Fusion runtime. The following properties are available:
|
||||
|
||||
: `enabled: boolean`
|
||||
: Whether Fusion is enabled.
|
||||
|
||||
: `version: String`
|
||||
: The Fusion version being used.
|
||||
|
||||
`homeDir: Path`
|
||||
: User system home directory.
|
||||
|
||||
`launchDir: Path`
|
||||
: Directory where the workflow was launched.
|
||||
|
||||
`manifest`
|
||||
: Namespace corresponding to the {ref}`config-manifest` config scope.
|
||||
|
||||
`outputDir: Path`
|
||||
: :::{versionadded} 24.10.0
|
||||
:::
|
||||
: Workflow output directory.
|
||||
|
||||
`preview: boolean`
|
||||
: :::{versionadded} 24.04.0
|
||||
:::
|
||||
: Whether the current workflow run is a preview run.
|
||||
|
||||
`profile: String`
|
||||
: Comma-separated list of active configuration profiles.
|
||||
|
||||
`projectDir: Path`
|
||||
: Directory where the workflow project is located.
|
||||
|
||||
`repository: String`
|
||||
: Project repository Git remote URL.
|
||||
|
||||
`resume: boolean`
|
||||
: Returns `true` whenever the current instance is resumed from a previous execution.
|
||||
|
||||
`revision: String`
|
||||
: Git branch/tag of the executed workflow repository.
|
||||
: When providing a Git tag or branch name using the `-r` CLI option, the `workflow.revision` is also populated.
|
||||
|
||||
`runName: String`
|
||||
: Mnemonic name assigned to this execution instance.
|
||||
|
||||
`scriptFile: Path`
|
||||
: Project main script file path.
|
||||
|
||||
`scriptId: String`
|
||||
: Project main script unique hash ID.
|
||||
|
||||
`scriptName: String`
|
||||
: Project main script file name.
|
||||
|
||||
`sessionId: UUID`
|
||||
: Unique identifier (UUID) associated to current execution.
|
||||
|
||||
`start: OffsetDateTime`
|
||||
: Timestamp of workflow at execution start.
|
||||
|
||||
`stubRun: boolean`
|
||||
: Returns `true` whenever the current instance is a stub-run execution .
|
||||
|
||||
`success: boolean`
|
||||
: *Available only in the `workflow.onComplete` and `workflow.onError` handlers*
|
||||
: Reports if the execution completed successfully.
|
||||
|
||||
`userName: String`
|
||||
: User system account name.
|
||||
|
||||
`wave`
|
||||
: Namespace containing Wave runtime information. The following properties are available:
|
||||
|
||||
: `enabled: boolean`
|
||||
: Whether Wave is enabled.
|
||||
|
||||
`workDir: Path`
|
||||
: The directory where task temporary files are stored.
|
||||
|
||||
**Functions**
|
||||
|
||||
`onComplete( action: Closure )`
|
||||
: Define an action to take when the workflow completes (whether successful or not).
|
||||
|
||||
`onError( action: Closure )`
|
||||
: Define an action to take if the workflow is terminated due to a runtime error or task failure.
|
||||
1050
nextflow/docs/reference/stdlib-types.md
Normal file
1050
nextflow/docs/reference/stdlib-types.md
Normal file
File diff suppressed because it is too large
Load Diff
12
nextflow/docs/reference/stdlib.md
Normal file
12
nextflow/docs/reference/stdlib.md
Normal file
@@ -0,0 +1,12 @@
|
||||
(stdlib-page)=
|
||||
|
||||
# Standard library
|
||||
|
||||
This section describes the Nextflow standard library, which consists of built-in namespaces and types.
|
||||
|
||||
```{toctree}
|
||||
:maxdepth: 1
|
||||
stdlib-namespaces
|
||||
stdlib-types
|
||||
stdlib-groovy
|
||||
```
|
||||
1062
nextflow/docs/reference/syntax.md
Normal file
1062
nextflow/docs/reference/syntax.md
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user