kevin/ma
1
0
Fork 0
Code Issues 1 Packages Activity

add nextflow d30e48d

Browse Source
This commit is contained in:
Kevin
2026-04-29 23:01:54 +02:00
parent d0b12d668d
commit 97cc9058d3
2840 changed files with 730250 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
nextflow/docs/.wci.yml Normal file
View File

@@ -0,0 +1,53 @@
---
# Metadata parsed by the workflow community initiative, https://workflows.community/systems
# Registered at https://github.com/workflowscommunity/workflowscommunity.github.io/blob/main/_data/workflow_systems.yml
name: Nextflow
headline: A DSL for data-driven computational pipelines
description: |
Nextflow is an open-source workflow orchestrator that simplifies writing and deploying containerised data pipelines in a portable and scalable manner. It supports deploying workflows on a variety of execution platforms including local computers, HPC batch schedulers, Kubernetes and cloud platforms such as AWS, Azure and Google Cloud.
language: Groovy
documentation:
general: http://docs.nextflow.io
installation: https://www.nextflow.io/docs/latest/getstarted.html
tutorial: https://training.nextflow.io
social:
twitter: nextflowio
youtube: https://www.youtube.com/c/Nextflow
execution_environment:
interfaces:
- GitHub
- GitLab
- BitBucket
resource_managers:
- Slurm
- LSF
- PBS/Torque
- PBS Pro
- SGE
- Moab
- HTCondor
- Bridge
- HyperQueue
- GLS
- NQSII
- OAR
- Local
- AWS
- GCP
- Azure
- Kubernetes
transfer_protocols:
- HTTP
- HTTPS
- FTP
- Amazon S3
- Google Storage
- Azure Storage
- Git+HTTPS

14
nextflow/docs/Dockerfile Normal file
View File

@@ -0,0 +1,14 @@
FROM mambaorg/micromamba:1.5.8
RUN micromamba install --yes --name base --channel conda-forge \
make=4.3 \
python=3.7 \
conda-forge:git=2.45.0 \
&& \
micromamba clean --all --yes
COPY requirements.txt .
RUN eval "$(micromamba shell hook --shell=bash)" && \
micromamba activate && \
pip install -r requirements.txt

428
nextflow/docs/LICENCE.txt Normal file
View File

@@ -0,0 +1,428 @@
Attribution-ShareAlike 4.0 International
=======================================================================
Creative Commons Corporation ("Creative Commons") is not a law firm and
does not provide legal services or legal advice. Distribution of
Creative Commons public licenses does not create a lawyer-client or
other relationship. Creative Commons makes its licenses and related
information available on an "as-is" basis. Creative Commons gives no
warranties regarding its licenses, any material licensed under their
terms and conditions, or any related information. Creative Commons
disclaims all liability for damages resulting from their use to the
fullest extent possible.
Using Creative Commons Public Licenses
Creative Commons public licenses provide a standard set of terms and
conditions that creators and other rights holders may use to share
original works of authorship and other material subject to copyright
and certain other rights specified in the public license below. The
following considerations are for informational purposes only, are not
exhaustive, and do not form part of our licenses.
Considerations for licensors: Our public licenses are
intended for use by those authorized to give the public
permission to use material in ways otherwise restricted by
copyright and certain other rights. Our licenses are
irrevocable. Licensors should read and understand the terms
and conditions of the license they choose before applying it.
Licensors should also secure all rights necessary before
applying our licenses so that the public can reuse the
material as expected. Licensors should clearly mark any
material not subject to the license. This includes other CC-
licensed material, or material used under an exception or
limitation to copyright. More considerations for licensors:
wiki.creativecommons.org/Considerations_for_licensors
Considerations for the public: By using one of our public
licenses, a licensor grants the public permission to use the
licensed material under specified terms and conditions. If
the licensor's permission is not necessary for any reason--for
example, because of any applicable exception or limitation to
copyright--then that use is not regulated by the license. Our
licenses grant only permissions under copyright and certain
other rights that a licensor has authority to grant. Use of
the licensed material may still be restricted for other
reasons, including because others have copyright or other
rights in the material. A licensor may make special requests,
such as asking that all changes be marked or described.
Although not required by our licenses, you are encouraged to
respect those requests where reasonable. More considerations
for the public:
wiki.creativecommons.org/Considerations_for_licensees
=======================================================================
Creative Commons Attribution-ShareAlike 4.0 International Public
License
By exercising the Licensed Rights (defined below), You accept and agree
to be bound by the terms and conditions of this Creative Commons
Attribution-ShareAlike 4.0 International Public License ("Public
License"). To the extent this Public License may be interpreted as a
contract, You are granted the Licensed Rights in consideration of Your
acceptance of these terms and conditions, and the Licensor grants You
such rights in consideration of benefits the Licensor receives from
making the Licensed Material available under these terms and
conditions.
Section 1 -- Definitions.
a. Adapted Material means material subject to Copyright and Similar
Rights that is derived from or based upon the Licensed Material
and in which the Licensed Material is translated, altered,
arranged, transformed, or otherwise modified in a manner requiring
permission under the Copyright and Similar Rights held by the
Licensor. For purposes of this Public License, where the Licensed
Material is a musical work, performance, or sound recording,
Adapted Material is always produced where the Licensed Material is
synched in timed relation with a moving image.
b. Adapter's License means the license You apply to Your Copyright
and Similar Rights in Your contributions to Adapted Material in
accordance with the terms and conditions of this Public License.
c. BY-SA Compatible License means a license listed at
creativecommons.org/compatiblelicenses, approved by Creative
Commons as essentially the equivalent of this Public License.
d. Copyright and Similar Rights means copyright and/or similar rights
closely related to copyright including, without limitation,
performance, broadcast, sound recording, and Sui Generis Database
Rights, without regard to how the rights are labeled or
categorized. For purposes of this Public License, the rights
specified in Section 2(b)(1)-(2) are not Copyright and Similar
Rights.
e. Effective Technological Measures means those measures that, in the
absence of proper authority, may not be circumvented under laws
fulfilling obligations under Article 11 of the WIPO Copyright
Treaty adopted on December 20, 1996, and/or similar international
agreements.
f. Exceptions and Limitations means fair use, fair dealing, and/or
any other exception or limitation to Copyright and Similar Rights
that applies to Your use of the Licensed Material.
g. License Elements means the license attributes listed in the name
of a Creative Commons Public License. The License Elements of this
Public License are Attribution and ShareAlike.
h. Licensed Material means the artistic or literary work, database,
or other material to which the Licensor applied this Public
License.
i. Licensed Rights means the rights granted to You subject to the
terms and conditions of this Public License, which are limited to
all Copyright and Similar Rights that apply to Your use of the
Licensed Material and that the Licensor has authority to license.
j. Licensor means the individual(s) or entity(ies) granting rights
under this Public License.
k. Share means to provide material to the public by any means or
process that requires permission under the Licensed Rights, such
as reproduction, public display, public performance, distribution,
dissemination, communication, or importation, and to make material
available to the public including in ways that members of the
public may access the material from a place and at a time
individually chosen by them.
l. Sui Generis Database Rights means rights other than copyright
resulting from Directive 96/9/EC of the European Parliament and of
the Council of 11 March 1996 on the legal protection of databases,
as amended and/or succeeded, as well as other essentially
equivalent rights anywhere in the world.
m. You means the individual or entity exercising the Licensed Rights
under this Public License. Your has a corresponding meaning.
Section 2 -- Scope.
a. License grant.
1. Subject to the terms and conditions of this Public License,
the Licensor hereby grants You a worldwide, royalty-free,
non-sublicensable, non-exclusive, irrevocable license to
exercise the Licensed Rights in the Licensed Material to:
a. reproduce and Share the Licensed Material, in whole or
in part; and
b. produce, reproduce, and Share Adapted Material.
2. Exceptions and Limitations. For the avoidance of doubt, where
Exceptions and Limitations apply to Your use, this Public
License does not apply, and You do not need to comply with
its terms and conditions.
3. Term. The term of this Public License is specified in Section
6(a).
4. Media and formats; technical modifications allowed. The
Licensor authorizes You to exercise the Licensed Rights in
all media and formats whether now known or hereafter created,
and to make technical modifications necessary to do so. The
Licensor waives and/or agrees not to assert any right or
authority to forbid You from making technical modifications
necessary to exercise the Licensed Rights, including
technical modifications necessary to circumvent Effective
Technological Measures. For purposes of this Public License,
simply making modifications authorized by this Section 2(a)
(4) never produces Adapted Material.
5. Downstream recipients.
a. Offer from the Licensor -- Licensed Material. Every
recipient of the Licensed Material automatically
receives an offer from the Licensor to exercise the
Licensed Rights under the terms and conditions of this
Public License.
b. Additional offer from the Licensor -- Adapted Material.
Every recipient of Adapted Material from You
automatically receives an offer from the Licensor to
exercise the Licensed Rights in the Adapted Material
under the conditions of the Adapter's License You apply.
c. No downstream restrictions. You may not offer or impose
any additional or different terms or conditions on, or
apply any Effective Technological Measures to, the
Licensed Material if doing so restricts exercise of the
Licensed Rights by any recipient of the Licensed
Material.
6. No endorsement. Nothing in this Public License constitutes or
may be construed as permission to assert or imply that You
are, or that Your use of the Licensed Material is, connected
with, or sponsored, endorsed, or granted official status by,
the Licensor or others designated to receive attribution as
provided in Section 3(a)(1)(A)(i).
b. Other rights.
1. Moral rights, such as the right of integrity, are not
licensed under this Public License, nor are publicity,
privacy, and/or other similar personality rights; however, to
the extent possible, the Licensor waives and/or agrees not to
assert any such rights held by the Licensor to the limited
extent necessary to allow You to exercise the Licensed
Rights, but not otherwise.
2. Patent and trademark rights are not licensed under this
Public License.
3. To the extent possible, the Licensor waives any right to
collect royalties from You for the exercise of the Licensed
Rights, whether directly or through a collecting society
under any voluntary or waivable statutory or compulsory
licensing scheme. In all other cases the Licensor expressly
reserves any right to collect such royalties.
Section 3 -- License Conditions.
Your exercise of the Licensed Rights is expressly made subject to the
following conditions.
a. Attribution.
1. If You Share the Licensed Material (including in modified
form), You must:
a. retain the following if it is supplied by the Licensor
with the Licensed Material:
i. identification of the creator(s) of the Licensed
Material and any others designated to receive
attribution, in any reasonable manner requested by
the Licensor (including by pseudonym if
designated);
ii. a copyright notice;
iii. a notice that refers to this Public License;
iv. a notice that refers to the disclaimer of
warranties;
v. a URI or hyperlink to the Licensed Material to the
extent reasonably practicable;
b. indicate if You modified the Licensed Material and
retain an indication of any previous modifications; and
c. indicate the Licensed Material is licensed under this
Public License, and include the text of, or the URI or
hyperlink to, this Public License.
2. You may satisfy the conditions in Section 3(a)(1) in any
reasonable manner based on the medium, means, and context in
which You Share the Licensed Material. For example, it may be
reasonable to satisfy the conditions by providing a URI or
hyperlink to a resource that includes the required
information.
3. If requested by the Licensor, You must remove any of the
information required by Section 3(a)(1)(A) to the extent
reasonably practicable.
b. ShareAlike.
In addition to the conditions in Section 3(a), if You Share
Adapted Material You produce, the following conditions also apply.
1. The Adapter's License You apply must be a Creative Commons
license with the same License Elements, this version or
later, or a BY-SA Compatible License.
2. You must include the text of, or the URI or hyperlink to, the
Adapter's License You apply. You may satisfy this condition
in any reasonable manner based on the medium, means, and
context in which You Share Adapted Material.
3. You may not offer or impose any additional or different terms
or conditions on, or apply any Effective Technological
Measures to, Adapted Material that restrict exercise of the
rights granted under the Adapter's License You apply.
Section 4 -- Sui Generis Database Rights.
Where the Licensed Rights include Sui Generis Database Rights that
apply to Your use of the Licensed Material:
a. for the avoidance of doubt, Section 2(a)(1) grants You the right
to extract, reuse, reproduce, and Share all or a substantial
portion of the contents of the database;
b. if You include all or a substantial portion of the database
contents in a database in which You have Sui Generis Database
Rights, then the database in which You have Sui Generis Database
Rights (but not its individual contents) is Adapted Material,
including for purposes of Section 3(b); and
c. You must comply with the conditions in Section 3(a) if You Share
all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not
replace Your obligations under this Public License where the Licensed
Rights include other Copyright and Similar Rights.
Section 5 -- Disclaimer of Warranties and Limitation of Liability.
a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE
EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS
AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF
ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS,
IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION,
WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS,
ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT
KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT
ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU.
b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE
TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION,
NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT,
INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES,
COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR
USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN
ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR
DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR
IN PART, THIS LIMITATION MAY NOT APPLY TO YOU.
c. The disclaimer of warranties and limitation of liability provided
above shall be interpreted in a manner that, to the extent
possible, most closely approximates an absolute disclaimer and
waiver of all liability.
Section 6 -- Term and Termination.
a. This Public License applies for the term of the Copyright and
Similar Rights licensed here. However, if You fail to comply with
this Public License, then Your rights under this Public License
terminate automatically.
b. Where Your right to use the Licensed Material has terminated under
Section 6(a), it reinstates:
1. automatically as of the date the violation is cured, provided
it is cured within 30 days of Your discovery of the
violation; or
2. upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any
right the Licensor may have to seek remedies for Your violations
of this Public License.
c. For the avoidance of doubt, the Licensor may also offer the
Licensed Material under separate terms or conditions or stop
distributing the Licensed Material at any time; however, doing so
will not terminate this Public License.
d. Sections 1, 5, 6, 7, and 8 survive termination of this Public
License.
Section 7 -- Other Terms and Conditions.
a. The Licensor shall not be bound by any additional or different
terms or conditions communicated by You unless expressly agreed.
b. Any arrangements, understandings, or agreements regarding the
Licensed Material not stated herein are separate from and
independent of the terms and conditions of this Public License.
Section 8 -- Interpretation.
a. For the avoidance of doubt, this Public License does not, and
shall not be interpreted to, reduce, limit, restrict, or impose
conditions on any use of the Licensed Material that could lawfully
be made without permission under this Public License.
b. To the extent possible, if any provision of this Public License is
deemed unenforceable, it shall be automatically reformed to the
minimum extent necessary to make it enforceable. If the provision
cannot be reformed, it shall be severed from this Public License
without affecting the enforceability of the remaining terms and
conditions.
c. No term or condition of this Public License will be waived and no
failure to comply consented to unless expressly agreed to by the
Licensor.
d. Nothing in this Public License constitutes or may be interpreted
as a limitation upon, or waiver of, any privileges and immunities
that apply to the Licensor or You, including from the legal
processes of any jurisdiction or authority.
=======================================================================
Creative Commons is not a party to its public
licenses. Notwithstanding, Creative Commons may elect to apply one of
its public licenses to material it publishes and in those instances
will be considered the “Licensor.” The text of the Creative Commons
public licenses is dedicated to the public domain under the CC0 Public
Domain Dedication. Except for the limited purpose of indicating that
material is shared under a Creative Commons public license or as
otherwise permitted by the Creative Commons policies published at
creativecommons.org/policies, Creative Commons does not authorize the
use of the trademark "Creative Commons" or any other trademark or logo
of Creative Commons without its prior written consent including,
without limitation, in connection with any unauthorized modifications
to any of its public licenses or any other arrangements,
understandings, or agreements concerning use of licensed material. For
the avoidance of doubt, this paragraph does not form part of the
public licenses.
Creative Commons may be contacted at creativecommons.org.

153
nextflow/docs/Makefile Normal file
View File

@@ -0,0 +1,153 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
# the i18n builder cannot share the environment and doctrees with the others
I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml singlehtml pickle json htmlhelp qthelp devhelp epub latex latexpdf text man changes linkcheck doctest gettext
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " singlehtml to make a single large HTML file"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " devhelp to make HTML files and a Devhelp project"
@echo " epub to make an epub"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " latexpdf to make LaTeX files and run them through pdflatex"
@echo " text to make text files"
@echo " man to make manual pages"
@echo " texinfo to make Texinfo files"
@echo " info to make Texinfo files and run them through makeinfo"
@echo " gettext to make PO message catalogs"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
-rm -rf $(BUILDDIR)/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
singlehtml:
$(SPHINXBUILD) -b singlehtml $(ALLSPHINXOPTS) $(BUILDDIR)/singlehtml
@echo
@echo "Build finished. The HTML page is in $(BUILDDIR)/singlehtml."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/Blow.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/Blow.qhc"
devhelp:
$(SPHINXBUILD) -b devhelp $(ALLSPHINXOPTS) $(BUILDDIR)/devhelp
@echo
@echo "Build finished."
@echo "To view the help file:"
@echo "# mkdir -p $$HOME/.local/share/devhelp/Blow"
@echo "# ln -s $(BUILDDIR)/devhelp $$HOME/.local/share/devhelp/Blow"
@echo "# devhelp"
epub:
$(SPHINXBUILD) -b epub $(ALLSPHINXOPTS) $(BUILDDIR)/epub
@echo
@echo "Build finished. The epub file is in $(BUILDDIR)/epub."
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make' in that directory to run these through (pdf)latex" \
"(use \`make latexpdf' here to do that automatically)."
latexpdf:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo "Running LaTeX files through pdflatex..."
$(MAKE) -C $(BUILDDIR)/latex all-pdf
@echo "pdflatex finished; the PDF files are in $(BUILDDIR)/latex."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text files are in $(BUILDDIR)/text."
man:
$(SPHINXBUILD) -b man $(ALLSPHINXOPTS) $(BUILDDIR)/man
@echo
@echo "Build finished. The manual pages are in $(BUILDDIR)/man."
texinfo:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo
@echo "Build finished. The Texinfo files are in $(BUILDDIR)/texinfo."
@echo "Run \`make' in that directory to run these through makeinfo" \
"(use \`make info' here to do that automatically)."
info:
$(SPHINXBUILD) -b texinfo $(ALLSPHINXOPTS) $(BUILDDIR)/texinfo
@echo "Running Texinfo files through makeinfo..."
make -C $(BUILDDIR)/texinfo info
@echo "makeinfo finished; the Info files are in $(BUILDDIR)/texinfo."
gettext:
$(SPHINXBUILD) -b gettext $(I18NSPHINXOPTS) $(BUILDDIR)/locale
@echo
@echo "Build finished. The message catalogs are in $(BUILDDIR)/locale."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."

47
nextflow/docs/README.md Normal file
View File

@@ -0,0 +1,47 @@
# Nextflow Documentation
Nextflow documentation is written using [Sphinx](http://www.sphinx-doc.org/), [MyST](https://myst-parser.readthedocs.io/en/latest/) which is an extended version of Markdown for Sphinx, and the [Read The Docs theme for Sphinx](https://github.com/readthedocs/sphinx_rtd_theme).
## Dependencies
The most convenient approach is to create a Conda environment with Python 3.7 (other versions may work but haven't been tested).
The build dependencies can be installed with `pip`:
```bash
cd docs
pip install -r requirements.txt
```
Alternatively, you can use the Dockerfile to build the docs in a container (see below).
## Contributing
To edit and contribute to the documentation, you only need a text editor to change the appropriate `.md` files in this directory.
Once you have made your changes, run the following command to build the HTML files:
```bash
make clean html
```
Alternatively, you can use the Dockerfile to build the docs in a container:
```bash
docker build -t nextflow/sphinx:5.3.0 .
docker run -v $(pwd):/tmp nextflow/sphinx:5.3.0 -- make html
```
Then start up a local http server and open `localhost:8080` in your browser to verify the changes:
```bash
python -m http.server 8080 --directory _build/html/
```
## License
Nextflow documentation is distributed under
[Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license](https://creativecommons.org/licenses/by-sa/4.0/).

BIN
nextflow/docs/_static/.DS_Store vendored Normal file
View File

Binary file not shown.

39
nextflow/docs/_static/dag.mmd vendored Normal file
View File

@@ -0,0 +1,39 @@
%%{
init: {
'theme': 'base',
'themeVariables': {
'primaryColor': '#B6ECE2',
'primaryTextColor': '#160F26',
'primaryBorderColor': '#065647',
'lineColor': '#545555',
'clusterBkg': '#BABCBD22',
'clusterBorder': '#DDDEDE',
'fontFamily': 'arial'
}
}
}%%
flowchart TB
subgraph " "
v0["Channel.fromFilePairs"]
v1["transcriptome"]
v7["config"]
end
subgraph RNASEQ
v2([INDEX])
v3([FASTQC])
v4([QUANT])
end
v8([MULTIQC])
subgraph " "
v9[" "]
end
v5(( ))
v0 --> v3
v0 --> v4
v1 --> v2
v2 --> v4
v3 --> v5
v4 --> v5
v7 --> v8
v5 --> v8
v8 --> v9

BIN
nextflow/docs/_static/degular/Degular-Bold.woff vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-Bold.woff2 vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-BoldItalic.woff vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-BoldItalic.woff2 vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-Italic.woff vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-Italic.woff2 vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-Regular.woff vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/degular/Degular-Regular.woff2 vendored Normal file
View File

Binary file not shown.

BIN
nextflow/docs/_static/favicon.ico vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 9.4 KiB

BIN
nextflow/docs/_static/nextflow-k8s-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 33 KiB

BIN
nextflow/docs/_static/nextflow-logo-bg-dark.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 21 KiB

BIN
nextflow/docs/_static/nextflow-logo-bg-light.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

BIN
nextflow/docs/_static/nextflow-logo.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 25 KiB

BIN
nextflow/docs/_static/report-resource-cpu-noheader.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 172 KiB

BIN
nextflow/docs/_static/report-resource-cpu.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 200 KiB

BIN
nextflow/docs/_static/report-resource-io-read.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
nextflow/docs/_static/report-resource-io-write.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 16 KiB

BIN
nextflow/docs/_static/report-resource-job-duration.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 29 KiB

BIN
nextflow/docs/_static/report-resource-memory-pctram.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
nextflow/docs/_static/report-resource-memory-ram.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
nextflow/docs/_static/report-resource-memory-vmem.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 18 KiB

BIN
nextflow/docs/_static/report-resources-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 67 KiB

BIN
nextflow/docs/_static/report-summary-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 133 KiB

BIN
nextflow/docs/_static/report-tasks-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 104 KiB

16
nextflow/docs/_static/seqera-logo-dark.svg vendored Normal file
View File

@@ -0,0 +1,16 @@
<svg width="1479" height="500" viewBox="0 0 1479 500" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M337.199 256.961L337.549 257.099L337.899 257.238L338.229 257.396L338.559 257.575L338.88 257.753L339.2 257.951L339.5 258.178L339.79 258.396L340.06 258.643L340.34 258.891L340.6 259.168L340.84 259.435L341.08 259.732L341.29 260.029L341.5 260.345L341.68 260.652L341.85 260.989L342.01 261.315L342.15 261.652L342.29 261.998L342.39 262.354L342.49 262.701L342.56 263.077L342.63 263.433L342.67 263.809L342.69 264.165L342.7 264.531L342.69 264.908L342.67 265.274L342.63 265.64L342.56 266.006L342.49 266.362L342.39 266.728L342.29 267.075L342.15 267.431L342.01 267.768L341.85 268.094L341.68 268.421L341.5 268.737L341.29 269.054L341.08 269.351L340.84 269.628L340.6 269.915L340.34 270.172L340.06 270.439L339.79 270.687L339.5 270.905L339.2 271.112L338.88 271.33L338.559 271.508L338.229 271.686L337.899 271.835L337.549 271.983L337.199 272.102L336.849 272.221L336.489 272.3L336.129 272.389L335.749 272.438L335.389 272.478L335.009 272.518H334.949L335.329 272.508L336.009 272.478L336.689 272.438L337.369 272.379L338.039 272.29L338.719 272.191L339.39 272.082L340.06 271.944L340.72 271.795L341.39 271.627L342.05 271.439L342.7 271.231L343.341 271.013L343.981 270.776L344.611 270.519L345.241 270.251L345.861 269.964L346.471 269.658L347.072 269.341L347.662 269.005L348.242 268.648L348.822 268.292L349.382 267.916L349.942 267.51L350.482 267.095L351.013 266.679L351.533 266.234L352.043 265.778L352.533 265.323L353.023 264.838L353.493 264.353L353.943 263.849L354.383 263.334L354.813 262.81L355.234 262.265L355.634 261.721L356.014 261.167L356.394 260.593L356.744 260.019L357.084 259.425L357.414 258.831L357.714 258.238L358.004 257.624L358.274 257.001L358.534 256.377L358.774 255.744L359.005 255.1L359.205 254.457L359.395 253.804L359.565 253.151L359.715 252.498L359.855 251.825L359.975 251.162L360.075 250.499L360.145 249.826L360.215 249.153L360.265 248.47L360.285 247.797L360.295 247.124L360.285 246.451L360.265 245.768L360.215 245.096L360.145 244.423L360.075 243.76L359.975 243.087L359.855 242.414L359.715 241.761L359.565 241.097L359.435 240.593H335.119L334.638 240.603H277.124L336.779 256.842L337.199 256.961Z" fill="#201637"/>
<path d="M374.918 262.849L374.868 261.998L374.808 261.157L374.728 260.306L374.628 259.465L374.518 258.633L374.378 257.792L374.217 256.961L374.057 256.13L373.867 255.308L373.657 254.487L373.437 253.666L373.187 252.854L372.927 252.052L372.657 251.241L372.357 250.449L372.047 249.668L371.717 248.876L371.377 248.104L371.017 247.332L370.647 246.57L370.246 245.828L369.836 245.076L369.406 244.343L368.966 243.611L368.516 242.899L368.036 242.186L367.556 241.493L367.056 240.81L366.886 240.593H359.434L359.564 241.097L359.714 241.761L359.854 242.414L359.974 243.087L360.074 243.76L360.144 244.423L360.214 245.096L360.264 245.768L360.284 246.451L360.294 247.124L360.284 247.797L360.264 248.47L360.214 249.153L360.144 249.826L360.074 250.499L359.974 251.162L359.854 251.825L359.714 252.498L359.564 253.151L359.394 253.804L359.204 254.457L359.004 255.1L358.774 255.744L358.534 256.377L358.274 257.001L358.003 257.624L357.713 258.238L357.413 258.831L357.083 259.425L356.743 260.019L356.393 260.593L356.013 261.167L355.633 261.721L355.233 262.265L354.813 262.81L354.383 263.334L353.942 263.849L353.492 264.353L353.022 264.838L352.532 265.323L352.042 265.778L351.532 266.234L351.012 266.679L350.482 267.095L349.941 267.51L349.381 267.916L348.821 268.292L348.241 268.648L347.661 269.005L347.071 269.341L346.471 269.658L345.86 269.964L345.24 270.251L344.61 270.519L343.98 270.776L343.34 271.013L342.7 271.231L342.05 271.439L341.389 271.627L340.719 271.795L340.059 271.944L339.389 272.082L338.719 272.191L338.039 272.29L337.368 272.379L336.688 272.438L336.008 272.478L335.328 272.508L334.638 272.518H124.986V304.443H334.638L335.498 304.433L336.348 304.413L337.208 304.363L338.059 304.294L338.909 304.215L339.759 304.126L340.609 303.997L341.449 303.869L342.29 303.72L343.13 303.542L343.97 303.354L344.79 303.156L345.62 302.928L346.451 302.691L347.261 302.443L348.071 302.157L348.871 301.87L349.671 301.563L350.462 301.246L351.242 300.91L352.022 300.543L352.792 300.167L353.552 299.781L354.303 299.376L355.043 298.96L355.783 298.515L356.493 298.069L357.213 297.604L357.923 297.119L358.614 296.625L359.294 296.11L359.964 295.585L360.634 295.041L361.284 294.487L361.914 293.933L362.535 293.349L363.145 292.755L363.745 292.152L364.325 291.528L364.905 290.905L365.465 290.261L366.005 289.608L366.536 288.945L367.056 288.262L367.556 287.58L368.036 286.887L368.516 286.174L368.966 285.462L369.406 284.729L369.836 283.997L370.246 283.255L370.647 282.503L371.017 281.741L371.377 280.979L371.717 280.197L372.047 279.415L372.357 278.633L372.657 277.832L372.927 277.03L373.187 276.229L373.437 275.407L373.657 274.596L373.867 273.774L374.057 272.943L374.217 272.122L374.378 271.291L374.518 270.449L374.628 269.608L374.728 268.767L374.808 267.926L374.868 267.075L374.918 266.234L374.948 265.392L374.958 264.541L374.948 263.69L374.918 262.849Z" fill="#0DC09D"/>
<path d="M162.746 248.164L162.396 248.036L162.046 247.897L161.716 247.739L161.376 247.561L161.066 247.373L160.746 247.175L160.446 246.957L160.155 246.73L159.875 246.482L159.605 246.235L159.345 245.968L159.105 245.69L158.865 245.394L158.655 245.097L158.445 244.79L158.265 244.483L158.095 244.147L157.925 243.81L157.795 243.484L157.655 243.137L157.555 242.771L157.455 242.425L157.375 242.059L157.315 241.702L157.275 241.326L157.255 240.97L157.245 240.594L157.255 240.228L157.275 239.852L157.315 239.495L157.375 239.129L157.455 238.763L157.555 238.407L157.655 238.061L157.795 237.704L157.925 237.368L158.095 237.031L158.265 236.715L158.445 236.388L158.655 236.081L158.865 235.784L159.105 235.497L159.345 235.22L159.605 234.953L159.875 234.696L160.155 234.448L160.446 234.231L160.746 234.013L161.066 233.805L161.376 233.617L161.716 233.449L162.046 233.301L162.396 233.152L162.746 233.023L163.096 232.915L163.456 232.826L163.816 232.746L164.196 232.697L164.557 232.657L164.927 232.618H164.997H164.617L163.936 232.647L163.256 232.697L162.576 232.756L161.906 232.835L161.226 232.934L160.546 233.053L159.885 233.182L159.215 233.34L158.555 233.508L157.895 233.696L157.245 233.904L156.605 234.122L155.954 234.349L155.334 234.607L154.704 234.884L154.084 235.171L153.474 235.478L152.864 235.794L152.284 236.121L151.703 236.477L151.123 236.843L150.553 237.219L150.003 237.625L149.453 238.031L148.933 238.456L148.413 238.892L147.902 239.347L147.412 239.812L146.922 240.287L146.452 240.782L146.002 241.277L145.552 241.791L145.132 242.326L144.702 242.86L144.312 243.404L143.921 243.969L143.551 244.533L143.201 245.116L142.861 245.7L142.531 246.304L142.231 246.898L141.941 247.511L141.671 248.135L141.411 248.758L141.171 249.392L140.941 250.035L140.741 250.678L140.551 251.331L140.381 251.974L140.231 252.638L140.09 253.301L139.97 253.973L139.87 254.637L139.8 255.309L139.73 255.973L139.68 256.655L139.66 257.328L139.65 258.001L139.66 258.684L139.68 259.357L139.73 260.03L139.8 260.703L139.87 261.376L139.97 262.049L140.09 262.712L140.231 263.375L140.381 264.028L140.511 264.543H164.827L165.307 264.533H222.821L163.166 248.293L162.746 248.164Z" fill="#201637"/>
<path d="M125.026 242.285L125.066 243.136L125.136 243.977L125.216 244.818L125.316 245.659L125.426 246.501L125.566 247.332L125.716 248.173L125.886 249.004L126.076 249.826L126.286 250.647L126.506 251.468L126.756 252.28L127.016 253.081L127.286 253.883L127.586 254.685L127.897 255.466L128.227 256.258L128.567 257.03L128.927 257.802L129.297 258.554L129.697 259.306L130.107 260.058L130.537 260.791L130.977 261.513L131.427 262.225L131.898 262.938L132.388 263.641L132.888 264.323L133.058 264.541H140.51L140.38 264.027L140.23 263.373L140.09 262.71L139.97 262.047L139.87 261.374L139.8 260.701L139.729 260.029L139.679 259.356L139.659 258.683L139.649 258L139.659 257.327L139.679 256.654L139.729 255.971L139.8 255.308L139.87 254.635L139.97 253.972L140.09 253.299L140.23 252.636L140.38 251.973L140.55 251.33L140.74 250.677L140.94 250.033L141.17 249.39L141.41 248.757L141.67 248.133L141.94 247.51L142.23 246.896L142.53 246.303L142.86 245.699L143.2 245.115L143.55 244.531L143.921 243.967L144.311 243.403L144.701 242.859L145.131 242.324L145.551 241.79L146.001 241.275L146.451 240.781L146.921 240.286L147.411 239.811L147.902 239.346L148.412 238.89L148.932 238.455L149.452 238.029L150.002 237.624L150.552 237.218L151.122 236.842L151.702 236.476L152.283 236.119L152.863 235.793L153.473 235.476L154.083 235.169L154.703 234.882L155.333 234.605L155.953 234.348L156.604 234.12L157.244 233.903L157.894 233.695L158.554 233.507L159.214 233.339L159.884 233.18L160.545 233.052L161.225 232.933L161.905 232.834L162.575 232.755L163.255 232.695L163.935 232.646L164.616 232.616L165.306 232.606H374.958V200.691H165.306L164.446 200.701L163.595 200.721L162.735 200.771L161.885 200.83L161.025 200.909L160.185 201.008L159.334 201.127L158.494 201.255L157.654 201.414L156.804 201.582L155.974 201.77L155.143 201.978L154.323 202.196L153.493 202.443L152.683 202.69L151.872 202.968L151.072 203.254L150.272 203.571L149.482 203.888L148.702 204.224L147.922 204.59L147.151 204.967L146.391 205.343L145.641 205.758L144.901 206.174L144.161 206.609L143.44 207.064L142.72 207.53L142.02 208.015L141.33 208.509L140.64 209.014L139.98 209.539L139.309 210.083L138.659 210.637L138.029 211.201L137.409 211.785L136.799 212.369L136.199 212.982L135.608 213.606L135.038 214.229L134.478 214.873L133.938 215.526L133.408 216.189L132.888 216.862L132.388 217.554L131.898 218.247L131.427 218.95L130.977 219.672L130.537 220.404L130.107 221.137L129.697 221.869L129.297 222.621L128.927 223.393L128.567 224.155L128.227 224.937L127.897 225.709L127.586 226.5L127.286 227.292L127.016 228.094L126.756 228.905L126.506 229.717L126.286 230.538L126.076 231.359L125.886 232.191L125.716 233.012L125.566 233.843L125.426 234.685L125.316 235.526L125.216 236.367L125.136 237.208L125.066 238.049L125.026 238.89L124.986 239.741V240.593V241.444L125.026 242.285Z" fill="#0DC09D"/>
<path d="M162.745 176.346L162.395 176.208L162.045 176.059L161.705 175.911L161.385 175.733L161.054 175.555L160.744 175.357L160.444 175.129L160.144 174.911L159.864 174.664L159.594 174.417L159.344 174.139L159.104 173.872L158.864 173.575L158.644 173.278L158.444 172.962L158.254 172.645L158.084 172.319L157.914 171.992L157.784 171.655L157.654 171.299L157.544 170.953L157.454 170.606L157.374 170.23L157.314 169.874L157.264 169.498L157.254 169.132L157.244 168.756L157.254 168.39L157.264 168.034L157.314 167.667L157.374 167.301L157.454 166.945L157.544 166.579L157.654 166.223L157.784 165.876L157.914 165.54L158.084 165.213L158.254 164.877L158.444 164.57L158.644 164.253L158.864 163.956L159.104 163.679L159.344 163.392L159.594 163.115L159.864 162.868L160.144 162.62L160.444 162.403L160.744 162.195L161.054 161.977L161.385 161.799L161.705 161.621L162.045 161.462L162.395 161.324L162.745 161.205L163.095 161.086L163.455 160.987L163.825 160.918L164.185 160.859L164.555 160.819L164.935 160.79H165.055H164.615L163.935 160.819L163.255 160.869L162.575 160.928L161.905 161.017L161.225 161.116L160.544 161.225L159.884 161.364L159.214 161.512L158.554 161.68L157.894 161.868L157.244 162.076L156.603 162.294L155.953 162.531L155.333 162.779L154.703 163.056L154.083 163.343L153.473 163.65L152.862 163.966L152.282 164.293L151.692 164.649L151.122 165.015L150.552 165.391L150.002 165.797L149.452 166.203L148.921 166.628L148.411 167.074L147.901 167.519L147.401 167.984L146.921 168.459L146.451 168.954L146.001 169.459L145.551 169.973L145.131 170.498L144.7 171.042L144.31 171.586L143.92 172.14L143.55 172.704L143.2 173.288L142.86 173.872L142.53 174.476L142.23 175.07L141.94 175.683L141.67 176.307L141.41 176.93L141.17 177.563L140.939 178.207L140.739 178.85L140.549 179.503L140.379 180.156L140.219 180.809L140.089 181.472L139.969 182.145L139.869 182.808L139.789 183.481L139.729 184.144L139.679 184.827L139.659 185.51L139.649 186.183L139.659 186.856L139.679 187.539L139.729 188.212L139.789 188.885L139.869 189.548L139.969 190.221L140.089 190.894L140.219 191.547L140.379 192.21L140.549 192.863L140.739 193.506L140.939 194.159L141.17 194.793L141.41 195.426L141.67 196.049L141.94 196.673L142.23 197.286L142.53 197.89L142.73 198.236L142.75 198.227L143.59 197.801L144.43 197.395L145.281 196.999L146.141 196.633L147.001 196.257L147.881 195.921L148.751 195.604L149.642 195.278L150.532 195L151.432 194.723L152.332 194.466L153.233 194.219L154.143 193.991L155.063 193.793L155.983 193.615L156.893 193.437L157.824 193.288L158.754 193.15L159.684 193.031L160.624 192.932L161.555 192.853L162.495 192.784L163.415 192.744L164.375 192.724L165.306 192.705H222.82L163.155 176.465L162.745 176.346Z" fill="#201637"/>
<path d="M334.637 344.364H335.327L336.008 344.325L336.688 344.285L337.368 344.226L338.048 344.137L338.728 344.038L339.398 343.929L340.069 343.79L340.729 343.642L341.399 343.474L342.049 343.285L342.699 343.078L343.349 342.86L343.99 342.622L344.61 342.365L345.24 342.098L345.86 341.811L346.47 341.504L347.08 341.188L347.67 340.851L348.251 340.495L348.821 340.129L349.391 339.753L349.951 339.357L350.491 338.941L351.021 338.525L351.541 338.08L352.042 337.625L352.542 337.17L353.022 336.685L353.492 336.2L353.952 335.695L354.392 335.181L354.822 334.656L355.242 334.112L355.632 333.568L356.023 333.013L356.393 332.439L356.743 331.865L357.083 331.272L357.413 330.678L357.723 330.074L358.013 329.471L358.273 328.847L358.543 328.214L358.773 327.59L359.003 326.947L359.213 326.304L359.393 325.651L359.573 324.997L359.723 324.344L359.853 323.671L359.974 323.008L360.074 322.345L360.154 321.662L360.224 320.999L360.264 320.317L360.294 319.644V318.971V318.298L360.264 317.615L360.224 316.942L360.154 316.269L360.074 315.606L359.974 314.933L359.853 314.26L359.723 313.607L359.573 312.944L359.393 312.291L359.213 311.648L359.003 310.994L358.773 310.351L358.543 309.728L358.273 309.104L358.013 308.481L357.723 307.867L357.413 307.264L357.213 306.897L357.193 306.907L356.353 307.333L355.512 307.739L354.662 308.134L353.802 308.511L352.932 308.867L352.062 309.213L351.191 309.53L350.301 309.847L349.401 310.143L348.511 310.411L347.62 310.668L346.7 310.915L345.8 311.133L344.88 311.341L343.96 311.529L343.039 311.697L342.119 311.846L341.189 311.984L340.259 312.103L339.328 312.202L338.388 312.281L337.448 312.34L336.508 312.39L335.577 312.419L334.637 312.429H277.123L336.778 328.669L337.198 328.798L337.548 328.926L337.898 329.065L338.228 329.223L338.558 329.401L338.878 329.589L339.198 329.787L339.498 330.005L339.789 330.233L340.059 330.48L340.339 330.727L340.599 330.995L340.839 331.272L341.079 331.568L341.289 331.865L341.499 332.172L341.679 332.479L341.849 332.815L342.009 333.152L342.149 333.478L342.289 333.825L342.389 334.191L342.489 334.537L342.559 334.903L342.629 335.26L342.669 335.636L342.689 335.992L342.699 336.368L342.689 336.734L342.669 337.11L342.629 337.467L342.559 337.833L342.489 338.199L342.389 338.555L342.289 338.902L342.149 339.258L342.009 339.594L341.849 339.931L341.679 340.247L341.499 340.574L341.289 340.881L341.079 341.178L340.839 341.465L340.599 341.742L340.339 342.009L340.059 342.266L339.789 342.514L339.498 342.731L339.198 342.949L338.878 343.157L338.558 343.345L338.228 343.513L337.898 343.662L337.548 343.81L337.198 343.939L336.848 344.047L336.488 344.137L336.128 344.216L335.748 344.265L335.387 344.305L335.007 344.344L334.637 344.354" fill="#201637"/>
<path d="M374.918 334.684L374.878 333.843L374.818 333.002L374.728 332.151L374.638 331.31L374.518 330.478L374.378 329.637L374.227 328.806L374.057 327.975L373.867 327.143L373.657 326.332L373.437 325.511L373.197 324.689L372.937 323.898L372.657 323.086L372.357 322.294L372.057 321.513L371.727 320.721L371.377 319.949L371.027 319.177L370.647 318.415L370.246 317.673L369.846 316.921L369.416 316.188L368.966 315.456L368.516 314.744L368.046 314.031L367.556 313.338L367.056 312.656L366.536 311.973L366.015 311.32L365.465 310.657L364.905 310.023L364.335 309.39L363.745 308.776L363.155 308.163L362.535 307.579L361.914 306.995L361.284 306.431L360.634 305.877L359.974 305.332L359.934 305.303L358.994 305.867L358.163 306.332L357.303 306.797L357.193 306.856L357.413 307.262L357.723 307.866L358.013 308.479L358.274 309.103L358.544 309.726L358.774 310.35L359.004 310.993L359.214 311.646L359.394 312.289L359.574 312.943L359.724 313.606L359.854 314.259L359.974 314.932L360.074 315.605L360.154 316.268L360.224 316.941L360.264 317.614L360.294 318.296V318.969V319.642L360.264 320.315L360.224 320.998L360.154 321.661L360.074 322.344L359.974 323.007L359.854 323.67L359.724 324.343L359.574 324.996L359.394 325.649L359.214 326.302L359.004 326.946L358.774 327.589L358.544 328.212L358.274 328.846L358.013 329.469L357.723 330.073L357.413 330.676L357.083 331.27L356.743 331.864L356.393 332.438L356.023 333.012L355.633 333.566L355.243 334.11L354.823 334.655L354.393 335.179L353.952 335.694L353.492 336.198L353.022 336.683L352.542 337.168L352.042 337.623L351.542 338.079L351.022 338.524L350.492 338.94L349.951 339.355L349.391 339.751L348.821 340.127L348.251 340.493L347.671 340.85L347.081 341.186L346.471 341.503L345.86 341.81L345.24 342.097L344.61 342.364L343.99 342.621L343.35 342.859L342.7 343.076L342.05 343.284L341.399 343.472L340.729 343.64L340.069 343.789L339.399 343.927L338.729 344.036L338.049 344.135L337.368 344.224L336.688 344.284L336.008 344.323L335.328 344.363H334.638H124.986V376.288H334.638L335.498 376.278L336.348 376.258L337.208 376.208L338.059 376.139L338.919 376.07L339.769 375.971L340.619 375.842L341.459 375.714L342.3 375.565L343.14 375.387L343.97 375.199L344.8 375.001L345.62 374.783L346.451 374.536L347.261 374.289L348.071 374.002L348.881 373.715L349.681 373.408L350.472 373.091L351.252 372.755L352.032 372.389L352.792 372.012L353.562 371.627L354.303 371.221L355.053 370.805L355.783 370.37L356.503 369.905L357.223 369.449L357.923 368.964L358.614 368.46L359.304 367.955L359.974 367.431L360.634 366.886L361.284 366.332L361.914 365.778L362.535 365.194L363.155 364.6L363.745 363.997L364.335 363.373L364.905 362.75L365.465 362.106L366.015 361.453L366.536 360.79L367.056 360.117L367.556 359.425L368.046 358.732L368.516 358.019L368.966 357.307L369.416 356.575L369.846 355.842L370.246 355.1L370.647 354.348L371.027 353.586L371.377 352.824L371.727 352.042L372.057 351.26L372.357 350.479L372.657 349.677L372.937 348.875L373.197 348.064L373.437 347.252L373.657 346.441L373.867 345.61L374.057 344.788L374.227 343.967L374.378 343.126L374.518 342.294L374.638 341.453L374.728 340.612L374.818 339.771L374.878 338.92L374.918 338.079L374.958 337.238V336.386V335.535L374.918 334.684Z" fill="#0DC09D"/>
<path d="M125.026 170.459L125.066 171.31L125.126 172.151L125.216 173.002L125.316 173.844L125.426 174.675L125.566 175.516L125.716 176.347L125.886 177.179L126.076 178L126.286 178.821L126.506 179.633L126.746 180.454L127.016 181.256L127.286 182.057L127.586 182.859L127.887 183.641L128.217 184.432L128.567 185.204L128.927 185.976L129.297 186.728L129.697 187.48L130.107 188.233L130.527 188.965L130.977 189.697L131.427 190.4L131.898 191.112L132.388 191.815L132.888 192.498L133.408 193.181L133.938 193.834L134.478 194.497L135.038 195.13L135.608 195.763L136.199 196.377L136.799 196.981L137.409 197.574L138.029 198.158L138.659 198.722L139.309 199.277L139.98 199.811L140.01 199.841L140.95 199.286L141.78 198.821L142.64 198.346L142.76 198.297L142.53 197.891L142.23 197.287L141.94 196.674L141.67 196.05L141.41 195.427L141.17 194.794L140.94 194.16L140.74 193.507L140.55 192.864L140.38 192.211L140.22 191.548L140.09 190.895L139.97 190.222L139.87 189.549L139.79 188.886L139.729 188.213L139.679 187.54L139.659 186.857L139.649 186.184L139.659 185.511L139.679 184.828L139.729 184.145L139.79 183.482L139.87 182.809L139.97 182.146L140.09 181.473L140.22 180.81L140.38 180.157L140.55 179.504L140.74 178.851L140.94 178.208L141.17 177.565L141.41 176.931L141.67 176.308L141.94 175.684L142.23 175.071L142.53 174.477L142.86 173.873L143.2 173.289L143.55 172.706L143.921 172.141L144.311 171.587L144.701 171.043L145.131 170.499L145.551 169.974L146.001 169.46L146.451 168.955L146.921 168.46L147.401 167.985L147.902 167.52L148.412 167.075L148.922 166.629L149.452 166.204L150.002 165.798L150.552 165.392L151.122 165.016L151.692 164.65L152.283 164.294L152.863 163.967L153.473 163.651L154.083 163.344L154.703 163.057L155.333 162.78L155.953 162.532L156.604 162.295L157.244 162.077L157.894 161.869L158.554 161.681L159.214 161.513L159.884 161.365L160.545 161.226L161.225 161.117L161.905 161.018L162.575 160.929L163.255 160.87L163.935 160.82L164.616 160.791H165.306H374.958V128.866H165.306H164.446L163.595 128.895L162.735 128.945L161.885 129.014L161.025 129.083L160.175 129.182L159.334 129.311L158.484 129.43L157.644 129.588L156.804 129.756L155.974 129.954L155.143 130.152L154.323 130.37L153.493 130.617L152.683 130.865L151.872 131.142L151.072 131.429L150.262 131.745L149.482 132.062L148.702 132.399L147.922 132.765L147.151 133.131L146.391 133.527L145.641 133.933L144.901 134.348L144.161 134.784L143.44 135.239L142.72 135.704L142.02 136.189L141.33 136.684L140.64 137.198L139.98 137.723L139.309 138.267L138.659 138.811L138.029 139.375L137.409 139.959L136.799 140.553L136.199 141.157L135.608 141.77L135.038 142.404L134.478 143.047L133.938 143.7L133.408 144.363L132.888 145.036L132.388 145.729L131.898 146.421L131.427 147.124L130.977 147.846L130.527 148.569L130.107 149.311L129.697 150.053L129.297 150.796L128.927 151.567L128.567 152.329L128.217 153.111L127.887 153.893L127.586 154.675L127.286 155.466L127.016 156.278L126.746 157.08L126.506 157.891L126.286 158.712L126.076 159.534L125.886 160.365L125.716 161.186L125.566 162.018L125.426 162.859L125.316 163.7L125.216 164.541L125.126 165.382L125.066 166.233L125.026 167.075L124.986 167.916V168.767V169.618L125.026 170.459Z" fill="#0DC09D"/>
<path d="M516.784 329.897H453.084V305.262H516.784C526.645 305.262 534.243 303.388 539.576 299.619C544.909 295.86 547.581 291.007 547.581 285.071C547.581 279.134 545.058 274.78 540.033 272.007C534.998 269.244 527.549 266.959 517.687 265.182L507.418 263.405C497.348 261.628 488.191 259.05 479.948 255.692C471.695 252.333 465.15 247.675 460.324 241.739C455.497 235.803 453.074 228.187 453.074 218.882C453.074 205.037 458.308 194.297 468.775 186.681C479.243 179.065 493.127 175.257 510.437 175.257H572.627V200.194H510.437C501.986 200.194 495.342 201.727 490.515 204.793C485.689 207.859 483.265 212.165 483.265 217.701C483.265 223.637 485.629 227.992 490.356 230.765C495.084 233.537 501.479 235.617 509.524 236.994L520.091 238.771C530.757 240.548 540.519 243.028 549.378 246.191C558.237 249.355 565.229 253.905 570.363 259.841C575.498 265.778 578.06 273.696 578.06 283.587C578.06 298.232 572.518 309.607 561.455 317.721C550.381 325.835 535.494 329.887 516.774 329.887L516.784 329.897Z" fill="#201637"/>
<path d="M892.291 377.653V307.648H887.432C885.202 311.665 881.963 315.525 877.704 319.226C873.445 322.937 868.026 325.995 861.447 328.4C854.858 330.804 846.71 332.012 836.982 332.012C824.414 332.012 812.866 329.003 802.328 322.987C791.79 316.97 783.382 308.301 777.103 296.96C770.824 285.629 767.685 271.942 767.685 255.891V251.378C767.685 235.337 770.874 221.65 777.263 210.309C783.642 198.978 792.1 190.299 802.638 184.283C813.176 178.266 824.624 175.257 836.982 175.257C851.569 175.257 862.767 177.87 870.565 183.075C878.364 188.291 884.193 194.208 888.042 200.829H892.901V175.257H923.605V377.633H892.301L892.291 377.653ZM845.79 304.936C859.567 304.936 870.815 300.621 879.523 292.002C888.232 283.382 892.591 271.042 892.591 255V252.289C892.591 236.445 888.182 224.213 879.373 215.584C870.555 206.964 859.357 202.65 845.79 202.65C832.223 202.65 821.325 206.964 812.506 215.584C803.688 224.213 799.289 236.445 799.289 252.289V255C799.289 271.052 803.698 283.382 812.506 292.002C821.325 300.631 832.413 304.936 845.79 304.936Z" fill="#201637"/>
<path d="M744.698 249.35C744.698 234.016 741.685 220.629 735.66 209.18C729.636 197.73 721.242 188.782 710.51 182.345C699.759 175.909 687.253 172.681 673.003 172.681C658.754 172.681 645.485 175.95 634.446 182.497C628.916 185.765 623.991 189.669 619.651 194.229C619.106 194.794 618.571 195.379 618.036 195.964C616.45 197.74 614.974 199.596 613.567 201.533C611.704 204.125 609.97 206.879 608.384 209.795C602.062 221.447 598.891 235.146 598.891 250.893V254.575C598.891 270.323 602.052 284.012 608.384 295.674C614.706 307.326 623.496 316.375 634.743 322.811C645.98 329.247 659.14 332.475 674.202 332.475C687.461 332.475 698.5 330.326 707.339 326.039C716.179 321.741 723.204 316.425 728.427 310.09C729.873 308.324 731.241 306.589 732.519 304.874C735.849 300.425 738.604 296.138 740.774 291.992L714.871 278.494C711.66 285.455 707.082 291.588 701.166 296.895C695.23 302.211 686.45 304.864 674.807 304.864C662.351 304.864 651.956 300.92 643.632 293.061C643.117 292.567 642.611 292.083 642.126 291.558C641.63 291.033 641.164 290.519 640.699 289.994C639.589 288.693 638.558 287.331 637.607 285.899C633.554 279.705 631.235 272.33 630.65 263.786C630.591 262.969 630.561 262.141 630.532 261.314H744.708V249.35H744.698ZM651.312 237.083L635.447 253.234H630.224C630.145 248.583 630.224 242.611 630.829 237.083C631.126 234.985 631.523 232.957 632.038 231.01C632.533 229.063 633.138 227.207 633.841 225.431C636.18 219.51 639.658 214.496 644.236 210.4C651.559 203.853 661.043 200.585 672.706 200.585C684.369 200.585 693.793 203.853 701.027 210.4C708.261 216.947 712.373 225.845 713.374 237.083H651.322H651.312Z" fill="#201637"/>
<path d="M1093.62 249.35C1093.62 234.016 1090.6 220.629 1084.58 209.18C1078.55 197.73 1070.16 188.782 1059.43 182.345C1048.68 175.909 1036.17 172.681 1021.92 172.681C1007.67 172.681 994.404 175.95 983.365 182.497C977.835 185.765 972.91 189.669 968.57 194.229C968.025 194.794 967.49 195.379 966.955 195.964C965.369 197.74 963.893 199.596 962.486 201.533C960.623 204.125 958.889 206.879 957.303 209.795C950.981 221.447 947.81 235.146 947.81 250.893V254.575C947.81 270.323 950.971 284.012 957.303 295.674C963.625 307.326 972.415 316.375 983.662 322.811C994.899 329.247 1008.06 332.475 1023.12 332.475C1036.38 332.475 1047.42 330.326 1056.26 326.039C1065.1 321.741 1072.12 316.425 1077.35 310.09C1078.79 308.324 1080.16 306.589 1081.44 304.874C1084.77 300.425 1087.52 296.138 1089.69 291.992L1063.79 278.494C1060.58 285.455 1056 291.588 1050.08 296.895C1044.15 302.211 1035.37 304.864 1023.73 304.864C1011.27 304.864 1000.87 300.92 992.551 293.061C992.036 292.567 991.53 292.083 991.045 291.558C990.549 291.033 990.083 290.519 989.618 289.994C988.508 288.693 987.477 287.331 986.526 285.899C982.473 279.705 980.154 272.33 979.569 263.786C979.51 262.969 979.48 262.141 979.451 261.314H1093.63V249.35H1093.62ZM1000.23 237.083L984.366 253.234H979.143C979.064 248.583 979.143 242.611 979.748 237.083C980.045 234.985 980.441 232.957 980.957 231.01C981.452 229.063 982.057 227.207 982.76 225.431C985.099 219.51 988.577 214.496 993.155 210.4C1000.48 203.853 1009.96 200.585 1021.62 200.585C1033.29 200.585 1042.71 203.853 1049.95 210.4C1057.18 216.947 1061.29 225.845 1062.29 237.083H1000.24H1000.23Z" fill="#201637"/>
<path d="M1124.88 330.029V173.285H1155.58V190.732H1160.44C1162.87 184.517 1166.77 179.955 1172.13 177.045C1177.5 174.136 1184.14 172.681 1192.04 172.681H1209.98V200.657H1190.83C1180.69 200.657 1172.39 203.418 1165.91 208.931C1159.42 214.453 1156.18 222.924 1156.18 234.354V330.019H1124.87L1124.88 330.029Z" fill="#201637"/>
<path d="M1354.01 230.335C1354.01 211.934 1348.37 197.721 1337.05 187.707C1325.75 177.693 1310.23 172.681 1290.48 172.681C1277.53 172.681 1266.58 174.72 1257.63 178.818C1248.69 182.916 1241.49 188.33 1236.05 195.07C1230.6 201.819 1226.64 209.182 1224.17 217.147L1253.77 226.659C1255.62 218.483 1259.43 211.834 1265.19 206.721C1270.96 201.619 1279.27 199.057 1290.17 199.057C1301.07 199.057 1309.5 201.769 1314.84 207.183C1320.2 212.597 1322.86 219.608 1322.86 228.196V254.351H1317.73L1301.28 238.009H1276.6C1259.95 238.009 1246.12 241.946 1235.12 249.821C1224.12 257.686 1218.61 269.297 1218.61 284.634C1218.61 294.859 1221.14 303.538 1226.17 310.699C1231.21 317.861 1238 323.274 1246.52 326.951C1255.07 330.627 1264.78 332.475 1275.66 332.475C1286.54 332.475 1294.58 330.988 1300.95 328.025C1307.33 325.062 1312.17 321.587 1315.45 317.599C1318.75 313.612 1321.2 309.976 1322.85 306.712H1327.79V329.898L1354 329.898V230.345L1354.01 230.335ZM1322.86 265.6C1322.86 278.286 1318.9 288.25 1310.98 295.502C1303.07 302.754 1292.85 306.39 1280.3 306.39C1271.05 306.39 1263.75 304.351 1258.4 300.253C1253.05 296.155 1250.38 290.641 1250.38 283.69C1250.38 276.739 1252.96 271.466 1258.09 267.901C1263.24 264.325 1270.01 262.537 1278.44 262.537H1322.85V265.6H1322.86Z" fill="#201637"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

16
nextflow/docs/_static/seqera-logo-light.svg vendored Normal file
View File

@@ -0,0 +1,16 @@
<svg width="1479" height="500" viewBox="0 0 1479 500" fill="none" xmlns="http://www.w3.org/2000/svg">
<path d="M337.199 256.961L337.549 257.099L337.899 257.238L338.229 257.396L338.559 257.575L338.88 257.753L339.2 257.951L339.5 258.178L339.79 258.396L340.06 258.643L340.34 258.891L340.6 259.168L340.84 259.435L341.08 259.732L341.29 260.029L341.5 260.345L341.68 260.652L341.85 260.989L342.01 261.315L342.15 261.652L342.29 261.998L342.39 262.354L342.49 262.701L342.56 263.077L342.63 263.433L342.67 263.809L342.69 264.165L342.7 264.531L342.69 264.908L342.67 265.274L342.63 265.64L342.56 266.006L342.49 266.362L342.39 266.728L342.29 267.075L342.15 267.431L342.01 267.768L341.85 268.094L341.68 268.421L341.5 268.737L341.29 269.054L341.08 269.351L340.84 269.628L340.6 269.915L340.34 270.172L340.06 270.439L339.79 270.687L339.5 270.905L339.2 271.112L338.88 271.33L338.559 271.508L338.229 271.686L337.899 271.835L337.549 271.983L337.199 272.102L336.849 272.221L336.489 272.3L336.129 272.389L335.749 272.438L335.389 272.478L335.009 272.518H334.949L335.329 272.508L336.009 272.478L336.689 272.438L337.369 272.379L338.039 272.29L338.719 272.191L339.39 272.082L340.06 271.944L340.72 271.795L341.39 271.627L342.05 271.439L342.7 271.231L343.341 271.013L343.981 270.776L344.611 270.519L345.241 270.251L345.861 269.964L346.471 269.658L347.072 269.341L347.662 269.005L348.242 268.648L348.822 268.292L349.382 267.916L349.942 267.51L350.482 267.095L351.013 266.679L351.533 266.234L352.043 265.778L352.533 265.323L353.023 264.838L353.493 264.353L353.943 263.849L354.383 263.334L354.813 262.81L355.234 262.265L355.634 261.721L356.014 261.167L356.394 260.593L356.744 260.019L357.084 259.425L357.414 258.831L357.714 258.238L358.004 257.624L358.274 257.001L358.534 256.377L358.774 255.744L359.005 255.1L359.205 254.457L359.395 253.804L359.565 253.151L359.715 252.498L359.855 251.825L359.975 251.162L360.075 250.499L360.145 249.826L360.215 249.153L360.265 248.47L360.285 247.797L360.295 247.124L360.285 246.451L360.265 245.768L360.215 245.096L360.145 244.423L360.075 243.76L359.975 243.087L359.855 242.414L359.715 241.761L359.565 241.097L359.435 240.593H335.119L334.638 240.603H277.124L336.779 256.842L337.199 256.961Z" fill="white"/>
<path d="M374.918 262.849L374.868 261.998L374.808 261.157L374.728 260.306L374.628 259.465L374.518 258.633L374.378 257.792L374.217 256.961L374.057 256.13L373.867 255.308L373.657 254.487L373.437 253.666L373.187 252.854L372.927 252.052L372.657 251.241L372.357 250.449L372.047 249.668L371.717 248.876L371.377 248.104L371.017 247.332L370.647 246.57L370.246 245.828L369.836 245.076L369.406 244.343L368.966 243.611L368.516 242.899L368.036 242.186L367.556 241.493L367.056 240.81L366.886 240.593H359.434L359.564 241.097L359.714 241.761L359.854 242.414L359.974 243.087L360.074 243.76L360.144 244.423L360.214 245.096L360.264 245.768L360.284 246.451L360.294 247.124L360.284 247.797L360.264 248.47L360.214 249.153L360.144 249.826L360.074 250.499L359.974 251.162L359.854 251.825L359.714 252.498L359.564 253.151L359.394 253.804L359.204 254.457L359.004 255.1L358.774 255.744L358.534 256.377L358.274 257.001L358.003 257.624L357.713 258.238L357.413 258.831L357.083 259.425L356.743 260.019L356.393 260.593L356.013 261.167L355.633 261.721L355.233 262.265L354.813 262.81L354.383 263.334L353.942 263.849L353.492 264.353L353.022 264.838L352.532 265.323L352.042 265.778L351.532 266.234L351.012 266.679L350.482 267.095L349.941 267.51L349.381 267.916L348.821 268.292L348.241 268.648L347.661 269.005L347.071 269.341L346.471 269.658L345.86 269.964L345.24 270.251L344.61 270.519L343.98 270.776L343.34 271.013L342.7 271.231L342.05 271.439L341.389 271.627L340.719 271.795L340.059 271.944L339.389 272.082L338.719 272.191L338.039 272.29L337.368 272.379L336.688 272.438L336.008 272.478L335.328 272.508L334.638 272.518H124.986V304.443H334.638L335.498 304.433L336.348 304.413L337.208 304.363L338.059 304.294L338.909 304.215L339.759 304.126L340.609 303.997L341.449 303.869L342.29 303.72L343.13 303.542L343.97 303.354L344.79 303.156L345.62 302.928L346.451 302.691L347.261 302.443L348.071 302.157L348.871 301.87L349.671 301.563L350.462 301.246L351.242 300.91L352.022 300.543L352.792 300.167L353.552 299.781L354.303 299.376L355.043 298.96L355.783 298.515L356.493 298.069L357.213 297.604L357.923 297.119L358.614 296.625L359.294 296.11L359.964 295.585L360.634 295.041L361.284 294.487L361.914 293.933L362.535 293.349L363.145 292.755L363.745 292.152L364.325 291.528L364.905 290.905L365.465 290.261L366.005 289.608L366.536 288.945L367.056 288.262L367.556 287.58L368.036 286.887L368.516 286.174L368.966 285.462L369.406 284.729L369.836 283.997L370.246 283.255L370.647 282.503L371.017 281.741L371.377 280.979L371.717 280.197L372.047 279.415L372.357 278.633L372.657 277.832L372.927 277.03L373.187 276.229L373.437 275.407L373.657 274.596L373.867 273.774L374.057 272.943L374.217 272.122L374.378 271.291L374.518 270.449L374.628 269.608L374.728 268.767L374.808 267.926L374.868 267.075L374.918 266.234L374.948 265.392L374.958 264.541L374.948 263.69L374.918 262.849Z" fill="#0DC09D"/>
<path d="M162.746 248.164L162.396 248.036L162.046 247.897L161.716 247.739L161.376 247.561L161.066 247.373L160.746 247.175L160.446 246.957L160.155 246.73L159.875 246.482L159.605 246.235L159.345 245.968L159.105 245.69L158.865 245.394L158.655 245.097L158.445 244.79L158.265 244.483L158.095 244.147L157.925 243.81L157.795 243.484L157.655 243.137L157.555 242.771L157.455 242.425L157.375 242.059L157.315 241.702L157.275 241.326L157.255 240.97L157.245 240.594L157.255 240.228L157.275 239.852L157.315 239.495L157.375 239.129L157.455 238.763L157.555 238.407L157.655 238.061L157.795 237.704L157.925 237.368L158.095 237.031L158.265 236.715L158.445 236.388L158.655 236.081L158.865 235.784L159.105 235.497L159.345 235.22L159.605 234.953L159.875 234.696L160.155 234.448L160.446 234.231L160.746 234.013L161.066 233.805L161.376 233.617L161.716 233.449L162.046 233.301L162.396 233.152L162.746 233.023L163.096 232.915L163.456 232.826L163.816 232.746L164.196 232.697L164.557 232.657L164.927 232.618H164.997H164.617L163.936 232.647L163.256 232.697L162.576 232.756L161.906 232.835L161.226 232.934L160.546 233.053L159.885 233.182L159.215 233.34L158.555 233.508L157.895 233.696L157.245 233.904L156.605 234.122L155.954 234.349L155.334 234.607L154.704 234.884L154.084 235.171L153.474 235.478L152.864 235.794L152.284 236.121L151.703 236.477L151.123 236.843L150.553 237.219L150.003 237.625L149.453 238.031L148.933 238.456L148.413 238.892L147.902 239.347L147.412 239.812L146.922 240.287L146.452 240.782L146.002 241.277L145.552 241.791L145.132 242.326L144.702 242.86L144.312 243.404L143.921 243.969L143.551 244.533L143.201 245.116L142.861 245.7L142.531 246.304L142.231 246.898L141.941 247.511L141.671 248.135L141.411 248.758L141.171 249.392L140.941 250.035L140.741 250.678L140.551 251.331L140.381 251.974L140.231 252.638L140.09 253.301L139.97 253.973L139.87 254.637L139.8 255.309L139.73 255.973L139.68 256.655L139.66 257.328L139.65 258.001L139.66 258.684L139.68 259.357L139.73 260.03L139.8 260.703L139.87 261.376L139.97 262.049L140.09 262.712L140.231 263.375L140.381 264.028L140.511 264.543H164.827L165.307 264.533H222.821L163.166 248.293L162.746 248.164Z" fill="white"/>
<path d="M125.026 242.285L125.066 243.136L125.136 243.977L125.216 244.818L125.316 245.659L125.426 246.501L125.566 247.332L125.716 248.173L125.886 249.004L126.076 249.826L126.286 250.647L126.506 251.468L126.756 252.28L127.016 253.081L127.286 253.883L127.586 254.685L127.897 255.466L128.227 256.258L128.567 257.03L128.927 257.802L129.297 258.554L129.697 259.306L130.107 260.058L130.537 260.791L130.977 261.513L131.427 262.225L131.898 262.938L132.388 263.641L132.888 264.323L133.058 264.541H140.51L140.38 264.027L140.23 263.373L140.09 262.71L139.97 262.047L139.87 261.374L139.8 260.701L139.729 260.029L139.679 259.356L139.659 258.683L139.649 258L139.659 257.327L139.679 256.654L139.729 255.971L139.8 255.308L139.87 254.635L139.97 253.972L140.09 253.299L140.23 252.636L140.38 251.973L140.55 251.33L140.74 250.677L140.94 250.033L141.17 249.39L141.41 248.757L141.67 248.133L141.94 247.51L142.23 246.896L142.53 246.303L142.86 245.699L143.2 245.115L143.55 244.531L143.921 243.967L144.311 243.403L144.701 242.859L145.131 242.324L145.551 241.79L146.001 241.275L146.451 240.781L146.921 240.286L147.411 239.811L147.902 239.346L148.412 238.89L148.932 238.455L149.452 238.029L150.002 237.624L150.552 237.218L151.122 236.842L151.702 236.476L152.283 236.119L152.863 235.793L153.473 235.476L154.083 235.169L154.703 234.882L155.333 234.605L155.953 234.348L156.604 234.12L157.244 233.903L157.894 233.695L158.554 233.507L159.214 233.339L159.884 233.18L160.545 233.052L161.225 232.933L161.905 232.834L162.575 232.755L163.255 232.695L163.935 232.646L164.616 232.616L165.306 232.606H374.958V200.691H165.306L164.446 200.701L163.595 200.721L162.735 200.771L161.885 200.83L161.025 200.909L160.185 201.008L159.334 201.127L158.494 201.255L157.654 201.414L156.804 201.582L155.974 201.77L155.143 201.978L154.323 202.196L153.493 202.443L152.683 202.69L151.872 202.968L151.072 203.254L150.272 203.571L149.482 203.888L148.702 204.224L147.922 204.59L147.151 204.967L146.391 205.343L145.641 205.758L144.901 206.174L144.161 206.609L143.44 207.064L142.72 207.53L142.02 208.015L141.33 208.509L140.64 209.014L139.98 209.539L139.309 210.083L138.659 210.637L138.029 211.201L137.409 211.785L136.799 212.369L136.199 212.982L135.608 213.606L135.038 214.229L134.478 214.873L133.938 215.526L133.408 216.189L132.888 216.862L132.388 217.554L131.898 218.247L131.427 218.95L130.977 219.672L130.537 220.404L130.107 221.137L129.697 221.869L129.297 222.621L128.927 223.393L128.567 224.155L128.227 224.937L127.897 225.709L127.586 226.5L127.286 227.292L127.016 228.094L126.756 228.905L126.506 229.717L126.286 230.538L126.076 231.359L125.886 232.191L125.716 233.012L125.566 233.843L125.426 234.685L125.316 235.526L125.216 236.367L125.136 237.208L125.066 238.049L125.026 238.89L124.986 239.741V240.593V241.444L125.026 242.285Z" fill="#0DC09D"/>
<path d="M162.745 176.346L162.395 176.208L162.045 176.059L161.705 175.911L161.385 175.733L161.054 175.555L160.744 175.357L160.444 175.129L160.144 174.911L159.864 174.664L159.594 174.417L159.344 174.139L159.104 173.872L158.864 173.575L158.644 173.278L158.444 172.962L158.254 172.645L158.084 172.319L157.914 171.992L157.784 171.655L157.654 171.299L157.544 170.953L157.454 170.606L157.374 170.23L157.314 169.874L157.264 169.498L157.254 169.132L157.244 168.756L157.254 168.39L157.264 168.034L157.314 167.667L157.374 167.301L157.454 166.945L157.544 166.579L157.654 166.223L157.784 165.876L157.914 165.54L158.084 165.213L158.254 164.877L158.444 164.57L158.644 164.253L158.864 163.956L159.104 163.679L159.344 163.392L159.594 163.115L159.864 162.868L160.144 162.62L160.444 162.403L160.744 162.195L161.054 161.977L161.385 161.799L161.705 161.621L162.045 161.462L162.395 161.324L162.745 161.205L163.095 161.086L163.455 160.987L163.825 160.918L164.185 160.859L164.555 160.819L164.935 160.79H165.055H164.615L163.935 160.819L163.255 160.869L162.575 160.928L161.905 161.017L161.225 161.116L160.544 161.225L159.884 161.364L159.214 161.512L158.554 161.68L157.894 161.868L157.244 162.076L156.603 162.294L155.953 162.531L155.333 162.779L154.703 163.056L154.083 163.343L153.473 163.65L152.862 163.966L152.282 164.293L151.692 164.649L151.122 165.015L150.552 165.391L150.002 165.797L149.452 166.203L148.921 166.628L148.411 167.074L147.901 167.519L147.401 167.984L146.921 168.459L146.451 168.954L146.001 169.459L145.551 169.973L145.131 170.498L144.7 171.042L144.31 171.586L143.92 172.14L143.55 172.704L143.2 173.288L142.86 173.872L142.53 174.476L142.23 175.07L141.94 175.683L141.67 176.307L141.41 176.93L141.17 177.563L140.939 178.207L140.739 178.85L140.549 179.503L140.379 180.156L140.219 180.809L140.089 181.472L139.969 182.145L139.869 182.808L139.789 183.481L139.729 184.144L139.679 184.827L139.659 185.51L139.649 186.183L139.659 186.856L139.679 187.539L139.729 188.212L139.789 188.885L139.869 189.548L139.969 190.221L140.089 190.894L140.219 191.547L140.379 192.21L140.549 192.863L140.739 193.506L140.939 194.159L141.17 194.793L141.41 195.426L141.67 196.049L141.94 196.673L142.23 197.286L142.53 197.89L142.73 198.236L142.75 198.227L143.59 197.801L144.43 197.395L145.281 196.999L146.141 196.633L147.001 196.257L147.881 195.921L148.751 195.604L149.642 195.278L150.532 195L151.432 194.723L152.332 194.466L153.233 194.219L154.143 193.991L155.063 193.793L155.983 193.615L156.893 193.437L157.824 193.288L158.754 193.15L159.684 193.031L160.624 192.932L161.555 192.853L162.495 192.784L163.415 192.744L164.375 192.724L165.306 192.705H222.82L163.155 176.465L162.745 176.346Z" fill="white"/>
<path d="M334.637 344.364H335.327L336.008 344.325L336.688 344.285L337.368 344.226L338.048 344.137L338.728 344.038L339.398 343.929L340.069 343.79L340.729 343.642L341.399 343.474L342.049 343.285L342.699 343.078L343.349 342.86L343.99 342.622L344.61 342.365L345.24 342.098L345.86 341.811L346.47 341.504L347.08 341.188L347.67 340.851L348.251 340.495L348.821 340.129L349.391 339.753L349.951 339.357L350.491 338.941L351.021 338.525L351.541 338.08L352.042 337.625L352.542 337.17L353.022 336.685L353.492 336.2L353.952 335.695L354.392 335.181L354.822 334.656L355.242 334.112L355.632 333.568L356.023 333.013L356.393 332.439L356.743 331.865L357.083 331.272L357.413 330.678L357.723 330.074L358.013 329.471L358.273 328.847L358.543 328.214L358.773 327.59L359.003 326.947L359.213 326.304L359.393 325.651L359.573 324.997L359.723 324.344L359.853 323.671L359.974 323.008L360.074 322.345L360.154 321.662L360.224 320.999L360.264 320.317L360.294 319.644V318.971V318.298L360.264 317.615L360.224 316.942L360.154 316.269L360.074 315.606L359.974 314.933L359.853 314.26L359.723 313.607L359.573 312.944L359.393 312.291L359.213 311.648L359.003 310.994L358.773 310.351L358.543 309.728L358.273 309.104L358.013 308.481L357.723 307.867L357.413 307.264L357.213 306.897L357.193 306.907L356.353 307.333L355.512 307.739L354.662 308.134L353.802 308.511L352.932 308.867L352.062 309.213L351.191 309.53L350.301 309.847L349.401 310.143L348.511 310.411L347.62 310.668L346.7 310.915L345.8 311.133L344.88 311.341L343.96 311.529L343.039 311.697L342.119 311.846L341.189 311.984L340.259 312.103L339.328 312.202L338.388 312.281L337.448 312.34L336.508 312.39L335.577 312.419L334.637 312.429H277.123L336.778 328.669L337.198 328.798L337.548 328.926L337.898 329.065L338.228 329.223L338.558 329.401L338.878 329.589L339.198 329.787L339.498 330.005L339.789 330.233L340.059 330.48L340.339 330.727L340.599 330.995L340.839 331.272L341.079 331.568L341.289 331.865L341.499 332.172L341.679 332.479L341.849 332.815L342.009 333.152L342.149 333.478L342.289 333.825L342.389 334.191L342.489 334.537L342.559 334.903L342.629 335.26L342.669 335.636L342.689 335.992L342.699 336.368L342.689 336.734L342.669 337.11L342.629 337.467L342.559 337.833L342.489 338.199L342.389 338.555L342.289 338.902L342.149 339.258L342.009 339.594L341.849 339.931L341.679 340.247L341.499 340.574L341.289 340.881L341.079 341.178L340.839 341.465L340.599 341.742L340.339 342.009L340.059 342.266L339.789 342.514L339.498 342.731L339.198 342.949L338.878 343.157L338.558 343.345L338.228 343.513L337.898 343.662L337.548 343.81L337.198 343.939L336.848 344.047L336.488 344.137L336.128 344.216L335.748 344.265L335.387 344.305L335.007 344.344L334.637 344.354" fill="white"/>
<path d="M374.918 334.684L374.878 333.843L374.818 333.002L374.728 332.151L374.638 331.31L374.518 330.478L374.378 329.637L374.227 328.806L374.057 327.975L373.867 327.143L373.657 326.332L373.437 325.511L373.197 324.689L372.937 323.898L372.657 323.086L372.357 322.294L372.057 321.513L371.727 320.721L371.377 319.949L371.027 319.177L370.647 318.415L370.246 317.673L369.846 316.921L369.416 316.188L368.966 315.456L368.516 314.744L368.046 314.031L367.556 313.338L367.056 312.656L366.536 311.973L366.015 311.32L365.465 310.657L364.905 310.023L364.335 309.39L363.745 308.776L363.155 308.163L362.535 307.579L361.914 306.995L361.284 306.431L360.634 305.877L359.974 305.332L359.934 305.303L358.994 305.867L358.163 306.332L357.303 306.797L357.193 306.856L357.413 307.262L357.723 307.866L358.013 308.479L358.274 309.103L358.544 309.726L358.774 310.35L359.004 310.993L359.214 311.646L359.394 312.289L359.574 312.943L359.724 313.606L359.854 314.259L359.974 314.932L360.074 315.605L360.154 316.268L360.224 316.941L360.264 317.614L360.294 318.296V318.969V319.642L360.264 320.315L360.224 320.998L360.154 321.661L360.074 322.344L359.974 323.007L359.854 323.67L359.724 324.343L359.574 324.996L359.394 325.649L359.214 326.302L359.004 326.946L358.774 327.589L358.544 328.212L358.274 328.846L358.013 329.469L357.723 330.073L357.413 330.676L357.083 331.27L356.743 331.864L356.393 332.438L356.023 333.012L355.633 333.566L355.243 334.11L354.823 334.655L354.393 335.179L353.952 335.694L353.492 336.198L353.022 336.683L352.542 337.168L352.042 337.623L351.542 338.079L351.022 338.524L350.492 338.94L349.951 339.355L349.391 339.751L348.821 340.127L348.251 340.493L347.671 340.85L347.081 341.186L346.471 341.503L345.86 341.81L345.24 342.097L344.61 342.364L343.99 342.621L343.35 342.859L342.7 343.076L342.05 343.284L341.399 343.472L340.729 343.64L340.069 343.789L339.399 343.927L338.729 344.036L338.049 344.135L337.368 344.224L336.688 344.284L336.008 344.323L335.328 344.363H334.638H124.986V376.288H334.638L335.498 376.278L336.348 376.258L337.208 376.208L338.059 376.139L338.919 376.07L339.769 375.971L340.619 375.842L341.459 375.714L342.3 375.565L343.14 375.387L343.97 375.199L344.8 375.001L345.62 374.783L346.451 374.536L347.261 374.289L348.071 374.002L348.881 373.715L349.681 373.408L350.472 373.091L351.252 372.755L352.032 372.389L352.792 372.012L353.562 371.627L354.303 371.221L355.053 370.805L355.783 370.37L356.503 369.905L357.223 369.449L357.923 368.964L358.614 368.46L359.304 367.955L359.974 367.431L360.634 366.886L361.284 366.332L361.914 365.778L362.535 365.194L363.155 364.6L363.745 363.997L364.335 363.373L364.905 362.75L365.465 362.106L366.015 361.453L366.536 360.79L367.056 360.117L367.556 359.425L368.046 358.732L368.516 358.019L368.966 357.307L369.416 356.575L369.846 355.842L370.246 355.1L370.647 354.348L371.027 353.586L371.377 352.824L371.727 352.042L372.057 351.26L372.357 350.479L372.657 349.677L372.937 348.875L373.197 348.064L373.437 347.252L373.657 346.441L373.867 345.61L374.057 344.788L374.227 343.967L374.378 343.126L374.518 342.294L374.638 341.453L374.728 340.612L374.818 339.771L374.878 338.92L374.918 338.079L374.958 337.238V336.386V335.535L374.918 334.684Z" fill="#0DC09D"/>
<path d="M125.026 170.459L125.066 171.31L125.126 172.151L125.216 173.002L125.316 173.844L125.426 174.675L125.566 175.516L125.716 176.347L125.886 177.179L126.076 178L126.286 178.821L126.506 179.633L126.746 180.454L127.016 181.256L127.286 182.057L127.586 182.859L127.887 183.641L128.217 184.432L128.567 185.204L128.927 185.976L129.297 186.728L129.697 187.48L130.107 188.233L130.527 188.965L130.977 189.697L131.427 190.4L131.898 191.112L132.388 191.815L132.888 192.498L133.408 193.181L133.938 193.834L134.478 194.497L135.038 195.13L135.608 195.763L136.199 196.377L136.799 196.981L137.409 197.574L138.029 198.158L138.659 198.722L139.309 199.277L139.98 199.811L140.01 199.841L140.95 199.286L141.78 198.821L142.64 198.346L142.76 198.297L142.53 197.891L142.23 197.287L141.94 196.674L141.67 196.05L141.41 195.427L141.17 194.794L140.94 194.16L140.74 193.507L140.55 192.864L140.38 192.211L140.22 191.548L140.09 190.895L139.97 190.222L139.87 189.549L139.79 188.886L139.729 188.213L139.679 187.54L139.659 186.857L139.649 186.184L139.659 185.511L139.679 184.828L139.729 184.145L139.79 183.482L139.87 182.809L139.97 182.146L140.09 181.473L140.22 180.81L140.38 180.157L140.55 179.504L140.74 178.851L140.94 178.208L141.17 177.565L141.41 176.931L141.67 176.308L141.94 175.684L142.23 175.071L142.53 174.477L142.86 173.873L143.2 173.289L143.55 172.706L143.921 172.141L144.311 171.587L144.701 171.043L145.131 170.499L145.551 169.974L146.001 169.46L146.451 168.955L146.921 168.46L147.401 167.985L147.902 167.52L148.412 167.075L148.922 166.629L149.452 166.204L150.002 165.798L150.552 165.392L151.122 165.016L151.692 164.65L152.283 164.294L152.863 163.967L153.473 163.651L154.083 163.344L154.703 163.057L155.333 162.78L155.953 162.532L156.604 162.295L157.244 162.077L157.894 161.869L158.554 161.681L159.214 161.513L159.884 161.365L160.545 161.226L161.225 161.117L161.905 161.018L162.575 160.929L163.255 160.87L163.935 160.82L164.616 160.791H165.306H374.958V128.866H165.306H164.446L163.595 128.895L162.735 128.945L161.885 129.014L161.025 129.083L160.175 129.182L159.334 129.311L158.484 129.43L157.644 129.588L156.804 129.756L155.974 129.954L155.143 130.152L154.323 130.37L153.493 130.617L152.683 130.865L151.872 131.142L151.072 131.429L150.262 131.745L149.482 132.062L148.702 132.399L147.922 132.765L147.151 133.131L146.391 133.527L145.641 133.933L144.901 134.348L144.161 134.784L143.44 135.239L142.72 135.704L142.02 136.189L141.33 136.684L140.64 137.198L139.98 137.723L139.309 138.267L138.659 138.811L138.029 139.375L137.409 139.959L136.799 140.553L136.199 141.157L135.608 141.77L135.038 142.404L134.478 143.047L133.938 143.7L133.408 144.363L132.888 145.036L132.388 145.729L131.898 146.421L131.427 147.124L130.977 147.846L130.527 148.569L130.107 149.311L129.697 150.053L129.297 150.796L128.927 151.567L128.567 152.329L128.217 153.111L127.887 153.893L127.586 154.675L127.286 155.466L127.016 156.278L126.746 157.08L126.506 157.891L126.286 158.712L126.076 159.534L125.886 160.365L125.716 161.186L125.566 162.018L125.426 162.859L125.316 163.7L125.216 164.541L125.126 165.382L125.066 166.233L125.026 167.075L124.986 167.916V168.767V169.618L125.026 170.459Z" fill="#0DC09D"/>
<path d="M516.784 329.897H453.084V305.262H516.784C526.645 305.262 534.243 303.388 539.576 299.619C544.909 295.86 547.581 291.007 547.581 285.071C547.581 279.134 545.058 274.78 540.033 272.007C534.998 269.244 527.549 266.959 517.687 265.182L507.418 263.405C497.348 261.628 488.191 259.05 479.948 255.692C471.695 252.333 465.15 247.675 460.324 241.739C455.497 235.803 453.074 228.187 453.074 218.882C453.074 205.037 458.308 194.297 468.775 186.681C479.243 179.065 493.127 175.257 510.437 175.257H572.627V200.194H510.437C501.986 200.194 495.342 201.727 490.515 204.793C485.689 207.859 483.265 212.165 483.265 217.701C483.265 223.637 485.629 227.992 490.356 230.765C495.084 233.537 501.479 235.617 509.524 236.994L520.091 238.771C530.757 240.548 540.519 243.028 549.378 246.191C558.237 249.355 565.229 253.905 570.363 259.841C575.498 265.778 578.06 273.696 578.06 283.587C578.06 298.232 572.518 309.607 561.455 317.721C550.381 325.835 535.494 329.887 516.774 329.887L516.784 329.897Z" fill="white"/>
<path d="M892.291 377.653V307.648H887.432C885.202 311.665 881.963 315.525 877.704 319.226C873.445 322.937 868.026 325.995 861.447 328.4C854.858 330.804 846.71 332.012 836.982 332.012C824.414 332.012 812.866 329.003 802.328 322.987C791.79 316.97 783.382 308.301 777.103 296.96C770.824 285.629 767.685 271.942 767.685 255.891V251.378C767.685 235.337 770.874 221.65 777.263 210.309C783.642 198.978 792.1 190.299 802.638 184.283C813.176 178.266 824.624 175.257 836.982 175.257C851.569 175.257 862.767 177.87 870.565 183.075C878.364 188.291 884.193 194.208 888.042 200.829H892.901V175.257H923.605V377.633H892.301L892.291 377.653ZM845.79 304.936C859.567 304.936 870.815 300.621 879.523 292.002C888.232 283.382 892.591 271.042 892.591 255V252.289C892.591 236.445 888.182 224.213 879.373 215.584C870.555 206.964 859.357 202.65 845.79 202.65C832.223 202.65 821.325 206.964 812.506 215.584C803.688 224.213 799.289 236.445 799.289 252.289V255C799.289 271.052 803.698 283.382 812.506 292.002C821.325 300.631 832.413 304.936 845.79 304.936Z" fill="white"/>
<path d="M744.698 249.35C744.698 234.016 741.685 220.629 735.66 209.18C729.636 197.73 721.242 188.782 710.51 182.345C699.759 175.909 687.253 172.681 673.003 172.681C658.754 172.681 645.485 175.95 634.446 182.497C628.916 185.765 623.991 189.669 619.651 194.229C619.106 194.794 618.571 195.379 618.036 195.964C616.45 197.74 614.974 199.596 613.567 201.533C611.704 204.125 609.97 206.879 608.384 209.795C602.062 221.447 598.891 235.146 598.891 250.893V254.575C598.891 270.323 602.052 284.012 608.384 295.674C614.706 307.326 623.496 316.375 634.743 322.811C645.98 329.247 659.14 332.475 674.202 332.475C687.461 332.475 698.5 330.326 707.339 326.039C716.179 321.741 723.204 316.425 728.427 310.09C729.873 308.324 731.241 306.589 732.519 304.874C735.849 300.425 738.604 296.138 740.774 291.992L714.871 278.494C711.66 285.455 707.082 291.588 701.166 296.895C695.23 302.211 686.45 304.864 674.807 304.864C662.351 304.864 651.956 300.92 643.632 293.061C643.117 292.567 642.611 292.083 642.126 291.558C641.63 291.033 641.164 290.519 640.699 289.994C639.589 288.693 638.558 287.331 637.607 285.899C633.554 279.705 631.235 272.33 630.65 263.786C630.591 262.969 630.561 262.141 630.532 261.314H744.708V249.35H744.698ZM651.312 237.083L635.447 253.234H630.224C630.145 248.583 630.224 242.611 630.829 237.083C631.126 234.985 631.523 232.957 632.038 231.01C632.533 229.063 633.138 227.207 633.841 225.431C636.18 219.51 639.658 214.496 644.236 210.4C651.559 203.853 661.043 200.585 672.706 200.585C684.369 200.585 693.793 203.853 701.027 210.4C708.261 216.947 712.373 225.845 713.374 237.083H651.322H651.312Z" fill="white"/>
<path d="M1093.62 249.35C1093.62 234.016 1090.6 220.629 1084.58 209.18C1078.55 197.73 1070.16 188.782 1059.43 182.345C1048.68 175.909 1036.17 172.681 1021.92 172.681C1007.67 172.681 994.404 175.95 983.365 182.497C977.835 185.765 972.91 189.669 968.57 194.229C968.025 194.794 967.49 195.379 966.955 195.964C965.369 197.74 963.893 199.596 962.486 201.533C960.623 204.125 958.889 206.879 957.303 209.795C950.981 221.447 947.81 235.146 947.81 250.893V254.575C947.81 270.323 950.971 284.012 957.303 295.674C963.625 307.326 972.415 316.375 983.662 322.811C994.899 329.247 1008.06 332.475 1023.12 332.475C1036.38 332.475 1047.42 330.326 1056.26 326.039C1065.1 321.741 1072.12 316.425 1077.35 310.09C1078.79 308.324 1080.16 306.589 1081.44 304.874C1084.77 300.425 1087.52 296.138 1089.69 291.992L1063.79 278.494C1060.58 285.455 1056 291.588 1050.08 296.895C1044.15 302.211 1035.37 304.864 1023.73 304.864C1011.27 304.864 1000.87 300.92 992.551 293.061C992.036 292.567 991.53 292.083 991.045 291.558C990.549 291.033 990.083 290.519 989.618 289.994C988.508 288.693 987.477 287.331 986.526 285.899C982.473 279.705 980.154 272.33 979.569 263.786C979.51 262.969 979.48 262.141 979.451 261.314H1093.63V249.35H1093.62ZM1000.23 237.083L984.366 253.234H979.143C979.064 248.583 979.143 242.611 979.748 237.083C980.045 234.985 980.441 232.957 980.957 231.01C981.452 229.063 982.057 227.207 982.76 225.431C985.099 219.51 988.577 214.496 993.155 210.4C1000.48 203.853 1009.96 200.585 1021.62 200.585C1033.29 200.585 1042.71 203.853 1049.95 210.4C1057.18 216.947 1061.29 225.845 1062.29 237.083H1000.24H1000.23Z" fill="white"/>
<path d="M1124.88 330.029V173.285H1155.58V190.732H1160.44C1162.87 184.517 1166.77 179.955 1172.13 177.045C1177.5 174.136 1184.14 172.681 1192.04 172.681H1209.98V200.657H1190.83C1180.69 200.657 1172.39 203.418 1165.91 208.931C1159.42 214.453 1156.18 222.924 1156.18 234.354V330.019H1124.87L1124.88 330.029Z" fill="white"/>
<path d="M1354.01 230.335C1354.01 211.934 1348.37 197.721 1337.05 187.707C1325.75 177.693 1310.23 172.681 1290.48 172.681C1277.53 172.681 1266.58 174.72 1257.63 178.818C1248.69 182.916 1241.49 188.33 1236.05 195.07C1230.6 201.819 1226.64 209.182 1224.17 217.147L1253.77 226.659C1255.62 218.483 1259.43 211.834 1265.19 206.721C1270.96 201.619 1279.27 199.057 1290.17 199.057C1301.07 199.057 1309.5 201.769 1314.84 207.183C1320.2 212.597 1322.86 219.608 1322.86 228.196V254.351H1317.73L1301.28 238.009H1276.6C1259.95 238.009 1246.12 241.946 1235.12 249.821C1224.12 257.686 1218.61 269.297 1218.61 284.634C1218.61 294.859 1221.14 303.538 1226.17 310.699C1231.21 317.861 1238 323.274 1246.52 326.951C1255.07 330.627 1264.78 332.475 1275.66 332.475C1286.54 332.475 1294.58 330.988 1300.95 328.025C1307.33 325.062 1312.17 321.587 1315.45 317.599C1318.75 313.612 1321.2 309.976 1322.85 306.712H1327.79V329.898L1354 329.898V230.345L1354.01 230.335ZM1322.86 265.6C1322.86 278.286 1318.9 288.25 1310.98 295.502C1303.07 302.754 1292.85 306.39 1280.3 306.39C1271.05 306.39 1263.75 304.351 1258.4 300.253C1253.05 296.155 1250.38 290.641 1250.38 283.69C1250.38 276.739 1252.96 271.466 1258.09 267.901C1263.24 264.325 1270.01 262.537 1278.44 262.537H1322.85V265.6H1322.86Z" fill="white"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 28 KiB

19
nextflow/docs/_static/seqera-logo.svg vendored Normal file
View File

@@ -0,0 +1,19 @@
<svg width="1261" height="250" viewBox="0 0 1261 250" fill="none" xmlns="http://www.w3.org/2000/svg">
<g clip-path="url(#clip0_933_7627)">
<path d="M249.96 207.96L249.91 207.11L249.85 206.26L249.76 205.4L249.67 204.55L249.55 203.71L249.41 202.86L249.26 202.03L249.1 201.18L248.9 200.35L248.7 199.53L248.47 198.7L248.22 197.86L247.97 197.06L247.7 196.25L247.4 195.44L247.09 194.65L246.76 193.85L246.41 193.08L246.06 192.29L245.69 191.52L245.29 190.76L244.89 190.01L244.45 189.26L244.01 188.54L243.56 187.81L243.09 187.1L242.6 186.39L242.1 185.7L241.57 185.01L241.05 184.35L240.51 183.69L239.95 183.04L239.37 182.4L238.78 181.78L238.18 181.16L237.57 180.58L236.96 179.98L236.32 179.41L235.67 178.85L235.02 178.31L233.87 179L233.06 179.46L232.23 179.91L231.39 180.34L230.56 180.75L229.71 181.15L228.84 181.53L227.97 181.89L227.1 182.24L226.23 182.56L225.34 182.88L224.44 183.18L223.55 183.45L222.65 183.71L221.74 183.96L220.84 184.18L219.91 184.39L219 184.58L218.08 184.75L217.15 184.9L216.23 185.04L215.3 185.16L214.36 185.26L213.42 185.34L212.49 185.4L211.55 185.45L210.61 185.48L209.67 185.49H152.15L211.81 201.9L212.24 202.03L212.59 202.16L212.94 202.3L213.26 202.46L213.6 202.64L213.91 202.83L214.24 203.03L214.54 203.25L214.83 203.48L215.1 203.73L215.38 203.98L215.64 204.25L215.88 204.52L216.11 204.83L216.33 205.13L216.54 205.44L216.71 205.75L216.89 206.09L217.05 206.43L217.19 206.76L217.33 207.11L217.43 207.48L217.53 207.83L217.6 208.2L217.66 208.56L217.7 208.94L217.73 209.3L217.74 209.68L217.73 210.05L217.7 210.43L217.66 210.79L217.6 211.16L217.53 211.53L217.43 211.89L217.33 212.24L217.19 212.6L217.05 212.94L216.89 213.28L216.71 213.6L216.54 213.93L216.33 214.24L216.11 214.54L215.88 214.83L215.64 215.11L215.38 215.38L215.1 215.64L214.83 215.89L214.54 216.11L214.24 216.33L213.91 216.54L213.6 216.73L213.26 216.9L212.94 217.05L212.59 217.2L212.24 217.33L211.89 217.44L211.52 217.53L211.16 217.61L210.79 217.66L210.42 217.7L210.05 217.74L209.67 217.75H0V250H209.67L210.52 249.99L211.39 249.96L212.25 249.93L213.09 249.85L213.95 249.78L214.8 249.68L215.65 249.55L216.49 249.43L217.33 249.28L218.18 249.1L219 248.9L219.83 248.7L220.66 248.48L221.49 248.24L222.29 247.98L223.1 247.7L223.92 247.4L224.72 247.09L225.5 246.78L226.28 246.43L227.07 246.06L227.83 245.69L228.59 245.3L229.34 244.88L230.08 244.46L230.82 244.01L231.54 243.56L232.26 243.09L232.96 242.6L233.64 242.1L234.33 241.59L235.01 241.05L235.67 240.5L236.32 239.95L236.96 239.38L237.57 238.79L238.18 238.2L238.78 237.59L239.37 236.95L239.95 236.33L240.51 235.68L241.05 235.01L241.57 234.34L242.1 233.66L242.6 232.96L243.09 232.26L243.56 231.55L244.01 230.82L244.45 230.09L244.89 229.34L245.29 228.59L245.69 227.84L246.06 227.06L246.41 226.29L246.76 225.5L247.09 224.71L247.4 223.91L247.7 223.11L247.97 222.3L248.22 221.49L248.47 220.66L248.7 219.84L248.9 219.01L249.1 218.18L249.26 217.34L249.41 216.5L249.55 215.65L249.67 214.8L249.76 213.95L249.85 213.1L249.91 212.24L249.96 211.4L249.99 210.54L250 209.68L249.99 208.83L249.96 207.96Z" fill="#7B7B7B"/>
<path d="M249.96 135.39L249.91 134.53L249.85 133.68L249.76 132.83L249.67 131.98L249.55 131.13L249.41 130.29L249.26 129.44L249.1 128.6L248.9 127.76L248.7 126.94L248.47 126.11L248.22 125.29L247.97 124.48L247.7 123.66L247.4 122.86L247.09 122.06L246.76 121.28L246.41 120.49L246.06 119.71L245.69 118.95L245.29 118.18L244.89 117.42L244.45 116.69L244.01 115.95L243.56 115.24L243.09 114.51L242.6 113.81L242.1 113.13L241.93 112.9H241.92H152.14L211.81 129.31L212.24 129.45L212.59 129.58L212.94 129.71L213.26 129.88L213.6 130.05L213.91 130.24L214.24 130.44L214.54 130.66L214.83 130.9L215.1 131.14L215.38 131.39L215.64 131.66L215.88 131.94L216.11 132.24L216.33 132.54L216.54 132.85L216.71 133.18L216.89 133.5L217.05 133.84L217.19 134.19L217.33 134.54L217.43 134.89L217.53 135.25L217.6 135.61L217.66 135.99L217.7 136.35L217.73 136.73L217.74 137.1L217.73 137.46L217.7 137.84L217.66 138.21L217.6 138.58L217.53 138.94L217.43 139.3L217.33 139.66L217.19 140L217.05 140.35L216.89 140.7L216.71 141.03L216.54 141.35L216.33 141.65L216.11 141.96L215.88 142.25L215.64 142.53L215.38 142.8L215.1 143.06L214.83 143.3L214.54 143.54L214.24 143.75L213.91 143.95L213.6 144.15L213.26 144.31L212.94 144.48L212.59 144.63L212.24 144.74L211.89 144.86L211.52 144.95L211.16 145.03L210.79 145.09L210.42 145.12L210.05 145.15L209.67 145.16H127.18H16.13H0V177.41H122.81H209.67H210.52L211.39 177.39L212.25 177.34L213.09 177.27L213.95 177.2L214.8 177.1L215.65 176.98L216.49 176.85L217.33 176.69L218.18 176.51L219 176.33L219.83 176.11L220.66 175.9L221.49 175.65L222.29 175.39L223.1 175.11L223.92 174.83L224.72 174.51L225.5 174.19L226.28 173.84L227.07 173.48L227.24 173.39L227.83 173.1L228.59 172.71L229.34 172.3L230.08 171.88L230.82 171.44L231.54 170.98L232.26 170.5L232.96 170.03L233.64 169.51L234.33 169L235.01 168.46L235.67 167.93L236.32 167.36L236.96 166.79L237.57 166.21L238.18 165.61L238.78 165L239.37 164.38L239.95 163.74L240.51 163.09L241.05 162.43L241.57 161.76L242.1 161.08L242.6 160.38L243.09 159.68L243.56 158.96L244.01 158.24L244.45 157.5L244.89 156.76L245.29 156.01L245.69 155.25L246.06 154.47L246.41 153.71L246.76 152.93L247.09 152.14L247.4 151.34L247.7 150.54L247.97 149.73L248.22 148.9L248.47 148.09L248.7 147.26L248.9 146.43L249.1 145.59L249.26 144.75L249.41 143.91L249.55 143.06L249.67 142.21L249.76 141.36L249.85 140.51L249.91 139.66L249.96 138.8L249.99 137.95L250 137.1L249.99 136.24L249.96 135.39Z" fill="#7B7B7B"/>
<path d="M40.33 72.5801L39.46 72.5901L38.61 72.6101L37.75 72.6601L36.9 72.7301L36.05 72.8001L35.2 72.9001L34.35 73.0301L33.51 73.1501L32.66 73.3101L31.82 73.4901L30.99 73.6701L30.16 73.8901L29.34 74.1001L28.51 74.3501L27.7 74.6101L26.88 74.8901L26.08 75.1801L25.28 75.4901L24.5 75.8101L23.72 76.1601L22.93 76.5301L22.76 76.6101L22.17 76.9001L21.41 77.2901L20.66 77.7001L19.91 78.1201L19.18 78.5601L18.46 79.0201L17.74 79.5001L17.03 79.9801L16.34 80.4901L15.66 81.0001L14.99 81.5401L14.33 82.0801L13.68 82.6401L13.04 83.2101L12.42 83.7901L11.82 84.3901L11.22 85.0001L10.63 85.6301L10.05 86.2601L9.49 86.9101L8.95 87.5601L8.42 88.2401L7.9 88.9301L7.4 89.6301L6.91 90.3301L6.44 91.0401L5.99 91.7601L5.54 92.5001L5.11 93.2401L4.71 93.9901L4.31 94.7501L3.94 95.5101L3.58 96.2901L3.24 97.0801L2.9 97.8601L2.6 98.6601L2.3 99.4601L2.03 100.28L1.76 101.1L1.53 101.91L1.3 102.74L1.09 103.57L0.9 104.41L0.74 105.25L0.58 106.09L0.44 106.94L0.33 107.79L0.23 108.63L0.15 109.48L0.09 110.34L0.04 111.19L0 112.04V112.9V113.76L0.04 114.61L0.09 115.48L0.15 116.33L0.23 117.18L0.33 118.03L0.44 118.88L0.58 119.71L0.74 120.56L0.9 121.4L1.09 122.24L1.3 123.06L1.53 123.89L1.76 124.71L2.03 125.53L2.3 126.34L2.6 127.14L2.9 127.94L3.24 128.73L3.58 129.51L3.94 130.29L4.31 131.05L4.71 131.83L5.11 132.58L5.54 133.31L5.99 134.05L6.44 134.76L6.91 135.49L7.4 136.19L7.9 136.88L8.07 137.1H97.86L38.18 120.68L37.76 120.55L37.41 120.43L37.06 120.29L36.73 120.12L36.4 119.95L36.08 119.76L35.76 119.55L35.46 119.34L35.16 119.1L34.89 118.86L34.61 118.61L34.36 118.34L34.12 118.06L33.89 117.76L33.66 117.46L33.46 117.15L33.27 116.83L33.1 116.5L32.94 116.16L32.8 115.81L32.67 115.46L32.56 115.11L32.47 114.75L32.39 114.39L32.34 114.01L32.29 113.65L32.27 113.28L32.26 112.9L32.27 112.54L32.29 112.16L32.34 111.79L32.39 111.43L32.47 111.05L32.56 110.7L32.67 110.34L32.8 109.99L32.94 109.65L33.1 109.3L33.27 108.97L33.46 108.65L33.66 108.35L33.89 108.04L34.12 107.75L34.36 107.46L34.61 107.2L34.89 106.94L35.16 106.7L35.46 106.46L35.76 106.25L36.08 106.05L36.4 105.85L36.73 105.69L37.06 105.53L37.41 105.37L37.76 105.26L38.11 105.14L38.48 105.05L38.84 104.98L39.2 104.91L39.58 104.88L39.95 104.84H40.33H122.83H233.86H250V72.5801H127.16H40.33Z" fill="#7B7B7B"/>
<path d="M40.33 0L39.46 0.01L38.61 0.02L37.75 0.08L36.9 0.15L36.05 0.23L35.2 0.33L34.35 0.44L33.51 0.58L32.66 0.73L31.82 0.9L30.99 1.09L30.16 1.3L29.34 1.53L28.51 1.76L27.7 2.03L26.88 2.3L26.08 2.59L25.28 2.91L24.5 3.23L23.72 3.57L22.93 3.94L22.17 4.31L21.41 4.7L20.66 5.11L19.91 5.54L19.18 5.99L18.46 6.44L17.74 6.91L17.03 7.4L16.34 7.9L15.66 8.41L14.99 8.94L14.33 9.49L13.68 10.05L13.04 10.61L12.42 11.21L11.82 11.8L11.22 12.41L10.63 13.04L10.05 13.68L9.49 14.32L8.95 14.99L8.42 15.66L7.9 16.34L7.4 17.04L6.91 17.74L6.44 18.45L5.99 19.18L5.54 19.91L5.11 20.65L4.71 21.4L4.31 22.16L3.94 22.94L3.58 23.71L3.24 24.5L2.9 25.29L2.6 26.08L2.3 26.89L2.03 27.7L1.76 28.51L1.53 29.34L1.3 30.16L1.09 30.99L0.9 31.83L0.74 32.66L0.58 33.5L0.44 34.35L0.33 35.2L0.23 36.05L0.15 36.9L0.09 37.75L0.04 38.6L0 39.46V40.31V41.18L0.04 42.04L0.09 42.89L0.15 43.74L0.23 44.6L0.33 45.45L0.44 46.29L0.58 47.14L0.74 47.98L0.9 48.81L1.09 49.65L1.3 50.47L1.53 51.3L1.76 52.13L2.03 52.94L2.3 53.75L2.6 54.55L2.9 55.35L3.24 56.15L3.58 56.93L3.94 57.71L4.31 58.48L4.71 59.23L5.11 59.99L5.54 60.72L5.99 61.46L6.44 62.19L6.91 62.9L7.4 63.61L7.9 64.3L8.42 64.99L8.95 65.65L9.49 66.31L10.05 66.96L10.63 67.6L11.22 68.23L11.82 68.84L12.42 69.43L13.04 70.01L13.68 70.59L14.33 71.15L14.98 71.69H14.99L16.13 71L16.94 70.54L17.77 70.09L18.61 69.66L19.44 69.25L20.29 68.85L21.16 68.48L22.02 68.1L22.9 67.76L23.77 67.44L24.66 67.11L25.55 66.83L26.45 66.55L27.35 66.29L28.25 66.04L29.16 65.81L30.09 65.61L31 65.43L31.91 65.25L32.84 65.1L33.77 64.96L34.7 64.84L35.64 64.74L36.58 64.66L37.51 64.59L38.44 64.55L39.39 64.53L40.33 64.51H97.85L38.18 48.1L37.76 47.98L37.41 47.84L37.06 47.69L36.73 47.54L36.4 47.36L36.08 47.18L35.76 46.98L35.46 46.75L35.16 46.53L34.89 46.28L34.61 46.02L34.36 45.75L34.12 45.48L33.89 45.18L33.66 44.88L33.46 44.56L33.27 44.24L33.1 43.91L32.94 43.58L32.8 43.24L32.67 42.87L32.56 42.53L32.47 42.18L32.39 41.8L32.34 41.44L32.29 41.06L32.27 40.69L32.26 40.31L32.27 39.94L32.29 39.58L32.34 39.21L32.39 38.84L32.47 38.48L32.56 38.11L32.67 37.75L32.8 37.4L32.94 37.06L33.1 36.73L33.27 36.39L33.46 36.08L33.66 35.76L33.89 35.46L34.12 35.18L34.36 34.89L34.61 34.61L34.89 34.36L35.16 34.11L35.46 33.89L35.76 33.68L36.08 33.46L36.4 33.28L36.73 33.1L37.06 32.94L37.41 32.8L37.76 32.68L38.11 32.56L38.48 32.46L38.84 32.39L39.2 32.33L39.58 32.29L39.95 32.26L40.33 32.25H250V0H40.33Z" fill="#7B7B7B"/>
</g>
<path d="M394.146 202.864H330.011V177.636H394.146C404.075 177.636 411.724 175.716 417.094 171.856C422.464 168.007 425.153 163.037 425.153 156.958C425.153 150.878 422.614 146.418 417.554 143.579C412.484 140.749 404.985 138.409 395.056 136.589L384.717 134.769C374.577 132.95 365.358 130.31 357.059 126.87C348.749 123.43 342.16 118.661 337.3 112.581C332.441 106.502 330.001 98.7022 330.001 89.173C330.001 74.9941 335.271 63.995 345.81 56.1956C356.349 48.3963 370.328 44.4966 387.756 44.4966H450.371V70.0345H387.756C379.247 70.0345 372.558 71.6044 367.698 74.7442C362.838 77.8839 360.399 82.2936 360.399 87.9631C360.399 94.0426 362.778 98.5023 367.538 101.342C372.298 104.182 378.737 106.312 386.836 107.722L397.476 109.541C408.215 111.361 418.044 113.901 426.963 117.141C435.883 120.381 442.922 125.04 448.092 131.12C453.261 137.199 455.841 145.309 455.841 155.438C455.841 170.437 450.261 182.086 439.122 190.395C427.973 198.704 412.984 202.854 394.136 202.854L394.146 202.864Z" fill="#7B7B7B"/>
<path d="M782.605 248.98V178.246H777.746C775.516 182.306 772.276 186.205 768.017 189.945C763.757 193.695 758.337 196.785 751.758 199.214C745.168 201.644 737.019 202.864 727.29 202.864C714.721 202.864 703.172 199.824 692.633 193.745C682.093 187.665 673.684 178.906 667.405 167.447C661.125 155.998 657.985 142.169 657.985 125.95V121.391C657.985 105.182 661.175 91.3529 667.565 79.8939C673.944 68.4448 682.403 59.6755 692.943 53.596C703.482 47.5164 714.931 44.4767 727.29 44.4767C741.879 44.4767 753.078 47.1165 760.877 52.3761C768.676 57.6456 774.506 63.6252 778.356 70.3146H783.215V44.4767H813.923V248.96H782.615L782.605 248.98ZM736.099 175.506C749.878 175.506 761.127 171.147 769.836 162.437C778.546 153.728 782.905 141.259 782.905 125.05V122.31C782.905 106.302 778.496 93.9427 769.686 85.2234C760.867 76.5141 749.668 72.1545 736.099 72.1545C722.53 72.1545 711.631 76.5141 702.812 85.2234C693.992 93.9427 689.593 106.302 689.593 122.31V125.05C689.593 141.269 694.002 153.728 702.812 162.437C711.631 171.157 722.72 175.506 736.099 175.506Z" fill="#7B7B7B"/>
<path d="M627.978 120.48C627.978 105.282 624.938 92.0128 618.859 80.6637C612.779 69.3146 604.31 60.4453 593.481 54.0658C582.631 47.6863 570.013 44.4866 555.634 44.4866C541.255 44.4866 527.866 47.7263 516.727 54.2158C511.147 57.4555 506.178 61.3252 501.798 65.8449C501.248 66.4048 500.708 66.9848 500.168 67.5647C498.568 69.3246 497.078 71.1644 495.658 73.0843C493.779 75.6541 492.029 78.3839 490.429 81.2736C484.049 92.8227 480.85 106.402 480.85 122.01V125.66C480.85 141.269 484.039 154.838 490.429 166.397C496.808 177.946 505.678 186.915 517.027 193.295C528.366 199.674 541.645 202.874 556.844 202.874C570.222 202.874 581.362 200.744 590.281 196.494C599.2 192.235 606.29 186.965 611.559 180.686C613.019 178.936 614.399 177.216 615.689 175.516C619.049 171.106 621.828 166.857 624.018 162.747L597.88 149.368C594.641 156.268 590.021 162.347 584.051 167.607C578.062 172.876 569.203 175.506 557.453 175.506C544.884 175.506 534.395 171.596 525.996 163.807C525.476 163.317 524.966 162.837 524.476 162.317C523.976 161.797 523.506 161.287 523.036 160.767C521.916 159.477 520.876 158.127 519.916 156.708C515.827 150.568 513.487 143.259 512.897 134.789C512.837 133.979 512.807 133.159 512.777 132.34H627.988V120.48H627.978ZM533.745 108.321L517.737 124.33H512.467C512.387 119.721 512.467 113.801 513.077 108.321C513.377 106.242 513.777 104.232 514.297 102.302C514.797 100.372 515.407 98.5322 516.117 96.7724C518.477 90.9029 521.986 85.9333 526.606 81.8736C533.995 75.3841 543.565 72.1444 555.334 72.1444C567.103 72.1444 576.612 75.3841 583.911 81.8736C591.211 88.3631 595.36 97.1824 596.37 108.321H533.755H533.745Z" fill="#7B7B7B"/>
<path d="M991.037 120.48C991.037 105.282 987.998 92.0128 981.918 80.6637C975.839 69.3146 967.369 60.4453 956.54 54.0658C945.691 47.6863 933.072 44.4866 918.693 44.4866C904.314 44.4866 890.925 47.7263 879.786 54.2158C874.207 57.4555 869.237 61.3252 864.858 65.8449C864.308 66.4048 863.768 66.9848 863.228 67.5647C861.628 69.3246 860.138 71.1644 858.718 73.0843C856.838 75.6541 855.088 78.3839 853.488 81.2736C847.109 92.8227 843.909 106.402 843.909 122.01V125.66C843.909 141.269 847.099 154.838 853.488 166.397C859.868 177.946 868.737 186.915 880.086 193.295C891.425 199.674 904.704 202.874 919.903 202.874C933.282 202.874 944.421 200.744 953.34 196.494C962.26 192.235 969.349 186.965 974.619 180.686C976.079 178.936 977.458 177.216 978.748 175.516C982.108 171.106 984.888 166.857 987.078 162.747L960.94 149.368C957.7 156.268 953.08 162.347 947.111 167.607C941.121 172.876 932.262 175.506 920.513 175.506C907.944 175.506 897.455 171.596 889.056 163.807C888.536 163.317 888.026 162.837 887.536 162.317C887.036 161.797 886.566 161.287 886.096 160.767C884.976 159.477 883.936 158.127 882.976 156.708C878.886 150.568 876.547 143.259 875.957 134.789C875.897 133.979 875.867 133.159 875.837 132.34H991.047V120.48H991.037ZM896.805 108.321L880.796 124.33H875.527C875.447 119.721 875.527 113.801 876.137 108.321C876.437 106.242 876.837 104.232 877.357 102.302C877.856 100.372 878.466 98.5322 879.176 96.7724C881.536 90.9029 885.046 85.9333 889.666 81.8736C897.055 75.3841 906.624 72.1444 918.393 72.1444C930.162 72.1444 939.672 75.3841 946.971 81.8736C954.27 88.3631 958.42 97.1824 959.43 108.321H896.815H896.805Z" fill="#7B7B7B"/>
<path d="M1021.04 202.864V44.4867H1051.73V62.1152H1056.59C1059.02 55.8357 1062.92 51.2261 1068.29 48.2864C1073.66 45.3466 1080.3 43.8767 1088.2 43.8767H1106.14V72.1444H1086.99C1076.85 72.1444 1068.55 74.9342 1062.06 80.5038C1055.57 86.0833 1052.33 94.6426 1052.33 106.192V202.854H1021.03L1021.04 202.864Z" fill="#7B7B7B"/>
<path d="M1257.17 101.282C1257.17 82.9636 1251.58 68.8147 1240.38 58.8455C1229.19 48.8763 1213.82 43.8867 1194.27 43.8867C1181.45 43.8867 1170.61 45.9166 1161.75 49.9962C1152.91 54.0759 1145.78 59.4655 1140.39 66.1749C1135 72.8944 1131.08 80.2238 1128.63 88.1532L1157.94 97.6224C1159.77 89.4831 1163.53 82.8636 1169.23 77.774C1174.94 72.6944 1183.17 70.1446 1193.96 70.1446C1204.75 70.1446 1213.1 72.8444 1218.39 78.234C1223.69 83.6235 1226.33 90.603 1226.33 99.1523V125.19H1221.25L1204.96 108.922H1180.53C1164.04 108.922 1150.36 112.841 1139.47 120.681C1128.58 128.51 1123.13 140.069 1123.13 155.338C1123.13 165.517 1125.63 174.156 1130.61 181.286C1135.6 188.415 1142.32 193.805 1150.76 197.464C1159.22 201.124 1168.83 202.964 1179.6 202.964C1190.37 202.964 1198.33 201.484 1204.64 198.534C1210.95 195.585 1215.74 192.125 1218.99 188.155C1222.26 184.185 1224.68 180.566 1226.32 177.316H1231.21V202.964H1257.16V101.292L1257.17 101.282ZM1226.33 136.389C1226.33 149.018 1222.41 158.938 1214.57 166.157C1206.73 173.376 1196.62 176.996 1184.19 176.996C1175.03 176.996 1167.81 174.966 1162.51 170.887C1157.22 166.807 1154.58 161.317 1154.58 154.398C1154.58 147.478 1157.13 142.229 1162.2 138.679C1167.3 135.119 1174 133.34 1182.35 133.34H1226.32V136.389H1226.33Z" fill="#7B7B7B"/>
<defs>
<clipPath id="clip0_933_7627">
<rect width="250" height="250" fill="white"/>
</clipPath>
</defs>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 17 KiB

256
nextflow/docs/_static/theme.css vendored Normal file
View File

@@ -0,0 +1,256 @@
/* Inter font */
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;700&display=swap');
/* Degular font */
@font-face {
font-family: "Degular";
src: url('degular/Degular-Regular.woff2');
src: url('degular/Degular-Regular.woff2') format('woff2'),
url('degular/Degular-Regular.woff') format('woff');
font-weight: normal;
font-style: normal;
}
@font-face {
font-family: "Degular";
src: url('degular/Degular-Italic.woff2');
src: url('degular/Degular-Italic.woff2') format('woff2'),
url('degular/Degular-Italic.woff') format('woff');
font-weight: normal;
font-style: italic;
}
@font-face {
font-family: "Degular";
src: url('degular/Degular-Bold.woff2');
src: url('degular/Degular-Bold.woff2') format('woff2'),
url('degular/Degular-Bold.woff') format('woff');
font-weight: bold;
font-style: normal;
}
@font-face {
font-family: "Degular";
src: url('degular/Degular-BoldItalic.woff2');
src: url('degular/Degular-BoldItalic.woff2') format('woff2'),
url('degular/Degular-BoldItalic.woff') format('woff');
font-weight: bold;
font-style: italic;
}
/* fonts */
body {
font-family: 'Inter', sans-serif;
}
.btn {
font-family: 'Inter', sans-serif;
}
.rst-content .toctree-wrapper>p.caption,
h1,
h2,
h3,
h4,
h5,
h6,
legend {
font-family: 'Degular', sans-serif;
}
/* colors */
.rst-content .danger {
background: #FEEDEC;
}
.rst-content .danger .admonition-title {
background: #C3514D;
}
.rst-content .warning {
background: #FDF0E9;
}
.rst-content .warning .admonition-title {
background: #BC6437;
}
.rst-content .note {
background: #E8F2FF;
}
.rst-content .note .admonition-title {
background: #3787E5;
}
.rst-content .tip {
background: #E2F7F3;
}
.rst-content .tip .admonition-title {
background: #0CAE8E;
}
a, a:visited {
color: #3787E5;
}
a:hover {
color: #77B5FE;
}
.rst-content code,
.rst-content tt,
code {
color: #C3514D;
}
.wy-menu-vertical header,
.wy-menu-vertical p.caption {
color: #0DC09D;
}
.wy-menu-vertical a {
color: #EAEBEB;
}
.wy-menu-vertical a:hover {
background-color: #7B7B7B;
}
.wy-menu-vertical a:active {
background-color: #0DC09D;
}
.wy-side-nav-search {
background-color: #0DC09D;
}
.wy-side-nav-search input[type=text] {
border-color: #0DC09D;
}
.wy-side-nav-search img {
background-color: #0DC09D;
}
.wy-nav-side {
background: #160F26;
}
.wy-nav-top {
background: #0DC09D;
}
.wy-nav-top a {
color: #fff;
}
.wy-nav-top img {
width: auto;
background-color: inherit;
border-radius: unset;
}
footer {
color: #7B7B7B;
}
#search-results .context {
color: #7B7B7B;
}
.rst-content code.literal,
.rst-content tt.literal {
color: #C3514D;
}
/* Custom footer with logo */
.nav-footer-logo {
width: 300px;
margin-top: 2rem;
border-top: 1px solid #7B7B7B;
text-align: center;
font-size: 80%;
}
.nav-footer-logo a {
display: block;
padding: 2rem 0 0.5rem;
color: #7B7B7B;
}
.nav-footer-logo a img {
margin-top: 0.5rem;
width: 60%;
}
/* Version change admonitions */
div.deprecated,
div.versionadded,
div.versionchanged {
border-left: 0.2rem solid;
border-radius: 0.25rem;
box-shadow: 0 0.2rem 0.5rem var(--pst-color-shadow),0 0 0.0625rem var(--pst-color-shadow)!important;
margin: 1rem auto;
overflow: hidden;
padding: 0.2rem 0.6rem;
page-break-inside: avoid;
position: relative;
transition: color .25s,background-color .25s,border-color .25s;
vertical-align: middle;
}
div.deprecated>p,
div.versionadded>p,
div.versionchanged>p {
margin-top: 0.5rem;
margin-bottom: 0.5rem;
}
div.deprecated>p:first-child::before,
div.versionadded>p:first-child::before,
div.versionchanged>p:first-child::before {
font-family: FontAwesome;
display: inline-block;
font-style: normal;
font-weight: 400;
line-height: 1;
text-decoration: inherit;
margin-right: 0.5rem;
content: "";
-webkit-font-smoothing: antialiased;
padding: 0.4rem 0.6rem;
margin: -0.2rem 0rem -0.2rem -0.6rem;
}
div.versionadded { border-color: #0CAE8E; background-color: #E2F7F3; }
div.versionadded p:first-child::before { color: #0CAE8E; }
div.versionchanged { border-color: #BC6437; background-color: #FDF0E9; }
div.versionchanged p:first-child::before { color: #BC6437; }
div.deprecated { border-color: #C3514D; background-color: #FEEDEC; }
div.deprecated p:first-child::before { color: #C3514D; }
/* Version in sidebar */
.sidebar_version {
margin-top: 0.5rem;
font-size: 0.8rem;
color: #160F26;
}
.sidebar_version samp {
font-size: 0.6rem;
}
.sidebar_version select {
background-color: #0DC09D;
box-shadow: none;
color: #160F26;
border: 1px solid #160f26a3;
border-radius: 1rem;
padding: 0.15rem 0.2rem;
font-size: 0.7rem;
margin-left: 0.5rem;
}
/* Site-wide announcement banner */
.admonition.site-banner {
background: #E2F7F3;
border-left: none;
}
.admonition.site-banner .admonition-title {
background: #0CAE8E;
}

BIN
nextflow/docs/_static/timeline-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 51 KiB

BIN
nextflow/docs/_static/workflow-notification-min.png vendored Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

54
nextflow/docs/_templates/layout.html vendored Normal file
View File

@@ -0,0 +1,54 @@
{% extends "!layout.html" %}
{% block document %}
<div class="admonition site-banner" style="background: #E2F7F3; border-left: none;">
<p class="admonition-title" style="background: #0CAE8E;">Nextflow documentation is migrating</p>
<p>Nextflow documentation is being migrated to <a href="https://docs.seqera.io/nextflow/">docs.seqera.io/nextflow</a>. This site will remain available throughout the migration.</p>
</div>
{{ super() }}
{% endblock %}
{% block sidebartitle %}
{{ super() }}
{%- set version_selected = "stable" -%}
{%- if "edge" in release %}{%- set version_selected = "edge" -%}{%- endif %}
{%- if "dev" in release %}{%- set version_selected = "dev" -%}{%- endif %}
<div class="sidebar_version">
<span class="version">Version {{ release }} {{ release_checked_out }}</span>
<select onchange="window.location = this.value;">
<option value="/docs/stable/index.html" {% if version_selected == 'stable' %}selected{% endif %}>Stable</option>
<option value="/docs/edge/index.html" {% if version_selected == 'edge' %}selected{% endif %}>Edge</option>
<option value="/docs/latest/index.html" {% if version_selected == 'dev' %}selected{% endif %}>Latest</option>
</select>
</select>
</div>
{% endblock %}
{% block navigation %}
{{ super() }}
<div class="nav-footer-logo">
<a href="https://seqera.io/" target="_blank" title="Developed by Seqera Labs">
Nextflow is developed by:<br>
<img src="{{ pathto('_static/seqera-logo.svg', 1) }}" alt="Seqera Labs">
</a>
</div>
{% endblock %}
{% block extrahead %}
{{ super() }}
<!-- Google Tag Manager -->
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-TNCXSWG');</script>
<!-- End Google Tag Manager -->
{% endblock %}
{% block footer %}
{{ super() }}
<!-- Google Tag Manager (noscript) -->
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-TNCXSWG"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<!-- End Google Tag Manager (noscript) -->
{% endblock %}

99
nextflow/docs/amazons3.md Normal file
View File

@@ -0,0 +1,99 @@
(amazons3-page)=
# Amazon S3
Nextflow includes support for AWS S3 storage. Files stored in an S3 bucket can be accessed transparently in your pipeline script like any other file in the local file system.
## S3 path
In order to access an S3 file, you only need to prefix the file path with the `s3` schema and the `bucket` name where it is stored.
For example, if you need to access the file `/data/sequences.fa` stored in a bucket named `my-bucket`, that file can be accessed using the following fully qualified path:
```
s3://my-bucket/data/sequences.fa
```
The usual file operations can be applied to a path handle with the above notation. For example, the content of an S3 file can be printed as follows:
```nextflow
println file('s3://my-bucket/data/sequences.fa').text
```
See {ref}`working-with-files` and the {ref}`stdlib-types-path` reference to learn more about available file operations.
## Security credentials
AWS access credentials can be provided in two ways:
1. Using AWS access and secret keys in your pipeline configuration.
2. Using IAM roles to grant access to S3 storage on AWS EC2 instances.
### AWS access and secret keys
The AWS access and secret keys can be specified by using the `aws` section in the `nextflow.config` configuration file as shown below:
```groovy
aws {
accessKey = '<Your AWS access key>'
secretKey = '<Your AWS secret key>'
region = '<AWS region identifier>'
}
```
If the access credentials are not found in the above file, Nextflow looks for AWS credentials in the following order:
1. The `nextflow.config` file in the pipeline execution directory
2. The environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`
3. The environment variables `AWS_ACCESS_KEY` and `AWS_SECRET_KEY`
4. The profile in the AWS credentials file located at `~/.aws/credentials`
- Uses the `default` profile or the environment variable `AWS_PROFILE` if set
5. The profile in the AWS client configuration file located at `~/.aws/config`
- Uses the `default` profile or the environment variable `AWS_PROFILE` if set
6. The temporary AWS credentials provided by an IAM instance role. See [IAM Roles](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) documentation for details.
More information regarding [AWS Security Credentials](http://docs.aws.amazon.com/general/latest/gr/aws-security-credentials.html) are available in the AWS documentation.
### IAM roles with AWS EC2 instances
When running your pipeline in an EC2 instance, IAM roles can be used to grant access to AWS resources.
In this scenario, you only need to launch the EC2 instance with an IAM role which includes the `AmazonS3FullAccess` policy. Nextflow will detect and automatically acquire the permission to access S3 storage, without any further configuration.
Learn more about [Using IAM Roles to Delegate Permissions to Applications that Run on AWS EC2](http://docs.aws.amazon.com/IAM/latest/UserGuide/roles-usingrole-ec2instance.html) in the AWS documentation.
## China regions
To use an AWS China region, make sure to specify the corresponding AWS API S3 endpoint in the Nextflow configuration file as shown below:
```groovy
aws {
client {
endpoint = "https://s3.cn-north-1.amazonaws.com.cn"
}
}
```
Read more about AWS API endpoints in the [AWS documentation](https://docs.aws.amazon.com/general/latest/gr/s3.html)
## S3-compatible storage
To use S3-compatible object storage such as [Ceph](https://ceph.io) or [Minio](https://min.io) specify the endpoint of
your storage provider and enable the [S3 path style access](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access)
in your Nextflow configuration as shown below:
```groovy
aws {
accessKey = '<Your access key>'
secretKey = '<Your secret key>'
client {
endpoint = '<Your storage endpoint URL>'
s3PathStyleAccess = true
}
}
```
## Advanced configuration
Read {ref}`AWS configuration<config-aws>` section to learn more about advanced S3 client configuration options.

552
nextflow/docs/aws.md Normal file
View File

@@ -0,0 +1,552 @@
(aws-page)=
# Amazon Web Services
:::{tip}
This page describes how to manually set up and use Nextflow with AWS Cloud.
You may be interested in using [Batch Forge](https://docs.seqera.io/platform/latest/compute-envs/aws-batch) in [Seqera Platform](https://seqera.io/platform/),
which automatically creates the required AWS infrastructure for you with minimal intervention.
:::
## AWS security credentials
Nextflow uses the [AWS security credentials](https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html) to make programmatic calls to AWS services.
The AWS credentials are selected from the following sources, in order of descending priority:
1. Nextflow configuration file - `aws.accessKey` and `aws.secretKey`. See {ref}`AWS configuration<config-aws>` for more details.
2. A custom profile in `$HOME/.aws/credentials` and/or `$HOME/.aws/config`. The profile can be supplied from the `aws.profile` config option, or the `AWS_PROFILE` or `AWS_DEFAULT_PROFILE` environmental variables.
3. Environment variables - `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
4. The `default` profile in `~/.aws/credentials` and/or `~/.aws/config`.
5. Single Sign-On (SSO) credentials. See the [AWS documentation](https://docs.aws.amazon.com/cli/latest/userguide/sso-configure-profile-token.html) for more details.
:::{versionadded} 23.07.0-edge
:::
6. EC2 instance profile credentials. See the [AWS documentation](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html) and [this blog post](https://aws.amazon.com/blogs/security/granting-permission-to-launch-ec2-instances-with-iam-roles-passrole-permission/) for more details.
The AWS region is selected from the following sources, in order of descending priority:
1. Nextflow configuration file - `aws.region`
2. Environment variables - `AWS_REGION` or `AWS_DEFAULT_REGION`
3. EC2 instance metadata (if Nextflow is running in an EC2 instance)
SSO credentials and instance profile credentials are the most recommended because they don't require you to manage and distribute AWS keys explicitly. SSO credentials are ideal for launching pipelines from outside of AWS (e.g. your laptop), while instance profile credentials are ideal for launching pipelines within AWS (e.g. an EC2 instance).
## AWS IAM policies
[IAM policies](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) are the mechanism used by AWS to define permissions for IAM identities. In order to access certain AWS services, the proper policies must be attached to the identity associated to the AWS credentials.
Minimal permissions policies to be attached to the AWS account used by Nextflow are:
- To use AWS Batch:
```json
"batch:CancelJob"
"batch:DescribeComputeEnvironments"
"batch:DescribeJobDefinitions"
"batch:DescribeJobQueues"
"batch:DescribeJobs"
"batch:ListJobs"
"batch:RegisterJobDefinition"
"batch:SubmitJob"
"batch:TagResource"
"batch:TerminateJob"
```
- To view [EC2](https://aws.amazon.com/ec2/) instances:
```json
"ec2:DescribeInstanceAttribute"
"ec2:DescribeInstances"
"ec2:DescribeInstanceStatus"
"ec2:DescribeInstanceTypes"
"ecs:DescribeContainerInstances"
"ecs:DescribeTasks"
```
- To pull container images from [ECR](https://aws.amazon.com/ecr/) repositories:
```json
"ecr:BatchCheckLayerAvailability"
"ecr:BatchGetImage"
"ecr:DescribeImages"
"ecr:DescribeImageScanFindings"
"ecr:DescribeRepositories"
"ecr:GetAuthorizationToken"
"ecr:GetDownloadUrlForLayer"
"ecr:GetLifecyclePolicy"
"ecr:GetLifecyclePolicyPreview"
"ecr:GetRepositoryPolicy"
"ecr:ListImages"
"ecr:ListTagsForResource"
```
Alternatively, you can use AWS provided `AmazonEC2ContainerRegistryReadOnly` managed policy.
:::{note}
If you are running Fargate or Fargate Spot, you may need the following policies in addition to the listed above:
```json
"ec2:DescribeSubnets"
"ecs:CreateCluster"
"ecs:DeleteCluster"
"ecs:DescribeClusters"
"ecs:ListClusters"
```
:::
### S3 policies
Nextflow also requires policies to access [S3 buckets](https://aws.amazon.com/s3/) in order to use the work directory, pull input data, and publish results.
Depending on the pipeline configuration, the above actions can be done all in a single bucket but, more likely, spread across multiple buckets. Once the list of buckets used by the pipeline is identified, there are two alternative ways to give Nextflow access to these buckets:
1. Grant access to all buckets by attaching the policy `"s3:*"` to the IAM identity. This works only if buckets do not set their own access policies (see point 2);
2. For more fine grained control, assign to each bucket the following policy (replace the placeholders with the actual values):
```json
{
"Version": "2012-10-17",
"Id": "<my policy id>",
"Statement": [
{
"Sid": "<my statement id>",
"Effect": "Allow",
"Principal": {
"AWS": "<ARN of the nextflow identity>"
},
"Action": [
"s3:GetObject",
"s3:PutObject",
"s3:DeleteObject",
"s3:PutObjectTagging",
"s3:AbortMultipartUpload"
],
"Resource": "arn:aws:s3:::<bucket name>/*"
},
{
"Sid": "AllowSSLRequestsOnly",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<bucket name>",
"arn:aws:s3:::<bucket name>/*"
],
"Condition": {
"Bool": {
"aws:SecureTransport": "false"
}
}
}
]
}
```
See the [bucket policy documentation](https://docs.aws.amazon.com/config/latest/developerguide/s3-bucket-policy.html) for additional details.
(aws-batch)=
## AWS Batch
[AWS Batch](https://aws.amazon.com/batch/) is a managed computing service that enables the execution of containerized workloads on AWS cloud infrastructure. It dynamically provisions the optimal quantity and type of compute resources (e.g., CPU or memory optimized compute resources) based on the volume and specific resource requirements of the submitted jobs.
Nextflow natively supports AWS Batch, allowing you to run pipeline tasks as Batch jobs in the cloud.
See {ref}`AWS Batch executor <awsbatch-executor>` to learn more about the `awsbatch` executor.
(aws-batch-cli)=
### AWS CLI
:::{tip}
The recommended approach for running Nextflow pipelines with AWS Batch is to use {ref}`wave-page` and {ref}`fusion-page` for S3 access, rather than relying on the AWS CLI. This approach removes the requirement to install and configure the AWS CLI in containers or custom AMIs.
:::
Nextflow uses the [AWS command line tool](https://aws.amazon.com/cli/) (`aws`) to stage input files and output files between S3 and the task containers.
The `aws` command can be made available by either (1) installing it in the container image(s) or (2) installing it in a {ref}`custom AMI <aws-custom-ami>` to be used instead of the default AMI when configuring AWS Batch.
### Get started
1. In the AWS Console, navigate to **AWS Batch** and create a [Compute environment](http://docs.aws.amazon.com/batch/latest/userguide/compute_environments.html) (CE).
1. If you are using a custom AMI (see following sections), the AMI ID must be specified in the CE configuration
2. Make sure to select an AMI (either custom or existing) with Docker installed (see following sections)
3. Make sure the policy `AmazonS3FullAccess` (granting access to S3 buckets) is attached to the instance role configured for the CE
4. If you plan to use Docker images from Amazon ECS container, make sure the `AmazonEC2ContainerServiceforEC2Role` policy is also attached to the instance role
2. In the AWS Console, create (at least) one [Job Queue](https://docs.aws.amazon.com/batch/latest/userguide/job_queues.html) and bind it to the Compute environment.
3. In the AWS Console, create an S3 bucket for the work directory (see below). You can also create separate buckets for input data and results, as needed.
4. Make sure that every process in your pipeline specifies a Docker container with the {ref}`process-container` directive.
5. Make sure that all of your container images are published in a Docker registry that can be reached by AWS Batch, such as [Docker Hub](https://hub.docker.com/), [Quay](https://quay.io/), or [Elastic Container Registry](https://aws.amazon.com/ecr/).
### Configuration
To configure your pipeline for AWS Batch:
1. Specify the AWS Batch {ref}`executor <awsbatch-executor>`
2. Specify the AWS Batch queue with the {ref}`process-queue` directive
3. Specify any Batch job container options with the {ref}`process-containerOptions` directive.
An example `nextflow.config` file is shown below:
```groovy
process {
executor = 'awsbatch'
queue = 'my-batch-queue'
container = 'quay.io/biocontainers/salmon'
containerOptions = '--shm-size 16000000 --ulimit nofile=1280:2560 --ulimit nproc=16:32'
}
aws {
batch {
// NOTE: this setting is only required if the AWS CLI is installed in a custom AMI
cliPath = '/home/ec2-user/miniconda/bin/aws'
}
region = 'us-east-1'
}
```
:::{tip}
Each process can be configured with its own queue by using the {ref}`process-queue` directive in the process definition or via {ref}`config-process-selectors` in your Nextflow configuration.
:::
## Container Options
:::{versionadded} 21.12.1-edge
:::
The {ref}`process-containerOptions` directive can be used to control the properties of the container execution associated with each Batch job.
The following container options are currently supported:
```
-e, --env string
Set environment variables (format: <name> or <name>=<value>)
--init
Run an init inside the container that forwards signals and reaps processes
--memory-swap int
The total amount of swap memory (in MiB) the container can use: '-1' to enable unlimited swap
--memory-swappiness int
Tune container memory swappiness (0 to 100) (default -1)
--privileged
Give extended privileges to the container
--read-only
Mount the container's root filesystem as read only
--shm-size int
Size (in MiB) of /dev/shm
--tmpfs string
Mount a tmpfs directory (format: <path>:<options>,size=<int>), size is in MiB
-u, --user string
Username or UID (format: <name|uid>[:<group|gid>])
--ulimit string
Ulimit options (format: <type>=<soft limit>[:<hard limit>])
```
Container options may be passed in long form (e.g `--option value`) or short form (e.g. `-o value`) where available.
Few examples:
```nextflow
containerOptions '--tmpfs /run:rw,noexec,nosuid,size=128 --tmpfs /app:ro,size=64'
containerOptions '-e MYVAR1 --env MYVAR2=foo2 --env MYVAR3=foo3 --memory-swap 3240000 --memory-swappiness 20 --shm-size 16000000'
containerOptions '--ulimit nofile=1280:2560 --ulimit nproc=16:32 --privileged'
```
Check the [AWS documentation](https://docs.aws.amazon.com/batch/latest/APIReference/API_ContainerProperties.html) for further details.
(aws-custom-ami)=
## Custom AMI
There are several reasons why you might need to create your own [AMI (Amazon Machine Image)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AMIs.html) to use in your Compute Environments:
- You do not want to install the AWS CLI into each of your Docker images and would rather provide it through the AMI
- The existing AMI (selected from the marketplace) does not have Docker installed
- You need to attach more storage to your EC2 instance (the default ECS instance AMI has only a 30GB EBS volume which is not enough for most data pipelines)
- You need to install additional software that is not available in your Docker image
### Create your custom AMI
From the EC2 Dashboard, select **Launch Instance**, then select **Browse more AMIs**. In the new page, select
**AWS Marketplace AMIs**, and then search for `Amazon ECS-Optimized Amazon Linux 2023 (AL2023) x86_64 AMI`. Select the AMI and continue as usual to configure and launch the instance.
:::{note}
The selected instance has a root volume of 30GB. Make sure to increase its size or add a second EBS volume with enough storage for real genomic workloads.
:::
When the instance is running, SSH into it (or connect with the Session Manager service), install the AWS CLI, and install any other tool that may be required (see following sections).
Finally, select **Create Image** from the EC2 Dashboard to create a new AMI from the running instance (you can also do it through the AWS CLI).
The new AMI ID needs to be specified when creating the Batch Compute Environment.
:::{warning}
Any additional software must be installed on the EC2 instance *before* creating the AMI.
:::
(id2)=
### AWS CLI installation
:::{tip}
The need for the AWS CLI is considered a legacy requirement for the deployment of Nextflow pipelines with AWS Batch.
Instead, consider using {ref}`wave-page` and {ref}`fusion-page` to facilitate access to S3 without using the AWS CLI.
:::
The [AWS CLI](https://aws.amazon.com/cli) should be installed in your custom AMI using a self-contained package manager such as [Conda](https://conda.io). That way, you can control which version of Python is used by the AWS CLI (which is written in Python).
If you don't use Conda, the `aws` command will attempt to use the version of Python that is installed in the container, and it won't be able to find the necessary dependencies.
The following snippet shows how to install AWS CLI with [Miniconda](https://conda.io/miniconda.html) in the home folder:
```bash
cd $HOME
sudo yum install -y bzip2 wget
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash Miniconda3-latest-Linux-x86_64.sh -b -f -p $HOME/miniconda
$HOME/miniconda/bin/conda install -c conda-forge --override-channels -y awscli
rm Miniconda3-latest-Linux-x86_64.sh
```
Afterwards, verify that the AWS CLI package works correctly:
```console
$ ./miniconda/bin/aws --version
aws-cli/1.29.20 Python/3.11.4 Linux/4.14.318-241.531.amzn2.x86_64 botocore/1.31.20
```
:::{note}
The `aws` tool will be placed in a directory named `bin` in the main installation folder. Modifying this directory structure after the tool is installed will cause it to not work properly.
:::
To configure Nextflow to use this installation, specify the `aws.batch.cliPath` option in the Nextflow configuration as shown below:
```groovy
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws'
```
Replace the path above with the one matching the location where the `aws` tool is installed in your AMI.
:::{versionchanged} 19.07.0
The `executor.awscli` config option was replaced by `aws.batch.cliPath`.
:::
:::{warning}
The grandparent directory of the `aws` tool will be mounted into the container at the same path as the host, e.g. `/home/ec2-user/miniconda`, which will shadow existing files in the container. Make sure you use a path that is not already present in the container.
:::
### Docker installation
Docker is required by Nextflow to execute tasks on AWS Batch. The **Amazon ECS-optimized Amazon Linux 2023 AMI** has Docker installed, however, if you create your AMI from a different AMI that does not have Docker installed, you will need to install it manually.
The following snippet shows how to install Docker on an Amazon EC2 instance:
```bash
# install Docker
sudo yum update -y
sudo yum install docker
# start the Docker service
sudo service docker start
# empower your user to run Docker without sudo
sudo usermod -a -G docker ec2-user
```
You must log out and log back in again to use the new `ec2-user` permissions.
These steps must be done *before* creating the AMI from the current EC2 instance.
### Amazon ECS container agent installation
The [ECS container agent](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_agent.html) is a component of Amazon Elastic Container Service (Amazon ECS) and is responsible for managing containers on behalf of ECS. AWS Batch uses ECS to execute containerized jobs, therefore it requires the agent to be installed on EC2 instances within your Compute Environments.
The ECS agent is included in the **Amazon ECS-optimized Amazon Linux 2023 AMI** . If you use a different base AMI, you can also install the agent on any EC2 instance that supports the Amazon ECS specification.
To install the agent, follow these steps:
```bash
sudo yum install ecs-init
sudo systemctl enable --now ecs
```
To test the installation:
```bash
curl -s http://localhost:51678/v1/metadata | python -mjson.tool
```
:::{note}
The `AmazonEC2ContainerServiceforEC2Role` policy must be attached to the instance role in order to be able to connect the EC2 instance created by the Compute Environment to the ECS container.
:::
:::{note}
The `AmazonEC2ContainerRegistryReadOnly` policy should be attached to the instance role in order to get read-only access to Amazon EC2 Container Registry repositories.
:::
## Jobs & Execution
### Custom job definition
Nextflow automatically creates the Batch [Job definitions](http://docs.aws.amazon.com/batch/latest/userguide/job_definitions.html) needed to execute tasks in your pipeline, so you don't need to define them beforehand.
However, sometimes you may still need to specify a custom **Job Definition** to fine tune the configuration of a specific job, for example to define custom mount paths.
To do that, first create a **Job Definition** in the AWS Console (or by other means). Note the name of the Job definition you created. You can then associate a process execution with this Job definition by using the {ref}`process-container` directive and specifying, in place of the container image name, the Job definition name prefixed by `job-definition://`, as shown below:
```groovy
process.container = 'job-definition://your-job-definition-name'
```
### Hybrid workloads
Nextflow allows the use of multiple executors in the same workflow application. This feature enables the deployment of hybrid workloads in which some jobs are executed in the local computer or local computing cluster and some jobs are offloaded to AWS Batch.
To enable this feature, use one or more {ref}`config-process-selectors` in your Nextflow configuration to apply the AWS Batch configuration to the subset of processes that you want to offload. For example:
```groovy
process {
withLabel: bigTask {
executor = 'awsbatch'
queue = 'my-batch-queue'
container = 'my/image:tag'
}
}
aws {
region = 'eu-west-1'
}
```
With the above configuration, processes with the `bigTask` {ref}`process-label` will run on AWS Batch, while the remaining processes will run in the local computer.
Then launch the pipeline with the -bucket-dir option to specify an AWS S3 path for the jobs computed with AWS Batch and, optionally, the -work-dir to specify the local storage for the jobs computed locally:
```bash
nextflow run <script or project name> -bucket-dir s3://my-bucket/some/path
```
:::{warning}
The AWS S3 path needs to contain at least one sub-directory (e.g. `s3://my-bucket/work` rather than `s3://my-bucket`).
:::
:::{note}
Nextflow will automatically manage the transfer of input and output files between the local and cloud environments when using hybrid workloads.
:::
### Volume mounts
:::{versionadded} 19.07.0
:::
User provided container volume mounts can be provided as shown below:
```groovy
aws {
region = 'eu-west-1'
batch {
volumes = '/tmp'
}
}
```
Multiple volumes can be specified as a comma-separated list of paths. The usual Docker volume mount syntax can be used to specify complex volumes where the container path is different from the host path or the volume should be *read-only*. For example:
```groovy
aws {
region = 'eu-west-1'
batch {
volumes = ['/tmp', '/host/path:/mnt/path:ro']
}
}
```
The above snippet defines two volume mounts for the jobs executed in your pipeline. The first volume mounts the host path `/tmp` to the same path in the container, with the *read-write* access mode. The second volume mounts the host path `/host/path` to `/mnt/path` in the container, with the *read-only* access mode.
### Troubleshooting
**Problem**: The Pipeline execution terminates with an AWS error message similar to the one shown below:
```
JobQueue <your queue> not found
```
Make sure you have defined a AWS region in the Nextflow configuration file and it matches the region in which your Batch environment has been created.
**Problem**: A process execution fails reporting the following error message:
```
Process <your task> terminated for an unknown reason -- Likely it has been terminated by the external system
```
This may happen when Batch is unable to execute the process script. A common cause of this problem is that the Docker container image you have specified uses a non standard [entrypoint](https://docs.docker.com/engine/reference/builder/#entrypoint) which does not allow the execution of the Bash launcher script required by Nextflow to run the job.
This may also happen if the AWS CLI doesn't run correctly.
Other places to check for error information:
- The `.nextflow.log` file.
- The Job execution log in the AWS Batch dashboard.
- The [CloudWatch](https://aws.amazon.com/cloudwatch/) logs found in the `/aws/batch/job` log group.
**Problem**: A process execution is stalled in the `RUNNABLE` status and the pipeline output is similar to the one below:
```
executor > awsbatch (1)
process > <your process> (1) [ 0%] 0 of ....
```
It may happen that the pipeline execution hangs indefinitely because one of the jobs is held in the queue and never gets executed. In AWS Console, the queue reports the job as `RUNNABLE` but it never moves from there.
There are multiple reasons why this can happen. They are mainly related to the Compute Environment workload/configuration, the docker service or container configuration, network status, etc.
This [AWS page](https://aws.amazon.com/premiumsupport/knowledge-center/batch-job-stuck-runnable-status/) provides several resolutions and tips to investigate and work around the issue.
(aws-fargate)=
## AWS Fargate
:::{versionadded} 23.12.0-edge
:::
Nextflow provides experimental support for the execution of [AWS Batch jobs with Fargate resources](https://docs.aws.amazon.com/batch/latest/userguide/fargate.html).
AWS Fargate is a technology that you can use with AWS Batch to run containers without having to manage servers or EC2 instances.
With AWS Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers.
To enable the use of AWS Fargate in your pipeline use the following settings in your `nextflow.config` file:
```groovy
process.executor = 'awsbatch'
process.queue = '<AWS BATCH QUEUE>'
aws.region = '<AWS REGION>'
aws.batch.platformType = 'fargate'
aws.batch.jobRole = 'JOB ROLE ARN'
aws.batch.executionRole = 'EXECUTION ROLE ARN'
wave.enabled = true
```
See the AWS documentation for details how to create the required AWS Batch queue for Fargate, the Batch Job Role
and the Batch Execution Role.
:::{note}
Nextflow uses [s5cmd](https://github.com/peak/s5cmd) to download the task input data and upload the task outputs.
To enable this capability, you need to enable the Wave service in the Nextflow configuration, as shown in the above example. See {ref}`Wave <wave-page>` documentation for more details.
:::
## Advanced configuration
Read the {ref}`AWS configuration<config-aws>` section to learn more about advanced configuration options.

718
nextflow/docs/azure.md Normal file
View File

@@ -0,0 +1,718 @@
(azure-page)=
# Azure
## Overview
Nextflow provides built-in support for Azure Cloud Services. It enables you to:
- Store and access data using Azure Blob Storage and Azure file shares.
- Execute workflows using Azure Batch.
:::{tip}
For automated Azure infrastructure setup, consider using [Batch Forge](https://docs.seqera.io/platform/latest/compute-envs/azure-batch#compute-environment) in [Seqera Platform](https://seqera.io/platform/).
:::
## Quick start
To run pipelines with Azure Batch:
1. Create an Azure Batch account in the Azure portal.
2. Increase the quotas in your Azure Batch account to the pipeline's needs. Quotas impact the number of pools, CPUs, and jobs you can create.
3. Create a storage account and Azure Blob Storage in the same region as the Batch account.
4. Add the following settings to your Nextflow configuration file:
- Add authentication and account details. See [Authentication](#authentication) for configuration examples.
- Set `process.executor` to `azurebatch` to make Nextflow submit tasks to Azure Batch.
- Set `workDir` to a working directory on Azure Blob Storage. For example, `az://<BLOB_CONTAINER>/work`, where `BLOB_CONTAINER` is a blob container in your storage account.
5. Launch your pipeline with the above configuration:
```bash
nextflow run <PIPELINE_NAME>
```
:::{tip}
You can list Azure regions with:
```bash
az account list-locations -o table
```
:::
## Authentication
Nextflow supports three authentication methods for Azure services, listed in order of security and recommended usage:
- [Managed identities](#managed-identities): The most secure option, available only within Azure. Uses Azure-managed credentials without storing secrets.
- [Service principals](#service-principals): A secure and flexible method that works across environments. Uses Microsoft Entra credentials for authentication.
- [Access keys](#access-keys): The most basic and least secure method. Relies on direct access keys for authentication.
### Required roles
The following role assignments are required to use Azure services:
For Azure Storage:
- Storage Blob Data Reader
- Storage Blob Data Contributor
For Azure Batch:
- Batch Data Contributor
To assign roles to a managed identity or service principal, see the [Azure documentation](https://learn.microsoft.com/en-us/azure/role-based-access-control/role-assignments-portal?tabs=current).
(azure-managed-identities)=
### Managed identities
:::{versionadded} 24.05.0-edge
:::
[Managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/overview) is the recommended authentication method when running Nextflow within Azure. It automatically manages credentials without requiring you to store secrets.
When using a managed identity, an Azure resource can authenticate based on what it is rather than using access keys. For example, Nextflow can submit jobs to Azure Batch and access private storage accounts without additional credentials if it is running on Azure Virtual Machines with a managed identity and the necessary permissions.
There are two types of managed identities:
- System-assigned managed identity
- User-assigned managed identity
**System-assigned managed identity**
A system-assigned identity is tied to a specific Azure resource. To use it:
1. Enable [system-assigned managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-to-configure-managed-identities?pivots=qs-configure-portal-windows-vm#system-assigned-managed-identity) on your Azure resource.
2. Configure the required role assignments. See [Required roles](#required-roles) for more information.
3. Add the following configuration:
```groovy
azure {
managedIdentity {
system = true
}
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
}
batch {
accountName = '<BATCH_ACCOUNT_NAME>'
location = '<BATCH_ACCOUNT_LOCATION>'
}
}
```
Replace the following:
- `STORAGE_ACCOUNT_NAME`: your Azure storage account name
- `BATCH_ACCOUNT_NAME`: your Azure Batch account name
- `BATCH_ACCOUNT_LOCATION`: your Azure Batch account location
:::{tip}
Nextflow uses the `AZURE_MANAGED_IDENTITY_SYSTEM` environment variable if the managed identity is not set in the Nextflow configuration file. Set this to `true` to enable a system-assigned managed identity
:::
**User-assigned managed identity**
A user-assigned identity can be shared across multiple Azure resources and its lifecycle is not tied to any specific resource. See the [Azure documentation](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/managed-identity-best-practice-recommendations#choosing-system-or-user-assigned-managed-identities) for more information.
To use user-assigned managed identity:
1. Create a [managed identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-manage-user-assigned-managed-identities?pivots=identity-mi-methods-azp#create-a-user-assigned-managed-identity).
2. Configure the required role assignments.
3. [Assign the identity](https://learn.microsoft.com/en-us/entra/identity/managed-identities-azure-resources/how-to-configure-managed-identities?pivots=qs-configure-portal-windows-vm#user-assigned-managed-identity) to your Azure resource.
4. Add the following configuration:
```groovy
azure {
managedIdentity {
clientId = '<USER_ASSIGNED_MANAGED_IDENTITY_CLIENT_ID>'
}
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
}
batch {
accountName = '<BATCH_ACCOUNT_NAME>'
location = '<BATCH_ACCOUNT_LOCATION>'
}
}
```
Replace the following:
- `USER_ASSIGNED_MANAGED_IDENTITY_CLIENT_ID`: your user assigned managed identity client ID
- `STORAGE_ACCOUNT_NAME`: your Azure Storage account name
- `BATCH_ACCOUNT_NAME`: your Azure Batch account name
- `BATCH_ACCOUNT_LOCATION`: your Azure Batch account location
:::{tip}
Nextflow uses the `AZURE_MANAGED_IDENTITY_USER` environment variable if the managed identity client ID is not provided in the Nextflow configuration file.
:::
### Service principals
:::{versionadded} 22.11.0-edge
:::
[Service principal](https://learn.microsoft.com/en-us/entra/identity-platform/app-objects-and-service-principals) credentials can be used to access Azure Batch and Storage accounts.
Similar to a managed identity, a service principal is an application identity that can have specific permissions and role-based access. Unlike managed identities, service principals require a secret key for authentication. The secret key is less secure, but it allows you to authenticate with Azure resources when operating outside of the Azure environment, such as from your local machine or a non-Azure cloud.
To create a service principal, follow the steps in the [Register a Microsoft Entra app and create a service principal](https://learn.microsoft.com/en-us/entra/identity-platform/howto-create-service-principal-portal) guide and add the credentials to your Nextflow configuration file. For example:
```groovy
azure {
activeDirectory {
servicePrincipalId = '<SERVICE_PRINCIPAL_CLIENT_ID>'
servicePrincipalSecret = '<SERVICE_PRINCIPAL_CLIENT_SECRET>'
tenantId = '<TENANT_ID>'
}
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
}
batch {
accountName = '<BATCH_ACCOUNT_NAME>'
location = '<BATCH_ACCOUNT_LOCATION>'
}
}
```
:::{tip}
Nextflow uses the following environment variables if storage settings are not set in the Nextflow config file:
- `AZURE_CLIENT_ID`: your Azure service principal application ID
- `AZURE_CLIENT_SECRET`: your Azure service principal secret
- `AZURE_TENANT_ID`: your Azure Entra account tenant ID
:::
### Access keys
The basic authentication method uses direct access keys. While easy to set up, it's less secure than managed identities or service principals because it provides full access to your resources. This approach requires you to obtain and manage the access keys for your Azure Batch and Storage accounts.
You can find your access keys in the Azure portal:
- For Storage accounts, navigate to your Storage account and select **Access keys** in the left-hand navigation menu.
- For Batch accounts, navigate to your Batch account and select **Keys** in the left-hand navigation menu.
Keep access keys secure and rotate them regularly. Use environment variables instead of hardcoding keys in configuration files, especially in shared or version-controlled environments:
```groovy
azure {
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
accountKey = '<STORAGE_ACCOUNT_KEY>'
}
batch {
accountName = '<BATCH_ACCOUNT_NAME>'
accountKey = '<BATCH_ACCOUNT_KEY>'
location = '<LOCATION>'
}
}
```
You can also use a Shared Access Token (SAS) instead of an account key to provide time-limited access to your storage account:
```groovy
azure.storage.sasToken = '<SAS_TOKEN>'
```
:::{tip}
When creating a SAS token, make sure to enable `Read`, `Write`, `Delete`, `List`, `Add`, `Create` permissions for the `Container` and `Object` resource types. The value of `sasToken` should be stripped of the leading `?` character.
:::
:::{tip}
Nextflow uses the following environment variables if storage settings are not provided in the Nextflow configuration file:
- `AZURE_STORAGE_ACCOUNT_NAME`: your Azure Storage account name
- `AZURE_STORAGE_ACCOUNT_KEY`: your Azure Storage account key
- `AZURE_BATCH_ACCOUNT_NAME`: your Azure Batch account name
- `AZURE_BATCH_ACCOUNT_KEY`: your Azure Batch account key
- `AZURE_STORAGE_SAS_TOKEN`: your Azure Storage account SAS token
:::
## Azure Blob Storage
Azure Storage provides several services for storing and accessing data in Azure. Nextflow primarily supports Azure Blob Storage for most workloads. Limited support is available for Azure file shares for specific use cases.
Nextflow uses the `az://` URI scheme to enable transparent access to files stored in Azure Blob Storage. Once Azure Storage is configured, you can use this scheme to reference files in any blob container as if they were local. For example, the path `/path/to/file.txt` in a container named `my-container` can be accessed as follows:
```groovy
file('az://my-container/path/to/file.txt')
```
The Blob storage account name and key need to be provided in the Nextflow configuration file as shown below:
```groovy
azure {
storage {
accountName = '<BLOB_ACCOUNT_NAME>'
}
}
```
:::{tip}
Nextflow uses the following environment variables if storage settings are not provided in the Nextflow configuration file:
- `AZURE_STORAGE_ACCOUNT_NAME`: your Azure Storage account name
- `AZURE_STORAGE_ACCOUNT_KEY`: your Azure Storage account access key
- `AZURE_STORAGE_SAS_TOKEN`: a shared access signature (SAS) token for Azure Storage access
:::
(azure-batch)=
## Azure Batch
[Azure Batch](https://learn.microsoft.com/en-us/azure/batch/) is a managed computing service that enables large-scale parallel and high-performance computing (HPC) batch jobs in Azure. Nextflow provides a built-in executor for Azure Batch, allowing you to offload workflow tasks to the cloud.
Nextflow integrates seamlessly with Azure Batch to:
- Dynamically create and manage compute pools based on workflow requirements.
- Automatically scale nodes up and down as needed during workflow execution.
- Support both regular and low-priority VMs for cost optimization.
- Manage task dependencies and data transfers between Azure Storage and compute nodes.
This section describes how to configure and use Azure Batch with Nextflow for efficient cloud-based workflow execution. For comprehensive information about Azure Batch features and capabilities, refer to the [official Azure Batch documentation](https://learn.microsoft.com/en-us/azure/batch/).
### Overview
Nextflow integrates with Azure Batch by mapping its execution model to Azure Batch's structure. A Nextflow process corresponds to an Azure Batch job, and every execution of that process (a Nextflow task) becomes an Azure Batch task. These Azure Batch tasks are executed on compute nodes within an Azure Batch pool, which is a collection of virtual machines that can scale up or down based on an autoscale formula.
Nextflow manages these pools dynamically. You can assign processes to specific, pre-existing pools using the process `queue` directive. Nextflow will create it if it doesn't exist and `azure.batch.allowPoolCreation` is set to `true`. Alternatively, `autoPoolMode` enables Nextflow to automatically create multiple pools based on the CPU and memory requirements defined in your processes.
An Azure Batch task is created for each Nextflow task. This task first downloads the necessary input files from Azure Blob Storage to its assigned compute node. It then runs the process script. Finally, it uploads any output files back to Azure Blob Storage.
Nextflow ensures that the corresponding Azure Batch jobs are marked as terminated when the pipeline completes. The compute pools, if configured for auto-scaling, automatically scale down according to their defined formula to minimize costs. Nextflow can also be configured to delete these node pools after pipeline completion.
To use Azure Batch, set the `executor` directive to `azurebatch` in your Nextflow configuration file and add a work directory on Azure Blob Storage:
```groovy
process {
executor = 'azurebatch'
}
workDir = 'az://<BLOB_CONTAINER>/work'
azure {
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
}
batch {
location = '<LOCATION>'
accountName = '<BATCH_ACCOUNT_NAME>'
autoPoolMode = true
allowPoolCreation = true
}
}
```
:::{note}
The work directory must be a subdirectory, for example, `az://container/work`, not `az://container`.
:::
Finally, launch your pipeline with the above configuration:
```bash
nextflow run <PIPELINE_NAME>
```
### Quotas
Azure Batch enforces quotas on resources like pools, jobs, and VMs.
- **Pools:** There is a limit on the number of pools that can exist at one time. Nextflow throws an error if it attempts to create a pool when this quota is exhausted. To proceed, you must delete existing pools.
- **Active jobs:** There is a limit on the number of active jobs. Completed jobs should be terminated. If this limit is reached, Nextflow throws an error. You must terminate or delete jobs before you run additional pipelines.
- **VM cores:** Each Azure VM family has a maximum core quota. If you request a VM family size for which you lack sufficient core quota, any jobs and tasks created by Nextflow will remain pending, as the pool cannot scale due to the quota. This can cause your pipeline to hang indefinitely.
You can increase your quota on the Azure Portal by opening your Batch account and selecting **Quotas** in the left-hand menu, then request an increase in quota numbers.
### Pools
Nextflow supports two approaches for managing Batch pools: auto pools and named pools. These approaches are described in the following sections.
:::{warning}
Clean up the Batch pools or use auto-scaling to avoid additional charges in the Batch account.
:::
### Auto pools
When using the `autoPoolMode` mode, Nextflow automatically creates pools of compute nodes appropriate for your pipeline.
The `cpus` and `memory` directives are used to find the smallest machine type in the Azure machine family, specified by the `machineType` directive, that satisfies the requested resources. If memory is not specified, 1 GB of memory is requested per CPU. When no options are specified, it uses one compute node of type *Standard_D4_v3*.
To use `autoPoolMode`, enable it in the `azure.batch` config scope:
```groovy
azure {
batch {
autoPoolMode = true
allowPoolCreation = true
}
}
process {
machineType = 'Standard_D4_v3'
cpus = 4
memory = 16.GB
}
```
You can specify multiple machine families per process using glob patterns in the `machineType` directive:
```groovy
// D or E v5 machines with data disk
process.machineType = "Standard_D*d_v5,Standard_E*d_v5"
```
For example, the following process creates a pool of *Standard_E8d_v5* machines when using `autoPoolMode`:
```groovy
process EXAMPLE_PROCESS {
machineType "Standard_E*d_v5"
cpus 8
memory 8.GB
script:
"""
echo "cpus: ${task.cpus}"
"""
}
```
Additional notes:
- For tasks using fewer than 4 CPUs, Nextflow creates pools with 4x CPUs to enable task packing.
- For tasks using fewer than 8 CPUs, Nextflow uses 2x CPUs.
- This behavior can be overridden by specifying an exact machine type (e.g., *Standard_E2d_v5*).
- Regexes can be used to avoid certain machine types (e.g., `Standard_*[^p]_v*` avoids ARM-based machines).
- The pool is not removed when the pipeline terminates unless `azure.batch.deletePoolsOnCompletion` is enabled in your Nextflow configuration file.
### Named pools
The `queue` directive can be used to specify the ID of the Azure Batch compute pool that should be used for a process.
When `azure.batch.allowPoolCreation = true` is set in your configuration, Nextflow creates the pools on-demand. Otherwise, the pool must be available in the Batch environment when Nextflow submits tasks.
Each pool can be configured separately in the Nextflow configuration. For example:
```groovy
azure {
batch {
pools {
small {
vmType = 'Standard_D2_v2'
vmCount = 5
}
large {
vmType = 'Standard_D8_v3'
vmCount = 2
}
}
}
}
process {
withLabel: small_jobs {
queue = 'small'
}
withLabel: large_jobs {
queue = 'large'
}
}
```
The above example defines the configuration for two node pools, `small` and `large`. The `small` pool provisions 10 compute nodes of type *Standard_D2_v2*, and the `large` pool provisions 5 nodes of type *Standard_E2_v3*. See the {ref}`config-azure` configuration scope for the list of available configuration options.
:::{warning}
Pool names may contain alphanumeric characters, hyphens, and underscores. Hyphenated names must be quoted, e.g., `'pool-1'`.
:::
**Requirements for pre-existing named pools**
The target pool must satisfy the following requirements when Nextflow is configured to use a pool that is already available in the Batch account:
- The pool must be declared as `dockerCompatible` (Container Type property).
- The task slots per node must match the number of cores for the selected VM. Otherwise, Nextflow returns an error of the form "Azure Batch pool 'ID' slots per node does not match the VM num cores (slots: N, cores: Y)".
- Unless you are using Fusion, all tasks must have the `azcopy` command line tool available in the path. If `azure.batch.copyToolInstallMode = 'node'`, every node must have `azcopy` available at `$AZ_BATCH_NODE_SHARED_DIR/bin/`.
### Auto-scaling
Azure Batch can automatically scale pools based on the parameters you define. With automatic scaling, Batch dynamically adds nodes to a pool as task demands increase and removes nodes as task demands decrease.
To enable this feature for pools created by Nextflow, add `autoScale = true` to the corresponding pool configuration scope. For example, when using the `autoPoolMode`:
```groovy
azure {
batch {
pools {
<POOL_NAME> {
autoScale = true
vmCount = 5 // Initial count
maxVmCount = 50 // Maximum count
}
}
}
}
```
The default auto-scaling formula initializes the pool with the number of VMs specified by the `vmCount` option. It dynamically scales up the pool based on the number of pending tasks. The pool limit is defined by `maxVmCount`. The formula automatically scales the pool down to zero nodes to minimize costs when no jobs are submitted for execution. The default formula is shown below:
```
// Get pool lifetime since creation.
lifespan = time() - time("{{poolCreationTime}}");
interval = TimeInterval_Minute * {{scaleInterval}};
// Compute the target nodes based on pending tasks.
$samples = $PendingTasks.GetSamplePercent(interval);
$tasks = $samples < 70 ? max(0, $PendingTasks.GetSample(1)) : max( $PendingTasks.GetSample(1), avg($PendingTasks.GetSample(interval)));
$targetVMs = $tasks > 0 ? $tasks : max(0, $TargetDedicatedNodes/2);
targetPoolSize = max(0, min($targetVMs, {{maxVmCount}}));
// For first interval deploy 1 node, for other intervals scale up/down as per tasks.
$TargetDedicatedNodes = lifespan < interval ? {{vmCount}} : targetPoolSize;
$NodeDeallocationOption = taskcompletion;
```
The `azure.batch.pools.<POOL_NAME>.scaleFormula` setting can be used to specify custom formulas.
### Task authentication
By default, Nextflow creates SAS tokens for specific containers and passes them to tasks to enable file operations with Azure Storage. SAS tokens expire after a set period of time. The expiration time is 48 hours by default and can be configured using `azure.storage.tokenDuration` in your configuration.
:::{versionadded} 25.05.0-edge
:::
:::{warning}
Only available when using [Fusion filesystem](fusion.md)!
:::
You can also authenticate to Azure Storage using a managed identity when using Fusion.
To do this:
1. Create a user-assigned managed identity with the Azure Storage Blob Data Contributor role for your storage account.
2. Attach the user-assigned managed identity to the node pool manually.
3. Set `azure.batch.poolIdentityClientId` to this identity in your configuration.
```groovy
azure {
batch {
poolIdentityClientId = '<MANAGED_IDENTITY_CLIENT_ID>'
}
}
```
Each task authenticates as the managed identity, and downloads and uploads files to Azure Storage using these credentials. It is possible to attach more than one managed identity to a pool. Fusion uses the identity specified by `azure.batch.poolIdentityClientId`.
### Task packing
Each Azure Batch node is allocated a specific number of task slots that determine how many tasks can run concurrently on that node. The number of slots is calculated based on the virtual machine's available CPUs. Nextflow intelligently assigns task slots to each process based on the proportion of total node resources requested through the `cpus`, `memory`, and `disk` directives.
For example, consider a *Standard_D4d_v5* machine with 4 vCPUs, 16 GB of memory, and 150 GB local disk:
- If a process requests `cpus 2`, `memory 8.GB`, or `disk 75.GB`, two task slots are allocated (50% of resources), allowing two tasks to run concurrently on the node.
- If a process requests `cpus 4`, `memory 16.GB`, or `disk 150.GB`, four task slots are allocated (100% of resources), allowing one task to run on the node.
Resource overprovisioning can occur if tasks consume more than their allocated share of resources. For instance, the node described above may become overloaded and fail if a task with `cpus 2` uses more than 8 GB of memory or 75 GB of disk space. Make sure to accurately specify resource requirements to ensure optimal performance and prevent task failures.
:::{warning}
Azure virtual machines come with fixed storage disks that are not expandable. Tasks will fail if the tasks running concurrently on a node use more storage than the machine has available.
:::
### Task containers
Every process in your pipeline must specify a container in order to be executed on Azure Batch.
Nextflow supports both public and private container images. You can pull public images from repositories such as <https://community.wave.seqera.io/>. You must authenticate for any private images that you use. Ensure the Batch pool has access when using a private registry like Azure Container Registry (ACR). Provide ACR credentials in the `azure.registry` configuration scope:
```groovy
azure {
registry {
server = '<REGISTRY_SERVER>' // e.g., 'myregistry.azurecr.io'
userName = '<REGISTRY_USER>'
password = '<REGISTRY_PASSWORD>'
}
}
```
:::{tip}
Nextflow uses the following environment variables if the registry credentials are not provided in the Nextflow configuration file:
- `AZURE_REGISTRY_USER_NAME`: the username for Azure Container Registry authentication
- `AZURE_REGISTRY_PASSWORD`: the password for Azure Container Registry authentication
:::
Container images from public registries such as Docker Hub can be used without additional configuration, even when a private registry is specified in the `azure.registry` scope. The Docker runtime on Azure Batch VMs automatically uses the appropriate registry based on the container image specified for the task.
### VM images
When Nextflow creates a pool of compute nodes, it selects:
- The virtual machine image reference (defines the operating system (OS) and software to be installed on the node).
- The Batch node agent SKU (a program that runs on each node and provides the interface between the node and the Azure Batch service).
These settings determine the operating system, version, and available features for each compute node in your pool.
By default, Nextflow creates pool nodes suitable for most bioinformatics workloads based on Ubuntu 22.04. You can customize the OS in your pool configuration to meet specific requirements.
Below are example configurations for common operating systems used in scientific computing:
**Ubuntu 24.04 (default)**
```groovy
azure {
batch {
pools {
<POOL_NAME> {
sku = "batch.node.ubuntu 24.04"
offer = "ubuntu-hpc"
publisher = "microsoft-dsvm"
}
}
}
}
```
**CentOS 8**
```groovy
azure {
batch {
pools {
<POOL_NAME> {
sku = "batch.node.centos 8"
offer = "centos-container"
publisher = "microsoft-azure-batch"
}
}
}
}
```
### Advanced features
**Virtual networks**
Pools can be configured to use virtual networks to connect to your existing network infrastructure.
```groovy
azure.batch.pools.<POOL_NAME>.virtualNetwork = '<SUBNET_ID>'
```
The subnet ID must be in the following format:
```
/subscriptions/<SUBSCRIPTION>/resourceGroups/<GROUP>/providers/Microsoft.Network/virtualNetworks/<VNET>/subnets/<SUBNET>
```
Nextflow can submit tasks to existing pools that are already attached to a virtual network without requiring additional configuration.
:::{warning}
Virtual networks require Microsoft Entra authentication (service principal or managed identity).
:::
**Start tasks**
Start tasks are optional commands that run when a node joins the pool. They are useful for setting up the node environment or for running other commands that need to be executed when the node is created.
```groovy
azure {
batch {
pools {
<POOL_NAME> {
startTask {
script = 'echo "Hello, world!"'
privileged = true // optional, defaults to false
}
}
}
}
}
```
**Azure file shares**
Azure file shares provide fully managed file shares that can be mounted to compute nodes. Files become immediately available in the file system and can be accessed as local files within processes.
The Azure file share must exist in the storage account configured for Blob Storage. You must provide the name of the source Azure file share and mount path (the path where the files are mounted). Additional mount options can also be set for further customization of the mounting process. See the [Azure Files documentation](https://learn.microsoft.com/en-us/azure/storage/files/storage-files-introduction) for more information.
For example:
```groovy
azure {
storage {
fileShares {
references { // any name of your choice
mountPath = '/mnt/references'
mountOptions = '-o vers=3.0,dir_mode=0777,file_mode=0777,sec=ntlmssp' // optional
}
}
}
}
```
:::{warning}
File shares must be authenticated using a storage account key. Managed identity and service principal are not supported.
:::
**Hybrid workloads**
It is possible to use multiple executors in the same Nextflow pipeline. This feature enables the deployment of hybrid workloads in which some tasks are executed locally (or on an HPC cluster) and some tasks are offloaded to Azure Batch.
Use process selectors in your Nextflow configuration to execute specific processes with Azure Batch. For example:
```groovy
process {
withLabel: bigTask {
executor = 'azurebatch'
queue = 'my-batch-pool'
container = 'my/image:tag'
}
}
bucketDir = 'az://my-container/some/path'
azure {
storage {
accountName = '<STORAGE_ACCOUNT_NAME>'
accountKey = '<STORAGE_ACCOUNT_KEY>'
}
batch {
location = '<LOCATION>'
accountName = '<BATCH_ACCOUNT_NAME>'
accountKey = '<BATCH_ACCOUNT_KEY>'
}
}
```
With the above configuration, processes labeled as `bigTask` run on Azure Batch and use `bucketDir` as the work directory, while all other processes run locally and use the local work directory specified by `workDir`. Nextflow handles file transfers between the local and remote work directories.
Launch the pipeline with the above configuration:
```bash
nextflow run <PIPELINE_NAME>
```
:::{note}
When using Fusion, `bucketDir` is not needed as all tasks will use the remote work directory. Specify `workDir` instead.
:::
## Advanced configuration
See the {ref}`config-azure` configuration scope for the list of available configuration options.

246
nextflow/docs/cache-and-resume.md Normal file
View File

@@ -0,0 +1,246 @@
(cache-resume-page)=
# Caching and resuming
One of the core features of Nextflow is the ability to cache task executions and re-use them in subsequent runs to minimize duplicate work. Resumability is useful both for recovering from errors and for iteratively developing a pipeline. It is similar to [checkpointing](https://en.wikipedia.org/wiki/Application_checkpointing), a common practice used by HPC applications.
You can enable resumability in Nextflow with the `-resume` flag when launching a pipeline with `nextflow run`. In most cases, that is all you need to do and resumability will "just work". This page describes Nextflow's caching behavior in more detail in order to help advanced users understand how the cache works and troubleshoot it when it doesn't work.
## Task cache
All task executions are automatically saved to the task cache, regardless of the `-resume` option (so that you always have the option to resume later). The task cache is a key-value store, where each key-value pair corresponds to a previously-executed task.
The task cache is used in conjunction with the [work directory](#work-directory) to recover cached tasks in a resumed run. It is also used by the {ref}`cli-log` sub-command to query task metadata.
(cache-resume-task-hash)=
### Task hash
The task hash is computed from the following metadata:
- Session ID (see `workflow.sessionId` in the {ref}`stdlib-namespaces-workflow` namespace)
- Task name (see `name` in {ref}`trace-report`)
- Task container image (if applicable)
- Task {ref}`environment modules <process-module>` (if applicable)
- Task {ref}`Conda environment <process-conda>` (if applicable)
- Task {ref}`Spack environment <process-spack>` and {ref}`CPU architecture <process-arch>` (if applicable)
- Task {ref}`inputs <process-input>`
- *New in 25.10:* Task {ref}`eval commands <process-out-eval>`
- Task {ref}`script <process-script>`
- Global variables referenced in the task script
- *New in 23.10:* Task {ref}`process-ext` properties referenced in the task script
- Any {ref}`bundled scripts <bundling-executables>` used in the task script
- Whether the task is a {ref}`stub run <process-stub>`
:::{note}
Nextflow also includes an incrementing component in the hash generation process, which allows it to iterate through multiple hash values until it finds one that does not match an existing execution directory. This mechanism typically aligns with task retries (i.e., task attempts), however this is not guaranteed.
:::
Nextflow computes this hash for every task before it is executed. If resumability is enabled, Nextflow checks whether the task cache contains a matching hash and whether the task outputs are still present in the [work directory](#work-directory). If both conditions are met, the task is resumed; otherwise, it is re-executed.
Files are hashed differently depending on the caching mode. See the {ref}`process-cache` directive for more details.
### Task entry
The task entry is a serialized blob of the task metadata required to resume a task, including the fields used by the {ref}`trace-report` and the task input variables.
### Cache stores
The default cache store uses the `.nextflow/cache` directory, relative to the launch directory (i.e. `workflow.launchDir`), to store the task cache, with a separate subdirectory for each session ID backed by [LevelDB](https://github.com/dain/leveldb).
Due to the limitations of LevelDB, the database for a given session ID can only be accessed by one reader/writer at a time. This means, for example, that you cannot use `nextflow log` to query the task metadata for a pipeline run while it is still running.
:::{versionadded} 23.07.0-edge
:::
The cloud cache is an alternative cache store that uses cloud storage instead of the local cache directory. You can use it by setting the `NXF_CLOUDCACHE_PATH` environment variable to the desired cache path (e.g. `s3://my-bucket/cache`) and providing the necessary credentials.
The cloud cache is particularly useful when launching Nextflow from within the cloud, where the default cache would be lost once the pipeline completes and the VM instance is terminated. Furthermore, because it is backed by cloud storage, it can support multiple readers and writers.
## Work directory
While the [task cache](#task-cache) stores the task metadata for subsequent runs, the work directory stores various files used during a pipeline run.
Each task uses a unique directory based on its hash. When a task is created, Nextflow stages the task input files, script, and other helper files into the task directory. The task writes any output files to this directory during its execution, and Nextflow uses these output files for downstream tasks and/or publishing.
When a previous task is retrieved from the task cache on a resumed run, Nextflow then checks the corresponding task directory in the work directory. If all the required outputs are present and the exit code is valid, then the task is successfully cached; otherwise, the task is re-executed.
For this reason, it is important to preserve both the task cache (`.nextflow/cache`) and work directories in order to resume runs successfully. You can use the {ref}`cli-clean` command to delete specific runs from the cache.
## Troubleshooting
Cache failures occur when a task that was supposed to be cached was re-executed or a task that was supposed to be re-executed was cached.
Common causes of cache failures include:
- [Resume not enabled](#resume-not-enabled)
- [Cache directive disabled](#cache-directive-disabled)
- [Modified inputs](#modified-inputs)
- [Inconsistent file attributes](#inconsistent-file-attributes)
- [Race condition on a global variable](#race-condition-on-a-global-variable)
- [Non-deterministic process inputs](#non-deterministic-process-inputs)
### Resume not enabled
The `-resume` option is required to resume a pipeline. Ensure you enable `-resume` in your run command or your Nextflow configuration file.
### Cache directive disabled
The `cache` directive is enabled by default. However, you can disable or modify its behavior for a specific process. For example:
```nextflow
process FOO {
cache false
// ...
}
```
Ensure that the `cache` directive has not been disabled. See {ref}`process-cache` for more information.
### Modified inputs
Modifying inputs that are used in the task hash invalidates the cache. Common causes of modified inputs include:
- Changing input files
- Resuming from a different session ID
- Changing the process name
- Changing the calling workflow name
- Changing the task container image or Conda environment
- Changing the task script
- Changing a bundled script used by the task
Nextflow calculates a hash for an input file using its full path, last modified timestamp, and file size. If any of these attributes change, Nextflow re-executes the task.
:::{warning}
If a process modifies its input files, it cannot be resumed. Avoid processes that modify their own input files as this is considered an anti-pattern.
:::
### Inconsistent file attributes
Some shared file systems, such as NFS, may report inconsistent file timestamps, which can invalidate the cache when using the standard caching mode.
To resolve this issue, use the `'lenient'` {ref}`caching mode <process-cache>` to ignore the last modified timestamp and use only the file path and size.
(cache-global-var-race-condition)=
### Race condition on a global variable
Race conditions can disrupt the caching behavior of your pipeline. For example:
```nextflow
channel.of(1,2,3).map { v -> X=v; X+=2 }.view { v -> "ch1 = $v" }
channel.of(1,2,3).map { v -> X=v; X*=2 }.view { v -> "ch2 = $v" }
```
In the above example, `X` is declared in each `map` closure. Without the `def` keyword, the variable `X` is global to the entire script. Because operators are executed concurrently and `X` is global, there is a *race condition* that causes the emitted values to vary depending on the order of the concurrent operations. If these values were passed to a process as inputs, the process would execute different tasks during each run due to the race condition.
To resolve this issue, avoid declaring global variables in closures:
```nextflow
channel.of(1,2,3).map { v -> def X=v; X+=2 }.view { v -> "ch1 = $v" }
```
:::{versionadded} 25.04.0
The {ref}`strict syntax <strict-syntax-page>` does not allow global variables to be declared in closures.
:::
(cache-nondeterministic-inputs)=
### Non-deterministic process inputs
A process that merges inputs from different sources non-deterministically may invalidate the cache. For example:
```nextflow
workflow {
ch_bam = channel.of( ['1', '1.bam'], ['2', '2.bam'] )
ch_bai = channel.of( ['2', '2.bai'], ['1', '1.bai'] )
check_bam_bai(ch_bam, ch_bai)
}
process check_bam_bai {
input:
tuple val(id), file(bam)
tuple val(id), file(bai)
script:
"""
check_bam_bai $bam $bai
"""
}
```
In the above example, the inputs will be merged without matching on `id`, in a similar manner as the {ref}`operator-merge` operator. As a result, the inputs are incorrect and non-deterministic.
To resolve this issue, use the `join` operator to join the channels into a single input channel before invoking the process:
```nextflow
workflow {
ch_bam = channel.of( ['1', '1.bam'], ['2', '2.bam'] )
ch_bai = channel.of( ['2', '2.bai'], ['1', '1.bai'] )
check_bam_bai(ch_bam.join(ch_bai))
}
process check_bam_bai {
input:
tuple val(id), file(bam), file(bai)
script:
"""
check_bam_bai $bam $bai
"""
}
```
## Tips
### Resuming from a specific run
Nextflow resumes from the previous run by default. If you want to resume from an earlier run, simply specify the session ID for that run with the `-resume` option:
```bash
nextflow run rnaseq-nf -resume 4dc656d2-c410-44c8-bc32-7dd0ea87bebf
```
You can use the {ref}`cli-log` command to view all previous runs as well as the task executions for each run.
(cache-compare-hashes)=
### Comparing the hashes of two runs
One way to debug a resumed run is to compare the task hashes of each run using the `-dump-hashes` option.
1. Perform an initial run: `nextflow -log run_initial.log run <pipeline> -dump-hashes`
2. Perform a resumed run: `nextflow -log run_resumed.log run <pipeline> -dump-hashes -resume`
3. Extract the task hash lines from each log (search for `cache hash:`)
4. Compare the runs with a diff viewer
While some manual effort is required, the final diff can often reveal the exact change that caused a task to be re-executed.
:::{versionadded} 23.10.0
:::
When using `-dump-hashes json`, the task hashes can be more easily extracted into a diff. Here is an example Bash script to perform two runs and produce a diff:
```bash
nextflow -log run_1.log run $pipeline -dump-hashes json
nextflow -log run_2.log run $pipeline -dump-hashes json -resume
get_hashes() {
cat $1 \
| grep 'cache hash:' \
| cut -d ' ' -f 10- \
| sort \
| awk '{ print; print ""; }'
}
get_hashes run_1.log > run_1.tasks.log
get_hashes run_2.log > run_2.tasks.log
diff run_1.tasks.log run_2.tasks.log
```
You can then view the `diff` output or use a graphical diff viewer to compare `run_1.tasks.log` and `run_2.tasks.log`.
:::{versionadded} 25.04.0
Nextflow now has a built-in way to compare two task runs. See the {ref}`data-lineage-page` guide for details.
:::

547
nextflow/docs/cli.md Normal file
View File

@@ -0,0 +1,547 @@
(cli-page)=
# Command line interface
Nextflow provides a robust command line interface (CLI) for the management and execution of pipelines. This page explains the key concepts and common usage patterns for the CLI.
For the complete reference of all commands, subcommands, and options, see {ref}`cli-reference`.
:::{note}
Nextflow uses two types of command line flags:
- Nextflow options use a single dash (e.g., `-log`) and modify Nextflow's behavior.
- Pipeline parameters use a double dash (e.g., `--input`) and are passed to your pipeline script.
:::
## Pipeline execution
Pipeline execution is the core function of Nextflow. These commands run Nextflow workflows, either from local files or remote Git repositories. Nextflow handles downloading, caching, and executing pipelines with minimal user intervention.
### Launching a project
The `run` command executes pipeline scripts from local files or remote repositories. It automatically manages repository downloads, caching, and execution, supporting various Git providers and authentication methods.
See {ref}`cli-run` for more information.
**Local pipelines**
Run a pipeline from your local filesystem:
```console
$ nextflow run main.nf
```
**Remote pipelines**
Use the format `<organization>/<repository>` to run a pipeline directly from Git repositories:
```console
$ nextflow run nextflow-io/hello
```
Nextflow automatically:
1. Downloads the repository to `$HOME/.nextflow/assets/`
2. Caches it for future runs
3. Executes the main script
If you omit the organization, Nextflow searches cached pipelines first, then attempts to download from the `NXF_ORG` organization (default: `nextflow-io`).
You can also use full repository URLs:
```console
$ nextflow run https://github.com/nextflow-io/hello
```
**Private repositories**
Use the `-user` option to add credentials for private repositories:
```console
$ nextflow run organization/private-repo -user my-username
```
Alternatively, configure Git authentication. See {ref}`Git configuration <git-page>` for more information.
**Non-GitHub providers**
Use the `-hub` option specify Bitbucket, GitLab, or other Git providers:
```console
$ nextflow run organization/repo -hub bitbucket
```
**Revision selection**
Use the `-r` option to specify Git branches, tags, or commits:
```console
$ nextflow run nextflow-io/hello -r v1.1
$ nextflow run nextflow-io/hello -r dev-branch
$ nextflow run nextflow-io/hello -r a3f5c8e
```
:::{versionadded} 25.12.0-edge
:::
Nextflow downloads and stores each explicitly requested Git branch, tag, or commit ID in a separate directory path, enabling you to run multiple revisions of the same pipeline simultaneously. Downloaded revisions are stored in a subdirectory of the local project: `$NXF_ASSETS/.repos/<org>/<repo>/clones/<commitId>`.
:::{tip}
Use tags or commit IDs instead of branches for reproducible pipeline runs. Branch references change as development progresses over time.
:::
(cli-params)=
### Pipeline parameters
Pipeline parameters are values defined with `params` in your script. Override them on the command line using the `--` prefix to customize pipeline behavior without modifying code.
```console
$ nextflow run main.nf --input data.csv --output results
```
Parameter names support automatic conversion between kebab-case and camelCase:
```console
$ nextflow run main.nf --input-file data.csv # Becomes params.inputFile
```
Parameters without values are set to `true`:
```console
$ nextflow run main.nf --verbose # params.verbose = true
```
:::{warning}
Quote parameters containing wildcards to prevent shell expansion:
```console
$ nextflow run main.nf --files "*.fasta"
```
:::
**Parameter files**
For complex parameter sets, use YAML or JSON files with `-params-file`. This is cleaner than long command lines.
```json
{
"input": "data.csv",
"output": "results/",
"min_quality": 20
}
```
```console
$ nextflow run main.nf -params-file params.json
```
**Parameter precedence**
Nextflow applies parameters defined in multiple places in the following order (lowest to highest priority):
1. Script parameters (`params.foo = 'default'`)
2. Configuration parameters (see {ref}`config-params`)
3. Parameter files (`-params-file`)
4. Command line parameters (`--foo bar`)
### Kubernetes execution
The `kuberun` command executes pipelines entirely within a Kubernetes cluster. This experimental feature runs the Nextflow driver itself inside Kubernetes.
Use this when you want both the Nextflow driver and tasks running in Kubernetes. This differs from using the Kubernetes executor with `run`, where only tasks run in Kubernetes while the driver runs externally.
```console
$ nextflow kuberun nextflow-io/hello
```
See {ref}`cli-kuberun` for more information.
## Project management
Project management commands interact with Git-hosted pipelines. Nextflow integrates with Git providers (e.g., GitHub, GitLab, and Bitbucket) to treat pipelines as versioned projects and maintains a local cache in `$HOME/.nextflow/assets/`.
Use these commands to explore available pipelines, inspect their code, maintain your cache, and clone projects.
### Listing downloaded projects
The `list` command shows all pipelines currently downloaded to your local cache. This helps you track which projects are available offline and manage your cache directory. Pipelines are stored in `$HOME/.nextflow/assets/` by default.
```console
$ nextflow list
```
### Showing project information
The `info` command displays detailed metadata about a downloaded project, including its repository location, local path, main script, and available revisions.
Use this to understand a project's structure, see available versions, or verify which revision is currently checked out.
```console
$ nextflow info hello
project name: nextflow-io/hello
repository : https://github.com/nextflow-io/hello
local path : $HOME/.nextflow/assets/.repos/nextflow-io/hello
main script : main.nf
revisions :
> master (default)
mybranch
> v1.1 [t]
v1.2 [t]
```
This shows:
- The full project name and repository URL
- Where it's cached locally
- Which script runs by default
- Available revisions (branches and tags marked with `[t]`)
- Which revisions are currently checked out (marked with `>`)
### Pulling or updating projects
The `pull` command downloads a pipeline or updates an existing one to the latest version from its Git repository.
Use this to manually download pipelines before running them, update cached pipelines, or download pipelines for offline use.
```console
$ nextflow pull nextflow-io/hello
```
You can specify a particular revision to download.
```console
$ nextflow pull nextflow-io/hello -r mybranch
```
See {ref}`cli-pull` for more information.
### Viewing project code
The `view` command displays the contents of a pipeline's main script or lists all files in the repository.
Use this to quickly inspect pipeline code without opening files or explore the project structure. Specify `-l` option lists all repository files instead of showing script contents.
```console
$ nextflow view nextflow-io/hello
$ nextflow view nextflow-io/hello -l
```
See {ref}`cli-view` for more information.
### Cloning projects locally
The `clone` command copies a pipeline from the cache to a local directory and creates a full Git repository you can modify.
Use this when you want to modify an existing pipeline, create a derivative pipeline, or study a pipeline's structure. If you omit the target directory it uses the pipeline name.
```console
$ nextflow clone nextflow-io/hello my-hello
```
See {ref}`cli-clone` for more information.
### Deleting cached projects
The `drop` command removes a downloaded pipeline from the local cache.
Use this to free disk space by removing pipelines you no longer need. The project is deleted from `$HOME/.nextflow/assets/`. The next `run` will download it again if needed.
```console
$ nextflow drop nextflow-io/hello
```
See {ref}`cli-drop` for more information.
## Module management
:::{versionadded} 26.04.0
:::
The `module` command allows you to discover, install, and run modules through a centralized module registry. See {ref}`using-modules-page` and the {ref}`cli-module` command reference for more information.
## Secret management
The `secrets` command manages secure pipeline secrets.
Use this to store credentials securely, reference them in pipelines without exposing values, and manage sensitive data centrally across your organization.
```console
$ nextflow secrets list
$ nextflow secrets set AWS_ACCESS_KEY_ID
$ nextflow secrets delete AWS_ACCESS_KEY_ID
```
See {ref}`cli-secrets` for more information.
## Configuration and validation
Configuration and validation options and commands help you control and verify pipeline settings. Configuration options supplement pipeline configuration at runtime, while validation commands inspect how Nextflow interprets your configuration files, process definitions, and scripts.
Use these to customize pipeline configuration, debug configuration issues, verify settings, and catch issues before execution.
### Soft configuration override
The `-c` option adds your configuration on top of the defaults and merges them together.
Use this when you want to override specific settings while keeping other defaults intact. Multiple configuration files can be specified as a comma separated list.
```console
$ nextflow -c my.config run nextflow-io/hello
```
See {ref}`config-page` for more information.
### Hard configuration override
The `-C` option replaces all default configuration with your custom configuration files. Multiple configuration files can be specified as a comma separated list.
Use this when you want to ensure no default configurations interfere with your custom settings. Unlike `-c` which merges configurations, `-C` ensures only your specified file is used.
```console
$ nextflow -C my.config run nextflow-io/hello
```
See {ref}`config-page` for more information.
### Configuration inspection
The `config` command prints the resolved configuration for a pipeline.
Use this to debug configuration issues, verify which settings will be applied, understand configuration precedence, or inspect specific configuration properties.
```console
$ nextflow config
$ nextflow config nextflow-io/hello
```
See {ref}`cli-config` for more information.
### Process inspection
:::{versionadded} 23.10.0
:::
The `inspect` command analyzes process settings in a pipeline without executing it. It outputs container information in JSON or Nextflow configuration format.
Use this to determine which container images will be used by each process before running the pipeline.
```console
$ nextflow inspect nextflow-io/hello
$ nextflow inspect nextflow-io/hello -format json
```
See {ref}`cli-inspect` for more information.
### Script validation
The `lint` command analyzes Nextflow scripts and configuration files for syntax errors and code issues. It can also automatically format your code to maintain consistent style across your project.
Use this to catch syntax errors before execution, enforce consistent code formatting, or validate entire directories of Nextflow code.
```console
$ nextflow lint main.nf
```
See {ref}`cli-lint` for more information.
## Execution history
Execution history and maintenance commands manage past runs and clean up cached files. Nextflow maintains metadata about all executions and stores intermediate files in work directories.
Use these commands to review past executions, free disk space, troubleshoot failures, or explore data lineage.
### Execution logs
The `log` command displays execution history and details about past pipeline runs, such as run names, timestamps, and customizable output fields.
Use this to find run names for resuming, review execution history, or debug failed runs. The command shows recent executions by default, with options to view specific runs or customize output fields.
```console
$ nextflow log
$ nextflow log dreamy_euler
$ nextflow log last -f name,status,duration
```
See {ref}`cli-log` for more information.
### Work directory cleanup
The `clean` command removes work directories and cached intermediate files from past executions.
Use this to free disk space, clean up failed or test runs, or maintain your work directory. Use `-n` to perform a dry run and show what would be deleted. Use `-f` to delete files.
```console
$ nextflow clean -n
$ nextflow clean dreamy_euler -f
```
See {ref}`cli-clean` for more information.
### Data lineage
:::{versionadded} 25.04.0
:::
:::{warning} *Experimental: may change in a future release.*
:::
The `lineage` command explores data lineage and provenance for workflow executions and tracks relationships between inputs, outputs, and processing steps.
Use this to understand input/output relationships between tasks, trace data flow through the pipeline, or establish file provenance. Lineage tracking must be enabled in configuration.
```console
$ nextflow lineage
```
See {ref}`data-lineage-page` to get started and {ref}`cli-lineage` for more information.
## Seqera Platform
[Seqera Platform](https://seqera.io) is a comprehensive workflow orchestration platform that extends Nextflow with features for workflow management, monitoring, and collaboration.
Use these commands to authenticate with Seqera Platform and launch workflows directly to the Platform's managed infrastructure.
### Platform authentication
:::{versionadded} 25.10.0
:::
The `auth` command manages authentication credentials for Seqera Platform, saving access tokens for API interactions.
Use this to log in or out of the Platform, establishing or removing your authentication credentials.
```console
$ nextflow auth login
```
Additional authentication operations include checking login status, viewing configuration details, and logging out:
```console
$ nextflow auth status
$ nextflow auth config
$ nextflow auth logout
```
See {ref}`cli-auth` for more information.
### Platform workflow submission
:::{versionadded} 25.10.0
:::
The `launch` command submits a workflow to run on Seqera Platform's infrastructure instead of your local machine.
Use this to leverage Platform's cloud resources, monitoring capabilities, and execution management. The Platform handles resource provisioning, execution monitoring, and result storage.
```console
$ nextflow launch nextflow-io/hello
```
See {ref}`cli-launch` for more information.
## System utilities
System utilities provide administrative and development tools for managing Nextflow itself, interacting with remote filesystems, working with plugins, and debugging.
Use these commands for system administration, development, and testing.
### Interactive console
The `console` command launches an interactive Groovy console with Nextflow's execution context loaded.
Use this to test Nextflow DSL code interactively, debug expressions, explore Nextflow's APIs, or experiment with syntax. It opens a GUI or REPL depending on your environment.
```console
$ nextflow console
```
### Remote filesystem operations
The `fs` command performs filesystem operations on remote storage systems supported by Nextflow, such as S3, Google Cloud Storage, and Azure Blob Storage.
Use this to manage remote data, test cloud storage access, or perform bulk file operations without additional tools.
```console
$ nextflow fs list s3://my-bucket/data/
$ nextflow fs cat s3://my-bucket/data/file.txt
$ nextflow fs cp s3://my-bucket/data/file.txt s3://dest/
$ nextflow fs delete s3://my-bucket/data/file.txt
```
See {ref}`cli-fs` for more information.
### Plugins
The `plugin` command creates plugins, installs them, and executes plugin-specific operations.
See {ref}`cli-plugin` for more information.
**Plugin creation**
:::{versionadded} 25.04.0
:::
Use the `create` subcommand to create a new plugin scaffold for development:
```console
$ nextflow plugin create
```
**Plugin installation**
Use the `install` subcommand to install a plugin and extend Nextflow functionality:
```console
$ nextflow plugin install my-plugin
```
**Plugin execution**
Use the the format `plugin-name:command` to execute plugin-specific commands:
```console
$ nextflow plugin my-plugin:hello --alpha --beta
```
See individual plugin documentation for plugin specific commands.
### Nextflow updates
The `self-update` command updates Nextflow to a newer version. It downloads and installs the latest release or a specific version.
Use this to upgrade Nextflow, switch versions, or install edge releases. By default, it updates to the latest stable release. Specify a particular version or use the `NXF_EDGE` environment variable for development releases.
```console
$ nextflow self-update
$ NXF_EDGE=1 nextflow self-update
```
### Command help
The `help` command displays detailed usage information for any Nextflow command.
Use this to learn about command-specific options, refresh your memory about syntax, or discover available features for a particular command.
```console
$ nextflow help run
```
### Version information
The `-v` and `-version` options print Nextflow version information.
Use `-v` for minimal output showing version and build number.
```console
$ nextflow -v
nextflow version 24.04.0.5917
```
Use `-version` for detailed output showing creation date, citation, and website.
```console
$ nextflow -version
N E X T F L O W
version 24.04.0 build 5917
created 03-05-2024 15:07 UTC
cite doi:10.1038/nbt.3820
http://nextflow.io
```

221
nextflow/docs/conda.md Normal file
View File

@@ -0,0 +1,221 @@
(conda-page)=
# Conda environments
[Conda](https://conda.io/) is an open source package and environment management system that simplifies the installation and the configuration of complex software packages in a platform agnostic manner.
Nextflow has built-in support for Conda that allows the configuration of workflow dependencies using Conda recipes and environment files.
This allows Nextflow applications to use popular tool collections such as [Bioconda](https://bioconda.github.io) and the [Python Package index](https://pypi.org/), while taking advantage of the configuration flexibility provided by Nextflow.
## Prerequisites
This feature requires the Conda or [Miniconda](https://conda.io/miniconda.html) package manager to be installed on your system.
## How it works
Nextflow automatically creates and activates the Conda environment(s) given the dependencies specified by each process.
Dependencies are specified by using the {ref}`process-conda` directive, providing either the names of the required Conda packages, the path of a Conda environment yaml file, or the path of an existing Conda environment directory.
:::{note}
Conda environments are stored on the file system. By default, Nextflow instructs Conda to save the required environments in the pipeline work directory. The same environment may be created/saved multiple times across multiple executions when using different work directories.
:::
You can specify the directory where the Conda environments are stored using the `conda.cacheDir` configuration property. When using a computing cluster, make sure to use a shared file system path accessible from all compute nodes. See the {ref}`configuration page <config-conda>` for details about Conda configuration.
:::{warning}
The Conda environment feature is not supported by executors that use remote object storage as a work directory. For example, AWS Batch.
:::
### Enabling Conda environment
:::{versionadded} 22.08.0-edge
:::
The use of Conda recipes specified using the {ref}`process-conda` directive needs to be enabled explicitly in the pipeline configuration file (i.e. `nextflow.config`):
```groovy
conda.enabled = true
```
Alternatively, it can be specified by setting the variable `NXF_CONDA_ENABLED=true` in your environment or by using the `-with-conda` command line option.
### Use Conda package names
Conda package names can be specified using the `conda` directive. Multiple package names can be specified by separating them with a blank space. For example:
```nextflow
process hello {
conda 'bwa samtools multiqc'
script:
"""
your_command --here
"""
}
```
Using the above definition, a Conda environment that includes BWA, Samtools and MultiQC tools is created and activated when the process is executed.
The usual Conda package syntax and naming conventions can be used. The version of a package can be specified after the package name as shown here `bwa=0.7.15`.
The name of the channel where a package is located can be specified prefixing the package with the channel name as shown here `bioconda::bwa=0.7.15`.
:::{versionchanged} 26.04.0
By default, Nextflow uses the `conda-forge` and `bioconda` channels to resolve Conda packages. You can override this using the `conda.channels` configuration option.
:::
(conda-env-files)=
### Use Conda environment files
Conda environments can also be defined using one or more Conda environment files. This is a file that lists the required packages and channels structured using the YAML format. For example:
```yaml
name: my-env
channels:
- conda-forge
- bioconda
dependencies:
- star=2.5.4a
- bwa=0.7.15
```
Read the Conda documentation for more details about how to create [environment files](https://conda.io/docs/user-guide/tasks/manage-environments.html#creating-an-environment-file-manually).
The path of an environment file can be specified using the `conda` directive:
```nextflow
process hello {
conda '/some/path/my-env.yaml'
script:
"""
your_command --here
"""
}
```
:::{warning}
The environment file name **must** have a `.yml` or `.yaml` extension or else it won't be properly recognized.
:::
(conda-pypi)=
### Python Packages from PyPI
Conda environment files can also be used to install Python packages from the [PyPI repository](https://pypi.org/), through the `pip` package manager (which must also be explicitly listed as a required package):
```yaml
name: my-env-2
channels:
- conda-forge
- bioconda
dependencies:
- pip
- pip:
- numpy
- pandas
- matplotlib
```
### Conda text files
It is possible to provide dependencies by listing each package name as a separate line in a plain text file. For example:
```
bioconda::star=2.5.4a
bioconda::bwa=0.7.15
bioconda::multiqc=1.4
```
:::{note}
Dependency files must be a text file with the `.txt` extension.
:::
### Conda lock files
The final method for providing packages to Conda is by using [Conda lock files](https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#identical-conda-envs).
To generate a lock file from an existing Conda environment, run the following command:
```bash
conda list --explicit > spec-file.txt
```
If you're using Mamba or Micromamba, use this command instead:
```bash
micromamba env export --explicit > spec-file.txt
```
You can also download Conda lock files from [Wave](https://seqera.io/wave/) container build pages.
These files list every package and its dependencies, so Conda doesn't need to perform dependency resolution. This makes environment setup faster and more reproducible.
Each file includes package URLs and, optionally, an MD5 hash for verifying file integrity:
```
# micromamba env export --explicit
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
@EXPLICIT
https://conda.anaconda.org/conda-forge/linux-64/_libgcc_mutex-0.1-conda_forge.tar.bz2#d7c89558ba9fa0495403155b64376d81
https://conda.anaconda.org/conda-forge/linux-64/libgomp-13.2.0-h77fa898_7.conda#abf3fec87c2563697defa759dec3d639
https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-2_gnu.tar.bz2#73aaf86a425cc6e73fcf236a5a46396d
https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-13.2.0-h77fa898_7.conda#72ec1b1b04c4d15d4204ece1ecea5978
# .. and so on
```
To use a Conda lock file with Nextflow, set the `conda` directive to the path of the lock file.
:::{note}
Conda lock files must be a text file with the `.txt` extension.
:::
### Use existing Conda environments
If you already have a local Conda environment, you can use it in your workflow specifying the installation directory of such environment by using the `conda` directive:
```nextflow
process hello {
conda '/path/to/an/existing/env/directory'
script:
"""
your_command --here
"""
}
```
### Use Mamba to resolve packages
:::{warning} *Experimental: may change in a future release.*
:::
It is also possible to use [mamba](https://github.com/mamba-org/mamba) to speed up the creation of conda environments. For more information on how to enable this feature please refer to {ref}`Conda <config-conda>`.
## Best practices
When a `conda` directive is used in any `process` definition within the workflow script, Conda tool is required for the workflow execution.
Specifying the Conda environments in a separate configuration {ref}`profile <config-profiles>` is therefore recommended to allow the execution via a command line option and to enhance the workflow portability. For example:
```groovy
profiles {
conda {
process.conda = 'samtools'
}
docker {
process.container = 'biocontainers/samtools'
docker.enabled = true
}
}
```
The above configuration snippet allows the execution either with Conda or Docker specifying `-profile conda` or `-profile docker` when running the workflow script.
## Advanced settings
Conda advanced configuration settings are described in the {ref}`Conda <config-conda>` section on the Nextflow configuration page.

424
nextflow/docs/conf.py Normal file
View File

@@ -0,0 +1,424 @@
# -*- coding: utf-8 -*-
#
# Nextflow documentation build configuration file, created by
# sphinx-quickstart on Sat May 5 16:57:28 2012.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import sys, os
import sphinx_rtd_theme
import datetime
import subprocess
import shlex
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
# -- General configuration -----------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.mathjax',
'sphinxcontrib.mermaid',
'sphinxext.rediraffe',
'sphinx_rtd_theme',
'myst_parser',
'sphinx_tabs.tabs',
'sphinx_copybutton'
]
myst_enable_extensions = ['colon_fence', 'deflist', 'dollarmath']
myst_heading_anchors = 3
rediraffe_redirects = {
'getstarted.md': 'install.md',
'basic.md': 'overview.md',
'tracing.md': 'reports.md',
'mail.md': 'notifications.md',
'operator.md': 'reference/operator.md',
'dsl1.md': 'migrations/dsl1.md',
'updating-syntax.md': 'strict-syntax.md',
'updating-spot-retries.md': 'guides/updating-spot-retries.md',
'metrics.md': 'tutorials/metrics.md',
'data-lineage.md' : 'tutorials/data-lineage.md',
'workflow-outputs.md': 'tutorials/workflow-outputs.md',
'flux.md': 'tutorials/flux.md',
'developer/plugins.md': 'plugins/developing-plugins.md',
'plugins.md': 'plugins/plugins.md',
'channel.md': 'workflow.md',
'module.md': 'modules/modules.md'
}
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.md'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Nextflow'
copyright = str(datetime.date.today().year) + u', Seqera Labs, S.L'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
# AUTO-GENERATED BY GIT
# Check if we are checked out at a specific version or not
process = subprocess.Popen(shlex.split("git tag --points-at HEAD"), stdout=subprocess.PIPE)
# The full version, including alpha/beta/rc tags.
release = process.communicate()[0].decode("utf-8").strip()
# The short X.Y version.
version = release.replace("-edge", "")
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
html_title = f"Nextflow {release} documentation"
# Get the current sha if not checked out at a specific version
if len(release) == 0:
process = subprocess.Popen(shlex.split("git rev-parse --short HEAD"), stdout=subprocess.PIPE)
current_sha = process.communicate()[0].decode("utf-8").strip()
release = f"dev <samp>({current_sha})</samp>"
html_title = "Nextflow documentation"
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = ['_build', '**README.md']
# The reST default role (used for this markup: `text`) to use for all documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'default'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# Add copyable selector
copybutton_selector = ".copyable pre"
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = "sphinx_rtd_theme"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
'logo_only': True,
'display_version': False
}
html_context = {
"display_github": True,
"github_user": "nextflow-io",
"github_repo": "nextflow",
"github_version": "master",
"conf_py_path": "/docs/"
}
# Nextflow theme
html_css_files = ['theme.css']
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
html_logo = '_static/nextflow-logo.png'
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
html_favicon = '_static/favicon.ico'
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
html_use_smartypants = False
# Custom sidebar templates, maps document names to template names.
html_sidebars = {
'**': ['globaltoc.html']
}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
html_show_sourcelink = False
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'Nextflowdoc'
# -- Options for LaTeX output --------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
('index', 'Nextflow.tex', u'Nextflow Documentation',
u'Paolo Di Tommaso', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
latex_logo = '_static/nextflow-logo.png'
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output --------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'nextflow', u'Nextflow Documentation',
[u'Paolo Di Tommaso'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output ------------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'Nextflow', u'Nextflow Documentation',
u'Paolo Di Tommaso', 'Nextflow', 'One line description of project.',
'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'
# -- Options for Epub output ---------------------------------------------------
# Bibliographic Dublin Core info.
epub_title = u'Nextflow'
epub_author = u'Paolo Di Tommaso'
epub_publisher = u'Paolo Di Tommaso'
epub_copyright = u'Copyright 2013-2024, Seqera Labs'
# The language of the text. It defaults to the language option
# or en if the language is not set.
#epub_language = ''
# The scheme of the identifier. Typical schemes are ISBN or URL.
#epub_scheme = ''
# The unique identifier of the text. This can be a ISBN number
# or the project homepage.
#epub_identifier = ''
# A unique identification for the text.
#epub_uid = ''
# A tuple containing the cover image and cover page html template filenames.
#epub_cover = ()
# HTML files that should be inserted before the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_pre_files = []
# HTML files that should be inserted after the pages created by sphinx.
# The format is a list of tuples containing the path and title.
#epub_post_files = []
# A list of files that should not be packed into the epub file.
#epub_exclude_files = []
# The depth of the table of contents in toc.ncx.
#epub_tocdepth = 3
# Allow duplicate toc entries.
#epub_tocdup = True
# see also: https://github.com/pygments/pygments/blob/master/pygments/lexers/jvm.py
from pygments.lexer import RegexLexer
from sphinx.highlighting import lexers
class NextflowLexer(RegexLexer):
"""
For Nextflow source code.
"""
import re
from pygments.lexer import bygroups, using, this, default
from pygments.token import Comment, Operator, Keyword, Name, String, Number, Whitespace
from pygments.util import shebang_matches
name = 'Nextflow'
url = 'https://nextflow.io/'
aliases = ['nextflow', 'nf']
filenames = ['*.nf']
mimetypes = ['text/x-nextflow']
# version_added = '1.5'
flags = re.MULTILINE | re.DOTALL
tokens = {
'root': [
# Nextflow allows a file to start with a shebang
(r'#!(.*?)$', Comment.Preproc, 'base'),
default('base'),
],
'base': [
(r'[^\S\n]+', Whitespace),
(r'(//.*?)(\n)', bygroups(Comment.Single, Whitespace)),
(r'/\*.*?\*/', Comment.Multiline),
# keywords: go before method names to avoid lexing "throw new XYZ"
# as a method signature
(r'(assert|catch|else|if|instanceof|new|return|throw|try|in|as)\b', Keyword),
(r'(channel|log)', Name.Namespace),
# method names
(r'^(\s*(?:[a-zA-Z_][\w.\[\]]*\s+)+?)' # return arguments
r'('
r'[a-zA-Z_]\w*' # method name
r'|"(?:\\\\|\\[^\\]|[^"\\])*"' # or double-quoted method name
r"|'(?:\\\\|\\[^\\]|[^'\\])*'" # or single-quoted method name
r')'
r'(\s*)(\()', # signature start
bygroups(using(this), Name.Function, Whitespace, Operator)),
(r'@[a-zA-Z_][\w.]*', Name.Decorator),
(r'(def|enum|include|from|output|params|process|workflow)\b', Keyword.Declaration),
(r'(boolean|byte|char|double|float|int|long|short|void)\b', Keyword.Type),
(r'(true|false|null)\b', Keyword.Constant),
(r'""".*?"""', String.Double),
(r"'''.*?'''", String.Single),
(r'"(\\\\|\\[^\\]|[^"\\])*"', String.Double),
(r"'(\\\\|\\[^\\]|[^'\\])*'", String.Single),
(r'/(\\\\|\\[^\\]|[^/\\])*/', String),
(r"'\\.'|'[^\\]'|'\\u[0-9a-fA-F]{4}'", String.Char),
(r'(\.)([a-zA-Z_]\w*)', bygroups(Operator, Name.Attribute)),
(r'[a-zA-Z_]\w*:', Name.Label),
(r'[a-zA-Z_$]\w*', Name),
(r'[~^*!%&\[\](){}<>|+=:;,./?-]', Operator),
(r'[0-9][0-9]*\.[0-9]+([eE][0-9]+)?[fd]?', Number.Float),
(r'0x[0-9a-fA-F]+', Number.Hex),
(r'[0-9]+L?', Number.Integer),
(r'\n', Whitespace)
],
}
def analyse_text(text):
return shebang_matches(text, r'nextflow')
lexers['nextflow'] = NextflowLexer()

347
nextflow/docs/config.md Normal file
View File

@@ -0,0 +1,347 @@
(config-page)=
# Configuration
## Configuration files
When you launch a pipeline script, Nextflow detects configuration files from multiple sources and applies them in the following order (from lowest to highest priority):
1. `$NXF_HOME/config` (defaults to `$HOME/.nextflow/config`)
2. `nextflow.config` in the project directory
3. `nextflow.config` in the launch directory
4. Config files specified with `-c <config-files>`
:::{tip}
You can use the `-C <config-file>` option to specify a fixed set of configuration files and ignore all other files.
:::
(config-syntax)=
## Syntax
Nextflow configuration uses a similar syntax as Nextflow scripts. Configuration options must be set declaratively, but the value of a config option can be an arbitrary expression.
A config file may contain any number of [assignments](#assignments), [blocks](#blocks), and [includes](#includes). You can add comments just like in scripts.
### Assignments
A config assignment sets a config option to a value:
```groovy
workDir = 'work'
docker.enabled = true
process.maxErrors = 10
```
The config option consists of an *option name* prefixed by any number of *scopes*. Scopes group related config options. See {ref}`config-options` for the full set of config options.
The value is typically a literal value, such as a number, boolean, or string. However, you can use any {ref}`expression <syntax-expressions>`:
```groovy
params.helper_file = "${projectDir}/assets/helper.txt"
```
### Blocks
You can specify config options in the same scope as a block:
```groovy
// dot syntax
docker.enabled = true
docker.runOptions = '-u $(id -u):$(id -g)'
// block syntax
docker {
enabled = true
runOptions = '-u $(id -u):$(id -g)'
}
```
As a result, deeply nested config options can be assigned in multiple ways. The following three assignments are equivalent:
```groovy
executor.retry.maxAttempt = 5
executor {
retry.maxAttempt = 5
}
executor {
retry {
maxAttempt = 5
}
}
```
### Includes
You can include configuration files in other configuration files with the `includeConfig` keyword:
```groovy
process.executor = 'sge'
process.queue = 'long'
process.memory = '10G'
includeConfig 'path/extra.config'
```
Relative paths are resolved against the location of the including file.
:::{note}
Config files should only be included at the top level or in a [profile](#config-profiles). This ensures the included config file is valid on its own and in the context in which it is included.
:::
## Constants and functions
The following constants are globally available in a Nextflow configuration file:
`baseDir: Path`
: :::{deprecated} 20.04.0
:::
: Alias for `projectDir`.
`launchDir: Path`
: The directory where the workflow was launched.
`projectDir: Path`
: The directory where the main script is located.
`secrets: Map<String,String>`
: Map of pipeline secrets. See {ref}`secrets-page` for more information.
The following functions are globally available in a Nextflow configuration file:
`env( name: String ) -> String`
: :::{versionadded} 25.04.0
:::
: Get the value of the environment variable with the specified name in the Nextflow launch environment.
(config-params)=
## Parameters
You can define pipeline parameters in config files using the `params` scope:
```groovy
// dot syntax
params.max_cpus = 64
params.publish_mode = 'copy'
// block syntax
params {
max_cpus = 64
publish_mode = 'copy'
}
```
:::{note}
When including a config file, the included config is evaluated with the parameters defined before the include. The included config cannot see parameters defined after the include.
:::
You should declare parameters in the config file only when other config options use them. When you use a parameter in the script, you should declare it there and override it in config profiles as needed:
```nextflow
// main.nf
params.input = null
```
```groovy
// nextflow.config
params {
publish_mode = 'copy'
}
workflow.output.mode = params.publish_mode
profiles {
test {
params.input = "${projectDir}/test/input.txt"
}
}
```
See {ref}`cli-params` for information about how Nextflow resolves pipeline parameters at runtime.
(config-process)=
## Process configuration
You can use the `process` scope to specify {ref}`process directives <process-reference>` separately from the pipeline code.
For example:
```groovy
process {
executor = 'sge'
queue = 'long'
clusterOptions = '-pe smp 10 -l virtual_free=64G,h_rt=30:00:00'
}
```
This configuration executes all processes using the SGE executor with the given settings.
(config-process-selectors)=
### Process selectors
You can use the `withLabel` selector to configure all processes annotated with a {ref}`process-label` directive:
```groovy
process {
withLabel: big_mem {
cpus = 16
memory = 64.GB
queue = 'long'
}
}
```
This configuration assigns 16 CPUs, 64 GB of memory, and the `long` queue to all processes with the `big_mem` label.
You can use the `withName` selector to configure a specific process by its name:
```groovy
process {
withName: hello {
cpus = 4
memory = 8.GB
queue = 'short'
}
}
```
The `withName` selector matches both:
- Processes defined with that name
- Processes included under that alias
When you include a process with an alias, selectors for the alias take priority over selectors for the original name. For example, if you define a process as `hello` and include it as `sayHello`, both `withName: hello` and `withName: sayHello` apply, with `withName: sayHello` taking priority.
:::{tip}
You don't need to enclose label and process names in quotes unless they contain special characters (`-`, `!`, etc.) or are keywords or built-in type identifiers.
:::
(config-selector-expressions)=
### Selector expressions
You can use regular expressions in process selectors to apply the same configuration to all processes matching the pattern:
```groovy
process {
withLabel: 'hello|bye' {
cpus = 2
memory = 4.GB
}
}
```
This configuration requests 2 CPUs and 4 GB of memory for processes labeled as `hello` or `bye`.
You can negate a selector expression by prefixing it with the special character `!`:
```groovy
process {
withLabel: 'hello' { cpus = 2 }
withLabel: '!hello' { cpus = 4 }
withName: '!align.*' { queue = 'long' }
}
```
This configuration sets 2 CPUs for processes labeled as `hello` and 4 CPUs for all processes *not* labeled as `hello`. It also specifies the `long` queue for processes whose name does *not* start with `align`.
(config-selector-priority)=
### Selector priority
Nextflow applies process configuration settings in the following order (from lowest to highest priority):
1. Process configuration settings (without a selector)
2. Process directives in the process definition
3. `withLabel` selectors matching any of the process labels
4. `withName` selectors matching the process name
5. `withName` selectors matching the process included alias
6. `withName` selectors matching the process fully qualified name
For example:
```groovy
process {
cpus = 4
withLabel: hello { cpus = 8 }
withName: bye { cpus = 16 }
withName: 'aloha:bye' { cpus = 32 }
}
```
This configuration:
- Sets 4 CPUs for all processes (unless otherwise specified in their process definition)
- Sets 8 CPUs for processes annotated with the `hello` label
- Sets 16 CPUs for any process named `bye` (or imported as `bye`)
- Sets 32 CPUs for any process named `bye` (or imported as `bye`) invoked by a workflow named `aloha`
(config-profiles)=
## Config profiles
Configuration files can define one or more *profiles*. A profile is a set of configuration settings that can be selected at runtime using the `-profile` command line option.
Configuration profiles are defined in the `profiles` scope. For example:
```groovy
profiles {
standard {
process.executor = 'local'
}
cluster {
process.executor = 'sge'
process.queue = 'long'
process.memory = '10GB'
}
cloud {
process.executor = 'cirrus'
process.container = 'cbcrg/imagex'
docker.enabled = true
}
}
```
This configuration defines three profiles: `standard`, `cluster`, and `cloud`. Each profile provides a different configuration for a given execution environment. When you do not specify a profile, Nextflow uses the `standard` profile by default.
You can enable configuration profiles at runtime as a comma-separated list:
```bash
nextflow run main.nf -profile standard,cloud
```
Nextflow applies config profiles in the order in which they were defined in the config, regardless of the order you specify them on the command line.
:::{versionadded} 25.04.0
When using the {ref}`strict syntax <strict-syntax-page>`, Nextflow applies profiles in the order you specify them on the command line.
:::
(config-workflow-handlers)=
## Workflow handlers
:::{deprecated} 25.10.0
Use a {ref}`trace observer <plugins-trace-observers>` in a plugin to add custom workflow handlers to a pipeline via configuration.
:::
You can define workflow event handlers in the config file:
```groovy
workflow.onComplete = {
println "Pipeline complete"
println "Command line: $workflow.commandLine"
}
workflow.onError = {
println "Error: something went wrong"
}
```
This approach is useful for handling workflow events without modifying the pipeline code. See {ref}`workflow-handlers` for more information.

754
nextflow/docs/container.md Normal file
View File

@@ -0,0 +1,754 @@
(container-page)=
# Containers
Nextflow supports a variety of container runtimes. Containerization allows you to write self-contained and truly reproducible computational pipelines, by packaging the binary dependencies of a script into a standard and portable format that can be executed on any platform that supports a container runtime. Furthermore, the same pipeline can be transparently executed with any of the supported container runtimes, depending on which runtimes are available in the target compute environment.
:::{note}
When creating a container image to use with Nextflow, make sure that Bash (3.x or later) and `ps` are installed in the image, along with other tools required for collecting metrics (See {ref}`this section <execution-report-tasks>`). Bash should be available on the path `/bin/bash` and it should be the container entrypoint.
:::
(container-apple)=
## Apple container
[Apple container](https://github.com/apple/container) is a lightweight container runtime that runs each container inside its own virtual machine on macOS, using the [Virtualization framework](https://developer.apple.com/documentation/virtualization).
### Prerequisites
You will need [Apple container](https://github.com/apple/container) installed on your macOS system. It requires Apple silicon (M1 or newer) and a recent macOS release.
Start the system service before running your pipeline:
```bash
container system start
```
### How it works
You won't need to modify your Nextflow script in order to run it with Apple container. Simply enable it in the Nextflow configuration file:
```groovy
process.container = 'nextflow/examples:latest'
appleContainer.enabled = true
```
Every time your script launches a process execution, Nextflow will run it inside a container created from the specified image. In practice Nextflow will automatically wrap your processes and run them by executing the `container run` command with the image you have provided.
:::{note}
Since each container runs in its own lightweight VM, tasks benefit from strong isolation without the overhead of a full VM per task. Images are standard OCI images and can be pulled from any registry.
:::
:::{note}
Apple container only supports Linux images. When running an `amd64` image on Apple silicon, pass `--rosetta` via `appleContainer.runOptions` and set the platform explicitly, e.g. `process.arch = 'linux/amd64'`.
:::
### Multiple containers
It is possible to specify a different image for each process definition in your pipeline script. Suppose you have two processes named `hello` and `bye`. You can specify two different images for them in the Nextflow script as shown below:
```nextflow
process hello {
container 'image_name_1'
script:
"""
do this
"""
}
process bye {
container 'image_name_2'
script:
"""
do that
"""
}
```
Alternatively, the same container definitions can be provided by using the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
appleContainer {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
### Advanced settings
Apple container advanced configuration settings are described in {ref}`config-apple-container` section in the Nextflow configuration page.
(container-apptainer)=
## Apptainer
:::{versionadded} 22.11.0-edge
:::
[Apptainer](https://apptainer.org) is an alternative container runtime to Docker and an open source fork of Singularity. The main advantages of Apptainer are that it can be used without root privileges and it doesn't require a separate daemon process. These, along with other features such as support for autofs mounts, makes Apptainer better suited to the requirements of HPC workloads. Apptainer is able to use existing Docker images and can pull from Docker registries.
### Prerequisites
You will need Apptainer installed on your execution environment e.g. your computer or a distributed cluster, depending on where you want to run your pipeline.
### Images
Apptainer makes use of a container image file, which physically contains the container. Refer to the [Apptainer documentation](https://apptainer.org/docs) to learn how to create Apptainer images.
Apptainer allows paths that do not currently exist within the container to be created and mounted dynamically by specifying them on the command line. However this feature is only supported on hosts that support the [Overlay file system](https://en.wikipedia.org/wiki/OverlayFS) and is not enabled by default.
:::{note}
Nextflow expects that data paths are defined system wide, and your Apptainer images need to be created having the mount paths defined in the container file system.
:::
If your Apptainer installation support the "user bind control" feature, enable the Nextflow support for that by defining the `apptainer.autoMounts = true` setting in the Nextflow configuration file.
### How it works
The integration for Apptainer follows the same execution model implemented for Docker. You won't need to modify your Nextflow script in order to run it with Apptainer. Simply specify the Apptainer image file from where the containers are started by using the `-with-apptainer` command line option. For example:
```bash
nextflow run <your script> -with-apptainer [apptainer image file]
```
Every time your script launches a process execution, Nextflow will run it into an Apptainer container created by using the specified image. In practice Nextflow will automatically wrap your processes and launch them by running the `apptainer exec` command with the image you have provided.
:::{note}
An Apptainer image can contain any tool or piece of software you may need to carry out a process execution. Moreover, the container is run in such a way that the process result files are created in the host file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.
:::
If you want to avoid entering the Apptainer image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the configuration file:
```groovy
process.container = '/path/to/apptainer.img'
apptainer.enabled = true
```
In the above example replace `/path/to/apptainer.img` with any Apptainer image of your choice.
Read the {ref}`config-page` page to learn more about the configuration file and how to use it to configure your pipeline execution.
:::{note}
Unlike Docker, Nextflow does not automatically mount host paths in the container when using Apptainer. It expects that the paths are configured and mounted system wide by the Apptainer runtime. If your Apptainer installation allows user defined bind points, read the {ref}`Apptainer configuration <config-apptainer>` section to learn how to enable Nextflow auto mounts.
:::
:::{warning}
When a process input is a *symbolic link* file, make sure the linked file is stored in a host folder that is accessible from a bind path defined in your Apptainer installation. Otherwise the process execution will fail because the launched container won't be able to access the linked file.
:::
:::{versionchanged} 23.07.0-edge
Nextflow no longer mounts the home directory when launching an Apptainer container. To re-enable the old behavior, set the environment variable `NXF_APPTAINER_HOME_MOUNT` to `true`.
:::
### Multiple containers
It is possible to specify a different Apptainer image for each process definition in your pipeline script. For example, suppose you have two processes named `hello` and `bye`. You can specify two different Apptainer images in the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
apptainer {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
### Apptainer & Docker Hub
Nextflow is able to transparently pull remote container images stored in any Docker compatible registry.
By default, when a container name is specified, Nextflow checks if an image file with that name exists in the local file system. If that image file exists, it's used to execute the container. If a matching file does not exist, Nextflow automatically tries to pull an image with the specified name from the container registry.
If you want Nextflow to check only for local file images, prefix the container name with the `file://` pseudo-protocol. For example:
```groovy
process.container = 'file:///path/to/apptainer.img'
apptainer.enabled = true
```
:::{warning}
Use three `/` slashes to specify an **absolute** file path, otherwise the path will be interpreted as relative to the workflow launch directory.
:::
To pull images from Apptainer Hub or a third party Docker registry, prefix the image name with the `shub://`, `docker://` or `docker-daemon://` pseudo-protocol as required by Apptainer. For example:
```groovy
process.container = 'docker://quay.io/biocontainers/multiqc:1.3--py35_2'
apptainer.enabled = true
```
You do not need to specify `docker://` to pull from a Docker repository. Nextflow will automatically prepend it to your image name when Apptainer is enabled. Additionally, the Docker engine will not work with containers specified as `docker://`.
:::{note}
This feature requires the `apptainer` tool to be installed where the workflow execution is launched (as opposed to the compute nodes).
:::
Nextflow caches Apptainer images in the `apptainer` directory, in the pipeline work directory, by default. However, it is recommended to provide a centralized cache directory using the `NXF_APPTAINER_CACHEDIR` environment variable or the `apptainer.cacheDir` setting in the Nextflow config file.
Nextflow uses the library directory to determine the location of Apptainer containers. The library directory can be defined using the `apptainer.libraryDir` configuration setting or the `NXF_APPTAINER_LIBRARYDIR` environment variable. The configuration file option overrides the environment variable if both are set.
Nextflow first checks the library directory when searching for the image. If the image is not found it then checks the cache directory. The main difference between the library directory and the cache directory is that the first is assumed to be a read-only container repository, while the latter is expected to be writable path where container images can be added for caching purposes.
:::{warning}
When using a compute cluster, the Apptainer cache directory must reside in a shared filesystem accessible to all compute nodes.
:::
:::{danger}
When pulling Docker images, Apptainer may be unable to determine the container size if the image was stored using an old Docker format, resulting in a pipeline execution error. See the Apptainer documentation for details.
:::
### Advanced settings
Apptainer advanced configuration settings are described in {ref}`config-apptainer` section in the Nextflow configuration page.
(container-charliecloud)=
## Charliecloud
:::{versionadded} 20.12.0-edge
:::
:::{versionchanged} 21.03.0-edge
Requires Charliecloud 0.22 to 0.27.
:::
:::{versionchanged} 22.09.0-edge
Requires Charliecloud 0.28 or later.
:::
:::{warning} *Experimental: not recommended for production environments.*
:::
[Charliecloud](https://hpc.github.io/charliecloud) is an alternative container runtime to Docker, that is better suited for use in HPC environments. Its main advantage is that it can be used without root privileges, making use of user namespaces in the Linux kernel. Charliecloud is able to pull from Docker registries.
### Prerequisites
You will need Charliecloud installed in your execution environment e.g. on your computer or a distributed cluster, depending on where you want to run your pipeline.
### How it works
You won't need to modify your Nextflow script in order to run it with Charliecloud. Simply specify the docker image from where the containers are started by using the `-with-charliecloud` command line option. For example:
```bash
nextflow run <your script> -with-charliecloud [container]
```
Every time your script launches a process execution, Nextflow will run it into a charliecloud container created by using the specified container image. In practice Nextflow will automatically wrap your processes and run them by executing the `ch-run` command with the container you have provided.
:::{note}
A container image can contain any tool or piece of software you may need to carry out a process execution. Moreover, the container is run in such a way that the process result files are created in the host file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.
:::
If you want to avoid entering the Container image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the configuration file:
```groovy
process.container = '/path/to/container'
charliecloud.enabled = true
```
:::{warning}
If an absolute path is provided, the container needs to be in the Charliecloud flat directory format. See the section below on compatibility with Docker registries.
:::
Read the {ref}`config-page` page to learn more about the configuration file and how to use it to configure your pipeline execution.
:::{warning}
Nextflow automatically manages the file system mounts whenever a container is launched depending on the process input files. However, when a process input is a *symbolic link*, the linked file **must** be stored in the same folder where the symlink is located, or a sub-folder of it. Otherwise the process execution will fail because the launched container won't be able to access the linked file.
:::
### Charliecloud & Docker Hub
Nextflow is able to transparently pull remote container images stored in any Docker compatible registry and converts them to the Charliecloud compatible format.
By default when a container name is specified, Nextflow checks if a container with that name exists in the local file system. If it exists, it's used to execute the container. If a matching file does not exist, Nextflow automatically tries to pull an image with the specified name from the Docker Hub.
To pull images from a third party Docker registry simply provide the URL to the image. If no URL is provided, Docker Hub is assumed. For example this can be used to pull an image from quay.io and convert it automatically to the Charliecloud container format:
```groovy
process.container = 'https://quay.io/biocontainers/multiqc:1.3--py35_2'
charliecloud.enabled = true
```
Whereas this would pull from Docker Hub:
```groovy
process.container = 'nextflow/examples:latest'
charliecloud.enabled = true
```
### Multiple containers
It is possible to specify a different Docker image for each process definition in your pipeline script. For example, suppose you have two processes named `hello` and `bye`. You can specify two different Docker images for them in the config file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
charliecloud {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
After running your pipeline, you can easily query the container image that each process used with the following command:
```bash
nextflow log last -f name,container
```
### Advanced settings
Charliecloud advanced configuration settings are described in {ref}`config-charliecloud` section in the Nextflow configuration page.
(container-docker)=
## Docker
[Docker](http://www.docker.io) is the industry standard container runtime.
### Prerequisites
You will need Docker installed on your execution environment e.g. your computer or a distributed cluster, depending on where you want to run your pipeline.
If you are running Docker on Mac OSX make sure you are mounting your local `/Users` directory into the Docker VM as explained in this excellent tutorial: [How to use Docker on OSX](http://viget.com/extend/how-to-use-docker-on-os-x-the-missing-guide).
### How it works
You won't need to modify your Nextflow script in order to run it with Docker. Simply specify the Docker image from where the containers are started by using the `-with-docker` command line option. For example:
```bash
nextflow run <your script> -with-docker [docker image]
```
Every time your script launches a process execution, Nextflow will run it into a Docker container created by using the specified image. In practice Nextflow will automatically wrap your processes and run them by executing the `docker run` command with the image you have provided.
:::{note}
A Docker image can contain any tool or piece of software you may need to carry out a process execution. Moreover, the container is run in such a way that the process result files are created in the host file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.
:::
If you want to avoid entering the Docker image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the configuration file:
```groovy
process.container = 'nextflow/examples:latest'
docker.enabled = true
```
In the above example replace `nextflow/examples:latest` with any Docker image of your choice.
Read the {ref}`config-page` page to learn more about the configuration file and how to use it to configure your pipeline execution.
:::{warning}
Nextflow automatically manages the file system mounts whenever a container is launched depending on the process input files. However, when a process input is a *symbolic link*, the linked file **must** be stored in the same folder where the symlink is located, or a sub-folder of it. Otherwise the process execution will fail because the launched container won't be able to access the linked file.
:::
### Multiple containers
It is possible to specify a different Docker image for each process definition in your pipeline script. Suppose you have two processes named `hello` and `bye`. You can specify two different Docker images for them in the Nextflow script as shown below:
```nextflow
process hello {
container 'image_name_1'
script:
"""
do this
"""
}
process bye {
container 'image_name_2'
script:
"""
do that
"""
}
```
Alternatively, the same containers definitions can be provided by using the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
docker {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
### Advanced settings
Docker advanced configuration settings are described in {ref}`config-docker` section in the Nextflow configuration page.
(container-podman)=
## Podman
:::{versionadded} 20.01.0
:::
[Podman](http://www.podman.io) is a drop-in replacement for Docker that can run containers with or without root privileges.
### Prerequisites
You will need Podman installed on your execution environment e.g. your computer or a distributed cluster, depending on where you want to run your pipeline. Running in rootless mode requires appropriate OS configuration. Due to current Podman limits using cpuset for cpus and memory such is only possible using sudo.
### How it works
You won't need to modify your Nextflow script in order to run it with Podman. Simply specify the Podman image from where the containers are started by using the `-with-podman` command line option. For example:
```bash
nextflow run <your script> -with-podman [OCI container image]
```
Every time your script launches a process execution, Nextflow will run it into a Podman container created by using the specified image. In practice Nextflow will automatically wrap your processes and run them by executing the `podman run` command with the image you have provided.
:::{note}
An OCI container image can contain any tool or piece of software you may need to carry out a process execution. Moreover, the container is run in such a way that the process result files are created in the host file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.
:::
If you want to avoid entering the Podman image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the configuration file:
```groovy
process.container = 'nextflow/examples:latest'
podman.enabled = true
```
In the above example replace `nextflow/examples:latest` with any Podman image of your choice.
Read the {ref}`config-page` page to learn more about the configuration file and how to use it to configure your pipeline execution.
:::{warning}
Nextflow automatically manages the file system mounts whenever a container is launched depending on the process input files. However, when a process input is a *symbolic link*, the linked file **must** be stored in the same folder where the symlink is located, or a sub-folder of it. Otherwise the process execution will fail because the launched container won't be able to access the linked file.
:::
### Multiple containers
It is possible to specify a different container image for each process definition in your pipeline script. Let's suppose you have two processes named `hello` and `bye`. You can specify two different container images for them in the Nextflow script as shown below:
```nextflow
process hello {
container 'image_name_1'
script:
"""
do this
"""
}
process bye {
container 'image_name_2'
script:
"""
do that
"""
}
```
Alternatively, the same containers definitions can be provided by using the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
podman {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
### Advanced settings
Podman advanced configuration settings are described in {ref}`config-podman` section in the Nextflow configuration page.
(container-sarus)=
## Sarus
:::{versionadded} 22.12.0-edge
Requires Sarus 1.5.1 or later.
:::
[Sarus](https://sarus.readthedocs.io) is an alternative container runtime to Docker. Sarus works by converting Docker images to a common format that can then be distributed and launched on HPC systems. The user interface to Sarus enables a user to select an image from [Docker Hub](https://hub.docker.com/) and then submit jobs which run entirely within the container.
### Prerequisites
You need Sarus installed in your execution environment, i.e. your personal computer or a distributed cluster, depending on where you want to run your pipeline.
### Images
Sarus converts a docker image to Squashfs layers which are distributed and launched in the cluster. For more information on how to build Sarus images see the [official documentation](https://sarus.readthedocs.io/en/stable/user/user_guide.html#develop-the-docker-image).
### How it works
The integration for Sarus, at this time, requires you to set up the following parameters in your config file:
```groovy
process.container = "dockerhub_user/image_name:image_tag"
sarus.enabled = true
```
and it will always try to search the Docker Hub registry for the images.
:::{note}
if you do not specify an image tag, the `latest` tag will be fetched by default.
:::
### Multiple containers
It is possible to specify a different Sarus image for each process definition in your pipeline script. For example, suppose you have two processes named `hello` and `bye`. You can specify two different Sarus images specifying them in the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
sarus {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
(container-shifter)=
## Shifter
:::{versionadded} 19.10.0
Requires Shifter 18.03 or later.
:::
[Shifter](https://docs.nersc.gov/programming/shifter/overview/) is an alternative container runtime to Docker. Shifter works by converting Docker images to a common format that can then be distributed and launched on HPC systems. The user interface to Shifter enables a user to select an image from [Docker Hub](https://hub.docker.com/) and then submit jobs which run entirely within the container.
### Prerequisites
You need Shifter and Shifter image gateway installed in your execution environment, i.e: your personal computer or the entry node of a distributed cluster. In the case of a distributed cluster, you should have Shifter installed on all of the compute nodes and the `shifterimg` command should also be available and Shifter properly setup to access the Image gateway, for more information see the [official documentation](https://github.com/NERSC/shifter/tree/master/doc).
### Images
Shifter converts a Docker image to squashfs layers which are distributed and launched in the cluster. For more information on how to build Shifter images see the [official documentation](https://docs.nersc.gov/programming/shifter/how-to-use/#building-shifter-images).
### How it works
The integration for Shifter requires you to set up the following parameters in your config file:
```groovy
process.container = "dockerhub_user/image_name:image_tag"
shifter.enabled = true
```
Shifter will search the Docker Hub registry for the images. If you do not specify an image tag, the `latest` tag will be fetched by default.
### Multiple containers
It is possible to specify a different Shifter image for each process definition in your pipeline script. For example, suppose you have two processes named `hello` and `bye`. You can specify two different Shifter images specifying them in the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
shifter {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
(container-singularity)=
## Singularity
[Singularity](http://singularity.lbl.gov/) is an alternative container runtime to Docker. The main advantages of Singularity are that it can be used without root privileges and it doesn't require a separate daemon process. These, along with other features such as support for autofs mounts, makes Singularity better suited to the requirements of HPC workloads. Singularity is able to use existing Docker images and can pull from Docker registries.
### Prerequisites
You will need Singularity installed on your execution environment e.g. your computer or a distributed cluster, depending on where you want to run your pipeline.
### Images
Singularity makes use of a container image file, which physically contains the container. Refer to the [Singularity documentation](https://www.sylabs.io/docs/) to learn how create Singularity images.
Singularity allows paths that do not currently exist within the container to be created and mounted dynamically by specifying them on the command line. However this feature is only supported on hosts that support the [Overlay file system](https://en.wikipedia.org/wiki/OverlayFS) and is not enabled by default.
:::{note}
Nextflow expects that data paths are defined system wide, and your Singularity images need to be created having the mount paths defined in the container file system.
:::
If your Singularity installation support the "user bind control" feature, enable the Nextflow support for that by defining the `singularity.autoMounts = true` setting in the Nextflow configuration file.
### How it works
The integration for Singularity follows the same execution model implemented for Docker. You won't need to modify your Nextflow script in order to run it with Singularity. Simply specify the Singularity image file from where the containers are started by using the `-with-singularity` command line option. For example:
```bash
nextflow run <your script> -with-singularity [singularity image file]
```
Every time your script launches a process execution, Nextflow will run it into a Singularity container created by using the specified image. In practice Nextflow will automatically wrap your processes and launch them by running the `singularity exec` command with the image you have provided.
:::{note}
A Singularity image can contain any tool or piece of software you may need to carry out a process execution. Moreover, the container is run in such a way that the process result files are created in the host file system, thus it behaves in a completely transparent manner without requiring extra steps or affecting the flow in your pipeline.
:::
If you want to avoid entering the Singularity image as a command line parameter, you can define it in the Nextflow configuration file. For example you can add the following lines in the configuration file:
```groovy
process.container = '/path/to/singularity.img'
singularity.enabled = true
```
In the above example replace `/path/to/singularity.img` with any Singularity image of your choice.
Read the {ref}`config-page` page to learn more about the configuration file and how to use it to configure your pipeline execution.
:::{note}
Unlike Docker, Nextflow does not automatically mount host paths in the container when using Singularity. It expects that the paths are configured and mounted system wide by the Singularity runtime. If your Singularity installation allows user defined bind points, read the {ref}`Singularity configuration <config-singularity>` section to learn how to enable Nextflow auto mounts.
:::
:::{warning}
When a process input is a *symbolic link* file, make sure the linked file is stored in a host folder that is accessible from a bind path defined in your Singularity installation. Otherwise the process execution will fail because the launched container won't be able to access the linked file.
:::
:::{versionchanged} 23.07.0-edge
Nextflow no longer mounts the home directory when launching a Singularity container. To re-enable the old behavior, set the environment variable `NXF_SINGULARITY_HOME_MOUNT` to `true`.
:::
:::{versionchanged} 23.09.0-edge
Nextflow automatically mounts the required host paths in the container. To re-enable the old behavior, set the environment variable `NXF_SINGULARITY_AUTO_MOUNTS` to `false` or set `singularity.autoMounts=false` in the Nextflow configuration file.
:::
:::{versionchanged} 23.09.0-edge
The execution command for Singularity/Apptainer containers can be set to `run` by means of the environment variable `NXF_SINGULARITY_RUN_COMMAND` (default command is `exec`).
:::
### Multiple containers
It is possible to specify a different Singularity image for each process definition in your pipeline script. For example, suppose you have two processes named `hello` and `bye`. You can specify two different Singularity images specifying them in the configuration file as shown below:
```groovy
process {
withName:hello {
container = 'image_name_1'
}
withName:bye {
container = 'image_name_2'
}
}
singularity {
enabled = true
}
```
Read the {ref}`Process scope <config-process>` section to learn more about processes configuration.
### Singularity & Docker Hub
*Requires Singularity 2.3 or later*
Nextflow is able to transparently pull remote container images stored in the [Singularity-Hub](https://singularity-hub.org/), [Singularity Library](https://cloud.sylabs.io/library/), or any Docker compatible registry.
By default when a container name is specified, Nextflow checks if an image file with that name exists in the local file system. If that image file exists, it's used to execute the container. If a matching file does not exist, Nextflow automatically tries to pull an image with the specified name from the Docker Hub.
If you want Nextflow to check only for local file images, prefix the container name with the `file://` pseudo-protocol. For example:
```groovy
process.container = 'file:///path/to/singularity.img'
singularity.enabled = true
```
:::{warning}
Use three `/` slashes to specify an **absolute** file path, otherwise the path will be interpreted as relative to the workflow launch directory.
:::
To pull images from the Singularity Hub or a third party Docker registry simply prefix the image name with the `shub://`, `docker://` or `docker-daemon://` pseudo-protocol as required by the Singularity tool. For example:
```groovy
process.container = 'docker://quay.io/biocontainers/multiqc:1.3--py35_2'
singularity.enabled = true
```
You do not need to specify `docker://` to pull from a Docker repository. Nextflow will automatically prepend it to your image name when Singularity is enabled. Additionally, the Docker engine will not work with containers specified as `docker://`.
:::{versionadded} 19.04.0
Requires Singularity 3.0.3 or later.
:::
Nextflow supports the [Singularity Library](https://cloud.sylabs.io/library/) repository:
```groovy
process.container = 'library://library/default/alpine:3.8'
```
The `library://` pseudo-protocol allows you to import Singularity images from a local Docker installation instead of downloading them from a Docker registry. This feature requires the `singularity` tool to be installed where the workflow execution is launched (as opposed to the compute nodes).
Nextflow caches Singularity images in the `singularity` directory, in the pipeline work directory, by default. However, it is recommended to provide a centralized cache directory using the `NXF_SINGULARITY_CACHEDIR` environment variable or the `singularity.cacheDir` setting in the Nextflow config file.
Nextflow uses the library directory to determine the location of Singularity images. The library directory can be defined using the `singularity.libraryDir` configuration setting or the `NXF_SINGULARITY_LIBRARYDIR` environment variable. The configuration file option overrides the environment variable if both are set.
Nextflow first checks the library directory when searching for the image. If the image is not found it then checks the cache directory. The main difference between the library directory and the cache directory is that the first is assumed to be a read-only container repository, while the latter is expected to be writable path where container images can be added for caching purposes.
:::{warning}
When using a compute cluster, the Singularity cache directory must reside in a shared filesystem accessible to all compute nodes.
:::
:::{danger}
When pulling Docker images, Singularity may be unable to determine the container size if the image was stored using an old Docker format, resulting in a pipeline execution error. See the Singularity documentation for details.
:::
### Advanced settings
Singularity advanced configuration settings are described in {ref}`config-singularity` section in the Nextflow configuration page.

323
nextflow/docs/developer-env.md Normal file
View File

@@ -0,0 +1,323 @@
(devenv-page)=
# Environment setup
Setting up a Nextflow development environment is a prerequisite for creating, testing, and optimizing data analysis pipelines.
<h3>Recommended tools</h3>
- {ref}`devenv-vscode`: A versatile code editor that enhances your Nextflow development with features like syntax highlighting and debugging.
- {ref}`devenv-extensions`: The VS Code marketplace offers a variety of extensions to enhance development. The {ref}`Nextflow extension <devenv-nextflow>` is specifically designed to enhance Nextflow development with diagnostics, hover hints, code navigation, code completion, and more.
- {ref}`devenv-docker`: A containerization platform that ensures your Nextflow workflows run consistently across different environments by packaging dependencies into isolated containers.
- {ref}`devenv-git`: A version control system that helps manage and track changes in your Nextflow projects, making collaboration and code management more efficient.
The sections below outline the steps for setting up these tools.
:::{note}
Nextflow must be installed separately. See {ref}`install-page` for Nextflow installation instructions.
:::
:::{note}
If you are using a Windows computer, first install and configure the Windows Subsystem for Linux (WSL). See {ref}`devenv-wsl` for installation instructions.
:::
(devenv-vscode)=
## VS Code
An Integrated Development Environment (IDE) provides a user-friendly interface for writing, editing, and managing code. Installing one is an essential step for setting up your environment.
Visual Studio Code (VS Code) is a popular lightweight IDE known for its versatility and extensibility. It offers features like syntax highlighting, intelligent code completion, and integrated debugging tools for various programming languages. VS Code supports Windows, macOS, and Linux, and is a good choice for both new and experienced Nextflow developers.
````{tabs}
```{group-tab} Windows
To install VS Code on Windows:
1. Visit the [VS Code](https://code.visualstudio.com/download) website.
1. Download VS Code for Windows.
1. Double-click the installer executable (`.exe`) file and follow the setup steps.
```
```{group-tab} macOS
To install VS Code on macOS:
1. Visit the [VS Code](https://code.visualstudio.com/download) website.
1. Download VS Code for macOS.
1. Drag the `Visual Studio Code.app` application to the Applications folder to add it to the macOS Launchpad.
```
```{group-tab} Linux
To install VS Code on Linux Debian/Ubuntu distributions:
1. Visit the [VS Code](https://code.visualstudio.com/download) website.
1. Download the VS Code Linux Debian/Ubuntu (`.deb`) distribution.
1. Open a new terminal window.
1. Navigate to the folder where you downloaded VS Code.
1. Run `sudo apt install ./<file>.deb`, replacing `<file>` with the full file name.
:::{note}
If you're using an older Linux distribution, run `sudo dpkg -i <file>.deb` to install VS Code and `sudo apt-get install -f` to install dependencies.
:::
See [Linux installation](https://code.visualstudio.com/docs/setup/linux#_installation) for information about installing VS Code on other distributions.
```
````
(devenv-extensions)=
## Extensions
Extensions are a key feature of IDEs and allow you to customize your development environment by adding support for various programming languages, tools, and features. The [VS Code Marketplace](https://marketplace.visualstudio.com/vscode) offers thousands of extensions that can enhance your productivity and tailor the editor to your specific needs. Popular VS Code extensions for Nextflow developers are listed below:
(devenv-nextflow)=
**Nextflow**
The VS Code [Nextflow extension](https://marketplace.visualstudio.com/items?itemName=nextflow.nextflow) adds Nextflow language support to the editor. The Nextflow extension enhances development with:
- Diagnostics
- Hover hints
- Code navigation
- Code completion
- Formatting
- Renaming
- Parameter schemas
- DAG previews
See {ref}`vscode-page` for more information about the Nextflow extension features and how it enforces the Nextflow syntax.
**nf-core**
[nf-core](https://nf-co.re/) is a community effort to collect a curated set of analysis pipelines built using Nextflow. The [nf-core extension pack](https://marketplace.visualstudio.com/items?itemName=nf-core.nf-core-extensionpack) adds a selection of tools that support development. For example, it includes [Code Spell Checker](https://marketplace.visualstudio.com/items?itemName=streetsidesoftware.code-spell-checker), [Prettier](https://marketplace.visualstudio.com/items?itemName=esbenp.prettier-vscode), [Todo Tree](https://marketplace.visualstudio.com/items?itemName=Gruntfuggly.todo-tree), and [Markdown Extended](https://marketplace.visualstudio.com/items?itemName=jebbs.markdown-extended).
See the [nf-core extension pack](https://marketplace.visualstudio.com/items?itemName=nf-core.nf-core-extensionpack) for more information about the included tools.
(devenv-remote)=
**Remote development**
The [Remote Development extension pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) enables you to run WSL, SSH, or a development container for editing and debugging with the full set of VS Code features.
The pack includes the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh), [Remote - Tunnels](https://marketplace.visualstudio.com/items?itemName=ms-vscode.remote-server), [Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers), and [WSL](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-wsl) extensions. See [Remote Development](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack) for more information about the tools included in the remote development extension pack.
:::{note}
The Remote Development extension pack is required if you are developing using remote servers, Windows Subsystem for Linux, or Development Containers.
:::
Installing VS Code extensions requires just a few clicks in the Extensions Marketplace.
````{tabs}
```{group-tab} Windows
To install a VS Code extension on Windows:
1. Open VS Code.
1. Open the **Extensions** view in the left-hand menu.
1. Search for the extension.
1. Select **Install**.
```
```{group-tab} macOS
To install a VS Code extension on macOS:
1. Open VS Code.
1. Open the **Extensions** view in the left-hand menu.
1. Search for the extension.
1. Select **Install**.
```
```{group-tab} Linux
To install a VS Code extension on Linux Debian/Ubuntu distributions:
1. Open VS Code.
1. Open the **Extensions** view in the left-hand menu.
1. Search for the extension.
1. Select **Install**.
```
````
(devenv-docker)=
## Docker
Docker is an open-source platform that simplifies application development, deployment, and execution by packaging applications and their dependencies into containers. Containerization enables the creation of self-contained and fully reproducible computational pipelines by bundling a script's binary dependencies into a standardized and portable format. Containers can be executed on any platform that supports a container runtime and ensures consistency across different environments.
Docker Desktop provides a Graphical User Interface (GUI) for managing Docker containers. Installing Docker Desktop is a straightforward process that allows you to create, deploy, and manage applications within containers.
````{tabs}
```{group-tab} Windows
To install Docker Desktop on Windows:
1. Go to [Install Docker Desktop on Windows](https://docs.docker.com/desktop/setup/install/windows-install/).
1. Download the installer.
1. Double-click Docker Desktop `Installer.exe` to run the installer. By default, Docker Desktop is installed at `C:\Program Files\Docker\Docker`.
1. Depending on your choice of backend, select the **Use WSL 2 instead of Hyper-V** option on the Configuration page.
:::{note}
You won't be able to select which backend to use if your system only supports one of the two options.
:::
1. Follow the instructions on the installation wizard to authorize the installer and proceed with the install.
1. When the installation is complete, select **Close**.
1. Start Docker Desktop.
1. Review the Docker Subscription Service Agreement and select **Accept** to continue.
:::{note}
Docker Desktop won't run if you do not agree to the terms. You can choose to accept the terms at a later date by opening Docker Desktop.
:::
1. Docker Desktop starts after you accept the terms.
```
```{group-tab} macOS
To install Docker Desktop on macOS:
1. Go to [Install Docker Desktop on Mac](https://docs.docker.com/desktop/install/mac-install/).
1. Download the installer for your chip type.
1. Double-click `Docker.dmg` to open the installer.
1. Drag the Docker icon to the **Applications** folder to add it to the macOS Launchpad.
1. Double-click **Docker.app** in the **Applications** folder to start Docker.
1. Review the Docker Subscription Service Agreement and select **Accept** to continue.
:::{note}
Docker Desktop won't run if you do not agree to the terms. You can choose to accept the terms at a later date by opening Docker Desktop.
:::
1. From the installation window, select **Use recommended settings (Requires password)**.
:::{note} The **recommended settings** allow Docker Desktop to automatically set the necessary configuration settings. Advanced settings allow you to set the location of the Docker CLI tools either in the system or user directory, enable the default Docker socket, and enable privileged port mapping. See [Settings](https://docs.docker.com/desktop/settings/#advanced), for more information and how to set the location of the Docker CLI tools.
:::
1. Select **Finish**. If you have applied any of the previous configurations that require a password, enter your password to confirm your choice.
```
```{group-tab} Linux
To install Docker Desktop on Linux Debian/Ubuntu distributions:
1. Set up Docker's package repository. See step one of [Install using the `apt` repository](https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository).
1. Download the latest Debian/Ubuntu (`.deb`) distribution.
1. In your terminal, run `sudo apt-get install ./docker-desktop-amd64.deb`
:::{note}
By default, Docker Desktop is installed at `/opt/docker-desktop`.
:::
1. Double-click **Docker Desktop** in your Applications menu to start Docker.
1. Review the Docker Subscription Service Agreement and select **Accept** to continue.
1. From the installation window, select **Use recommended settings (Requires password)**. Docker Desktop starts after you accept the terms.
:::{note}
Docker Desktop won't run if you do not agree to the terms. You can choose to accept the terms at a later date by opening Docker Desktop.
:::
```
````
Nextflow supports multiple container technologies (e.g., Singularity and Podman) so you can choose the one that best fits your needs. See {ref}`container-page` for more information about other supported container engines.
(devenv-git)=
## Git
Git provides powerful version control that helps track code changes. Git operates locally, meaning you don't need an internet connection to track changes, but it can also be used with remote platforms like GitHub, GitLab, or Bitbucket for collaborative development.
Nextflow seamlessly integrates with Git for source code management providers to manage pipelines as version-controlled Git repositories.
````{tabs}
```{group-tab} Windows
Git is already installed on most WSL distributions. You can check if it is already installed by running `git version`.
To install the latest stable Git version on Linux Debian/Ubuntu distributions:
1. Open a terminal window and run `sudo apt-get install git-all`.
1. Once complete, run `git version` to verify Git was installed.
See [git-scm documentation](https://git-scm.com/downloads/linux) for more information about installing Git on other Linux distributions.
```
```{group-tab} macOS
To install Git on macOS with [Homebrew](https://docs.brew.sh/):
1. Open a terminal window and run `brew install git`.
:::{note}
You must have Homebrew installed. See [Homebrew installation](https://docs.brew.sh/Installation) for instructions.
:::
1. Once complete, run `git version` to verify Git was installed.
To install Git on macOS with [Xcode](https://developer.apple.com/xcode/):
1. Open the App Store on your Mac.
1. Search for Xcode.
1. Select **Install**.
1. Once complete, open a new terminal window and run `git version` to verify Git was installed.
```
```{group-tab} Linux
Git is already installed on most Linux Debian/Ubuntu distributions.
To install the latest stable Git version on Linux Debian/Ubuntu distributions:
1. Open a terminal window and run `sudo apt-get install git-all`.
1. Once complete, run `git version` to verify Git was installed.
See [git-scm documentation](https://git-scm.com/downloads/linux) for more information about installing Git on other Linux distributions.
```
````
(devenv-wsl)=
## Windows Subsystem for Linux
Developers can access the power of both Windows and Linux on a Windows machine. The Windows Subsystem for Linux (WSL) lets developers install a Linux distribution and use Linux applications, utilities, and Bash command-line tools directly on Windows without the overhead of a virtual machine or dual-boot setup.
WSL is an optional feature on Windows 10 version 2004 and higher (Build 19041 and higher) or Windows 11. You can enable it through PowerShell or Windows Command Prompt. The steps below outline the recommended setup.
To enable WSL on Windows using Powershell or Windows Command Prompt:
1. Right-click and select **Run as administrator** to use PowerShell or Windows Command Prompt in administrator mode.
1. Run `wsl --install`.
:::{note}
This command will enable the features necessary to run WSL and install the Ubuntu distribution.
:::
1. When prompted, restart Windows.
1. After restarting Windows, open the Ubuntu distribution and create a new Linux **User Name** and **Password** when prompted.
:::{note}
The **User Name** and **Password** is specific to each Linux distribution that you install and has no bearing on your Windows user name.
:::
See [Set up a WSL development environment](https://learn.microsoft.com/en-us/windows/wsl/setup/environment) for more about installing WSL.

176
nextflow/docs/developer/config-scopes.md Normal file
View File

@@ -0,0 +1,176 @@
(config-scopes-page)=
# Configuration scopes
This page provides guidance on defining configuration scopes in the Nextflow runtime.
## Overview
The Nextflow configuration is defined as a collection of *scope classes*. Each scope class defines the set of available options, including their name, type, and an optional description for a specific configuration scope.
Scope classes are used to generate a configuration spec, which is in turn used for several purposes:
- Validating config options at runtime (`nextflow run` and `nextflow config`)
- Providing code intelligence in the language server (validation, hover hints, code completion)
- Generating reference documentation (in progress)
Scope classes are also used by the runtime itself as type-safe domain objects. This way, the construciton of domain objects from the configuration map is isolated from the rest of the runtime.
## Definition
### Config scopes
A *config scope* is defined as a class that implements the `ConfigScope` interface. Top-level scope classes must have the `@ScopeName` annotation, which defines the name of the config scope.
For example:
```groovy
package nextflow.hello
import nextflow.config.spec.ConfigScope
import nextflow.config.spec.ScopeName
@ScopeName('hello')
class HelloConfig implements ConfigScope {
}
```
A scope class must provide a default constructor, so that it can be instantiated as an extension point. If no such constructor is defined, the config scope will not be detected by Nextflow. In the above example, this constructor is implicitly defined because no constructors were declared.
The fully-qualified class name (in this case, `nextflow.hello.HelloConfig`) must be included in the list of extension points.
### Config options
A *config option* is defined as a field with the `@ConfigOption` annotation. The field name determines the name of the config option.
For example:
```groovy
@ConfigOption
String createMessage
```
The `@ConfigOption` annotation can specify an optional set of types that are valid in addition to the field type. For example, the `fusion.tags` option, which accepts either a String or Boolean, is declared as follows:
```groovy
@ConfigOption(types=[Boolean])
String tags
```
The field type and any additional types are included in the config spec, allowing them to be used for validation.
The field type can be any Java or Groovy class, but in practice it should be a class that can be constructed from primitive values (numbers, booleans, strings). For example, `Duration` and `MemoryUnit` are standard Nextflow types that can each be constructed from an integer or string.
### Nested scopes
A *nested scope* is defined as a field whose type is an implementation of `ConfigScope`. The field name determines the name of the nested scope.
The scope class referenced by the field type defines config options and scopes in the same manner as top-level scope classes. Unlike top-level scopes, nested scope classes do not need to use the `@ScopeName` annotation or provide a default constructor.
See `ExecutorConfig` and `ExecutorRetryConfig` for an example of how a nested scope is defined and constructed.
### Placeholder scopes
A *placeholder scope* is a config scope that applies to a collection of user-defined names.
For example, the `azure.batch.pools` scope allows the user to define a set of named pools, where each pool is configured with a standard set of options such as `autoScale`, `lowPriority`, `maxVmCount`, etc. These options are defined in a placeholder scope with a placeholder name of `<name>`. Thus, the generic name for the `autoScale` option is `azure.batch.pools.<name>.autoScale`.
A placeholder scope is defined as a field with type `Map<String, P>`, where `P` is a nested scope class which defines the scope options. The field should have the `@PlaceholderName` annotation which defines the placeholder name (e.g. `<name>`).
See `AzBatchOpts` and `AzPoolOpts` for an example of how placeholder scopes are defined and constructed.
### Descriptions
Top-level scope classes and config options should use the `@Description` annotation to provide a description of the scope or option. This description is included in the config spec, which is in turn used by the language server to provide hover hints.
For example:
```groovy
@ScopeName('hello')
@Description('''
The `hello` scope controls the behavior of the `nf-hello` plugin.
''')
class HelloConfig implements ConfigScope {
@ConfigOption
@Description('''
Message to print to standard output when a run is initialized.
''')
String createMessage
}
```
Nested scopes and placeholder scopes may also use this annotation, but will inherit the description of the top-level scope by default.
### Best practices
The Nextflow runtime adheres the following best practices where appropriate:
- Config options should be declared as public and final, so that the scope class can be used as an immutable domain object.
- Scope classes should define a constructor that initializes each field from a map, casting each map property to the required type and providing default values as needed.
- In cases where an option defaults to an environment variable, the environment map should be provided as an additional constructor argument rather than accessing the system environment directly.
- In cases where an option with a primitive type (e.g., `int`, `float`, `boolean`) can be unspecified without a default value, it should be declared with the equivalent reference type (e.g. `Integer`, `Float`, `Boolean`), otherwise it should use the primitive type.
- In cases where an option represents a path, it should be declared as a `String` and allow clients to construct paths as needed, since path construction may depend on plugins which aren't yet loaded.
For example:
```groovy
import nextflow.config.spec.ConfigOption
import nextflow.config.spec.ConfigScope
import nextflow.config.spec.ScopeName
@ScopeName('hello')
class HelloConfig implements ConfigScope {
@ConfigOption
final String createMessage
@ConfigOption
final boolean verbose
HelloConfig() {}
HelloConfig(Map opts, Map env) {
this.createMessage = opts.createMessage ?: env.get('NXF_HELLO_CREATE_MESSAGE')
this.verbose = opts.verbose as boolean
}
}
```
## Usage
### Runtime
Nextflow validates the config map after it is loaded. Top-level config scopes are loaded by the plugin system as extension points and converted into a config spec, which is used to validate the config map.
Plugins are loaded after the config is loaded and before it is validated, since plugins can also define config scopes. If a third-party plugin declares a config scope, it must be explicitly enabled in order to validate config options from the plugin. Otherwise, Nextflow will report these options as unrecognized.
Core plugins are loaded automatically based on other config options. Therefore, Nextflow only validates config from a core plugin when that plugin is loaded. Otherwise, any config options from the plugin are ignored -- they are neither validated nor reported as unrecognized.
For example, when the `process.executor` config option is set to `'awsbatch'`, the `nf-amazon` is automatically loaded. In this case, all options in the `aws` config scope will be validated. If the executor is not set to `'awsbatch'`, all `aws` options will be ignored. This way, config files can be validated appropriately without loading additional core plugins that won't be used by the run.
The scope classes themselves can be used to construct domain objects on-demand from the config map. For example, an `ExecutorConfig` can be constructed from the `executor` config scope as follows:
```groovy
new ExecutorConfig( Global.session.config.executor as Map ?: Collections.emptyMap() )
```
:::{note}
In practice, it is better to avoid the use of `Global` and provide an instance of `Session` to the client class instead.
:::
### Config spec
Config scope classes can be converted into a config spec with the `SpecNode` class, which uses reflection to extract metadata such as scope names, option names, types, and descriptions. This spec is rendered to JSON and used by the language server at build-time to provide code intelligence such as code completion and hover hints.
### Documentation
The config spec described above can be rendered to Markdown using the `MarkdownRenderer` class. It produces a Markdown document approximating the {ref}`config-options` page.
This approach to docs generation is not yet complete, and has not been incorporated into the build process yet. However, it can be used to check for discrepancies between the source code and docs when making changes. The documentation should match the `@Description` annotations as closely as possible, but may contain additional details such as version notes and extra paragraphs.

10
nextflow/docs/developer/diagram.md Normal file
View File

@@ -0,0 +1,10 @@
(diagram-page)=
# Workflow Diagram
The following diagram is a high-level overview of the Nextflow source code in a similar style as the {ref}`workflow diagram <workflow-diagram>` visualization for Nextflow pipelines. Each node and subgraph is a class. Arrows depict the flow of data and/or communication between classes.
In general, nodes with sharp corners are "record" classes that simply hold information, while nodes with rounded edges are "function" classes that transform some input into an output. Subgraphs are either long-running classes, i.e. "places where things happen", or one of the other two types for which it was useful to expand and show internal details.
```{mermaid} diagrams/overview.mmd
```

1
nextflow/docs/developer/diagrams/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
nextflow-merged.mmd

17
nextflow/docs/developer/diagrams/README.md Normal file
View File

@@ -0,0 +1,17 @@
# Class Diagrams
This directory contains class diagrams of the Nextflow source code, abridged and annotated for relevance and ease of use.
Each node is a class. Fields are selectively documented in order to show only core data structures and the classes that "own" them. Methods are not explicitly documented, but they are mentioned in certain links where appropriate.
Links between classes denote one of the following relationships:
- Inheritance (`A <|-- B`): `B` is a subclass of `A`
- Composition (`A --* B`): `A` contains `B`
- Instantiation (`A --> B : f`): `A` creates instance(s) of `B` at runtime via `A::f()`
Some links are commented out or not included at all, in order to focus on the most important classes and relationships. You can view these "hidden" links by simply uncommenting them, but I have found that their significance is sufficiently clear from the description files.
A separate diagram description is provided for each package. These files are interoperable, which means that you can combine any subset of files into a larger diagram description. The `merge-diagrams.sh` can create a merged file for you automatically, and it includes a sensible default set of packages.
You can use the [Mermaid Live Editor](https://mermaid-js.github.io/mermaid-live-editor/edit) or the [Mermaid CLI](https://github.com/mermaid-js/mermaid-cli) to render the diagram in a variety of image formats.

35
nextflow/docs/developer/diagrams/merge-diagrams.sh Executable file
View File

@@ -0,0 +1,35 @@
#!/bin/bash
packages=()
packages+=("nextflow")
# packages+=("nextflow.ast")
packages+=("nextflow.cache")
packages+=("nextflow.cli")
# packages+=("nextflow.cloud.aws")
# packages+=("nextflow.cloud.aws.nio")
# packages+=("nextflow.cloud.azure")
# packages+=("nextflow.cloud.google")
packages+=("nextflow.config")
# packages+=("nextflow.container")
packages+=("nextflow.dag")
# packages+=("nextflow.executor")
# packages+=("nextflow.extension")
# packages+=("nextflow.ga4gh")
# packages+=("nextflow.k8s")
# packages+=("nextflow.plugin")
packages+=("nextflow.processor")
# packages+=("nextflow.scm")
packages+=("nextflow.script")
# packages+=("nextflow.secret")
# packages+=("nextflow.trace")
outfile="nextflow-merged.mmd"
echo "classDiagram" > ${outfile}
for package in "${packages[@]}"; do
echo "${package}"
tail -n +2 "${package}.mmd" >> ${outfile}
echo >> ${outfile}
done

7
nextflow/docs/developer/diagrams/nextflow.ast.mmd Normal file
View File

@@ -0,0 +1,7 @@
classDiagram
%%
%% nextflow.ast
%%
ScriptParser --> NextflowDSLImpl : parse
ScriptParser --> NextflowXformImpl : parse
ScriptParser --> OpXformImpl : parse

22
nextflow/docs/developer/diagrams/nextflow.cache.mmd Normal file
View File

@@ -0,0 +1,22 @@
classDiagram
%%
%% nextflow.cache
%%
Session --* CacheDB
CacheDB --* CacheStore
CacheStore <|-- DefaultCacheStore
CacheStore <|-- CloudCacheStore
class DefaultCacheStore {
uniqueId : UUID
runName : String
baseDir : Path
}
class CloudCacheStore {
uniqueId : UUID
runName : String
basePath : Path
}

29
nextflow/docs/developer/diagrams/nextflow.cli.mmd Normal file
View File

@@ -0,0 +1,29 @@
classDiagram
%%
%% nextflow.cli
%%
class Launcher {
cliOptions : CliOptions
command : CmdBase
}
Launcher --* CliOptions
Launcher --* CmdBase
%% CmdBase <|-- CmdClean
%% CmdBase <|-- CmdClone
%% CmdBase <|-- CmdConfig
CmdBase <|-- CmdConsole
%% CmdBase <|-- CmdDrop
%% CmdBase <|-- CmdFs
CmdBase <|-- CmdHelp
CmdBase <|-- CmdInfo
%% CmdBase <|-- CmdKubeRun
%% CmdBase <|-- CmdList
%% CmdBase <|-- CmdLog
%% CmdBase <|-- CmdNode
%% CmdBase <|-- CmdPlugin
%% CmdBase <|-- CmdPull
CmdBase <|-- CmdRun
%% CmdBase <|-- CmdSecret
%% CmdBase <|-- CmdSelfUpdate
%% CmdBase <|-- CmdView

21
nextflow/docs/developer/diagrams/nextflow.cloud.aws.mmd Normal file
View File

@@ -0,0 +1,21 @@
classDiagram
%%
%% nextflow.cloud.aws
%%
Executor <|-- AwsBatchExecutor
TaskHandler <|-- AwsBatchTaskHandler
BashWrapperBuilder <|-- AwsBatchScriptLauncher
AwsBatchExecutor --* AwsOptions
AwsOptions --* AwsConfig
AwsConfig --* AwsBatchConfig
AwsConfig --* AwsS3Config
AwsBatchExecutor --> ParallelPollingMonitor : init
AwsBatchExecutor --> AwsBatchTaskHandler : submit
AwsBatchTaskHandler --> AwsBatchScriptLauncher : submit
%% TaskPollingMonitor <|-- ParallelPollingMonitor
SimpleFileCopyStrategy <|-- AwsBatchFileCopyStrategy
AwsBatchScriptLauncher --* AwsBatchFileCopyStrategy

33
nextflow/docs/developer/diagrams/nextflow.cloud.aws.nio.mmd Normal file
View File

@@ -0,0 +1,33 @@
classDiagram
%%
%% nextflow.cloud.aws.nio
%%
FileSystemProvider <|-- S3FileSystemProvider
S3FileSystemProvider --> S3FileSystem : newFileSystem
class S3FileSystem {
client : S3Client
endpoint : String
}
S3FileSystem --* S3Client
class S3Client {
client : AmazonS3
cannedAcl : CannedAccessControlList
kmsKeyId : String
storageEncryption : SSEAlgorithm
transferManager : TransferManager
transferPool : ExecutorService
uploadChunkSize : Long
uploadMaxThreads : Integer
}
Path <|-- S3Path
class S3Path {
bucket : String
parts : List~String~
fileSystem : S3FileSystem
}
S3Path --* S3FileSystem

17
nextflow/docs/developer/diagrams/nextflow.cloud.azure.mmd Normal file
View File

@@ -0,0 +1,17 @@
classDiagram
%%
%% nextflow.cloud.azure
%%
Executor <|-- AzBatchExecutor
TaskHandler <|-- AzBatchTaskHandler
BashWrapperBuilder <|-- AzBatchScriptLauncher
AzBatchExecutor --* AzConfig
AzBatchExecutor --> AzBatchService : register
AzBatchExecutor --> TaskPollingMonitor : init
AzBatchExecutor --> AzBatchTaskHandler : submit
AzBatchTaskHandler --> AzBatchScriptLauncher : submit
SimpleFileCopyStrategy <|-- AzFileCopyStrategy
AzBatchScriptLauncher --* AzFileCopyStrategy

13
nextflow/docs/developer/diagrams/nextflow.cloud.google.mmd Normal file
View File

@@ -0,0 +1,13 @@
classDiagram
%%
%% nextflow.cloud.google
%%
Executor <|-- GoogleBatchExecutor
TaskHandler <|-- GoogleBatchTaskHandler
BashWrapperBuilder <|-- GoogleBatchScriptLauncher
GoogleBatchExecutor --* BatchConfig
GoogleBatchExecutor --> TaskPollingMonitor : init
GoogleBatchExecutor --> GoogleBatchTaskHandler : submit
GoogleBatchTaskHandler --> GoogleBatchScriptLauncher : submit

7
nextflow/docs/developer/diagrams/nextflow.config.mmd Normal file
View File

@@ -0,0 +1,7 @@
classDiagram
%%
%% nextflow.config
%%
Session --* ConfigMap
CmdRun --> ConfigBuilder : run
ConfigBuilder --> ConfigMap : build

16
nextflow/docs/developer/diagrams/nextflow.container.mmd Normal file
View File

@@ -0,0 +1,16 @@
classDiagram
%%
%% nextflow.container
%%
direction LR
BashWrapperBuilder --> ContainerBuilder : build
ContainerBuilder <|-- CharliecloudBuilder
ContainerBuilder <|-- DockerBuilder
ContainerBuilder <|-- PodmanBuilder
ContainerBuilder <|-- ShifterBuilder
ContainerBuilder <|-- SingularityBuilder
ContainerBuilder <|-- UdockerBuilder
SingularityBuilder <|-- ApptainerBuilder

33
nextflow/docs/developer/diagrams/nextflow.dag.mmd Normal file
View File

@@ -0,0 +1,33 @@
classDiagram
%%
%% nextflow.dag
%%
Session --* DAG
class DAG {
vertices : List~Vertex~
edges : List~Edge~
}
DAG "1" --* "*" Vertex
DAG "1" --* "*" Edge
class Vertex {
label : String
type : Type
operators : List~DataflowProcessor~
process : TaskProcessor
}
class Edge {
channel : Object
from : Vertex
to : Vertex
label : String
}
%% DagRenderer <|-- CytoscapeHtmlRenderer
%% DagRenderer <|-- CytoscapeJsRenderer
%% DagRenderer <|-- DotRenderer
%% DagRenderer <|-- GexfRenderer
%% DagRenderer <|-- GraphvizRenderer
%% DagRenderer <|-- MermaidRenderer

66
nextflow/docs/developer/diagrams/nextflow.executor.mmd Normal file
View File

@@ -0,0 +1,66 @@
classDiagram
%%
%% nextflow.executor
%%
ExecutorFactory --> Executor : getExecutor
class Executor {
name : String
monitor : TaskMonitor
}
Executor --* TaskMonitor
Executor --> TaskHandler : submit
TaskMonitor <|-- TaskPollingMonitor
class TaskPollingMonitor {
capacity : int
submitRateLimit : RateLimiter
pollIntervalMillis : long
dumpInterval : Duration
}
TaskPollingMonitor <|-- LocalPollingMonitor
class LocalPollingMonitor {
maxCpus : int
maxMemory : long
}
Executor <|-- AbstractGridExecutor
Executor <|-- LocalExecutor
%% Executor <|-- NopeExecutor
%% AbstractGridExecutor <|-- CondorExecutor
%% AbstractGridExecutor <|-- HyperQueueExecutor
%% AbstractGridExecutor <|-- LsfExecutor
%% AbstractGridExecutor <|-- MoabExecutor
%% AbstractGridExecutor <|-- NqsiiExecutor
%% AbstractGridExecutor <|-- OarExecutor
%% AbstractGridExecutor <|-- PbsExecutor
%% AbstractGridExecutor <|-- SgeExecutor
%% AbstractGridExecutor <|-- SlurmExecutor
%% PbsExecutor <|-- PbsProExecutor
%% SgeExecutor <|-- CrgExecutor
%% TaskHandler <|-- CachedTaskHandler
TaskHandler <|-- GridTaskHandler
TaskHandler <|-- LocalTaskHandler
TaskHandler <|-- NativeTaskHandler
%% TaskHandler <|-- NopeTaskHandler
%% TaskHandler <|-- StoredTaskHandler
class BashWrapperBuilder {
bean : TaskBean
copyStrategy : ScriptFileCopyStrategy
}
BashWrapperBuilder --* TaskBean
BashWrapperBuilder --* ScriptFileCopyStrategy
ScriptFileCopyStrategy <|-- SimpleFileCopyStrategy
class SimpleFileCopyStrategy {
stageinMode : String
stageoutMode : String
targetDir : Path
workDir : Path
}

30
nextflow/docs/developer/diagrams/nextflow.extension.mmd Normal file
View File

@@ -0,0 +1,30 @@
classDiagram
%%
%% nextflow.extension
%%
direction LR
ChannelEx --> DumpOp : dump
Nextflow --> GroupKey : groupKey
OperatorImpl --> BranchOp : branch
OperatorImpl --> BufferOp : buffer
OperatorImpl --> CollectFileOp : collectFile
OperatorImpl --> CollectOp : collect
OperatorImpl --> CombineOp : combine
OperatorImpl --> ConcatOp : concat
OperatorImpl --> CrossOp : cross
OperatorImpl --> GroupTupleOp : groupTuple
OperatorImpl --> JoinOp : join
OperatorImpl --> MapOp : map
OperatorImpl --> MergeOp : merge
OperatorImpl --> MixOp : mix
OperatorImpl --> MultiMapOp : multiMap
OperatorImpl --> RandomSampleOp : randomSample
OperatorImpl --> SplitOp : splitCsv, splitFasta, splitFastq, splitText
OperatorImpl --> TakeOp : take
OperatorImpl --> ToListOp : toList, toSortedList
OperatorImpl --> TransposeOp : transpose
OperatorImpl --> UntilOp : until
WorkflowBinding --> OpCall : invokeMethod

14
nextflow/docs/developer/diagrams/nextflow.ga4gh.mmd Normal file
View File

@@ -0,0 +1,14 @@
classDiagram
%%
%% nextflow.ga4gh
%%
Executor <|-- TesExecutor
%% TaskHandler <|-- TesTaskHandler
%% BashWrapperBuilder <|-- TesBashBuilder
TesExecutor --> TaskPollingMonitor : init
TesExecutor --> TesTaskHandler : submit
TesTaskHandler --> TesBashBuilder : submit
%% ScriptFileCopyStrategy <|-- TesFileCopyStrategy
TesBashBuilder --* TesFileCopyStrategy

57
nextflow/docs/developer/diagrams/nextflow.k8s.mmd Normal file
View File

@@ -0,0 +1,57 @@
classDiagram
%%
%% nextflow.k8s
%%
Executor <|-- K8sExecutor
TaskHandler <|-- K8sTaskHandler
BashWrapperBuilder <|-- K8sWrapperBuilder
K8sExecutor --> TaskPollingMonitor : init
K8sExecutor --> K8sTaskHandler : submit
K8sExecutor --* K8sClient
K8sTaskHandler --> K8sWrapperBuilder : submit
CmdKubeRun --> K8sDriverLauncher : run
class K8sDriverLauncher {
args : List~String~
cmd : CmdKubeRun
config : ConfigObject
configMapName : String
headCpus : int
headImage : String
headMemory : String
headPreScript : String
paramsFile : String
pipelineName : String
runName : String
}
K8sDriverLauncher --* K8sClient
K8sDriverLauncher --* K8sConfig
K8sClient --* ClientConfig
%% ConfigDiscovery --> ClientConfig : discover
class K8sConfig {
target : Map
podOptions : PodOptions
}
K8sConfig --* PodOptions
class PodOptions {
affinity : Map
annotations : Map
automountServiceAccountToken : boolean
configMaps : Collection~PodMountConfig~
envVars : Collection~PodEnv~
imagePullPolicy : String
imagePullSecret : String
labels : Map
nodeSelector : PodNodeSelector
priorityClassName : String
privileged : Boolean
secrets : Collection~PodMountSecret~
securityContext : PodSecurityContext
tolerations : List~Map~
volumeClaims : Collection~PodVolumeClaim~
}

21
nextflow/docs/developer/diagrams/nextflow.mmd Normal file
View File

@@ -0,0 +1,21 @@
classDiagram
%%
%% nextflow
%%
class Nextflow
class Channel
class Session {
baseDir : Path
binding : ScriptBinding
cache : CacheDB
commandLine : String
commitId : String
config : Map
configFiles : List~Path~
dag : DAG
profile : String
runName : String
script : BaseScript
uniqueId : UUID
workDir : Path
}

14
nextflow/docs/developer/diagrams/nextflow.plugin.mmd Normal file
View File

@@ -0,0 +1,14 @@
classDiagram
%%
%% nextflow.plugin
%%
CmdRun --> Plugins : run
Plugins --> PluginsFacade : init
PluginsFacade "1" --> "*" PluginRef : load
class PluginRef {
id : String
version : String
}

44
nextflow/docs/developer/diagrams/nextflow.processor.mmd Normal file
View File

@@ -0,0 +1,44 @@
classDiagram
%%
%% nextflow.processor
%%
%% ProcessDef --> TaskProcessor : run
class TaskProcessor {
config : ProcessConfig
executor : Executor
id : int
name : String
operator : DataflowProcessor
taskBody : BodyDef
}
TaskProcessor --> TaskRun : invokeTask
TaskProcessor --> PublishDir : finalizeTask
class TaskRun {
config : TaskConfig
context : TaskContext
hash : HashCode
id : TaskId
index : int
inputs : Map
name : String
outputs : Map
runType : RunType
type : ScriptType
workDir : Path
}
TaskRun --* TaskConfig
TaskRun --* TaskContext
TaskRun --> TaskBean : toTaskBean
class TaskConfig {
target : Map
binding : Map
}
class TaskContext {
holder : Map
script : Script
name : String
}

65
nextflow/docs/developer/diagrams/nextflow.scm.mmd Normal file
View File

@@ -0,0 +1,65 @@
classDiagram
%%
%% nextflow.scm
%%
direction LR
CmdRun --> AssetManager : run
class AssetManager {
project : String
mainScript : String
provider : RepositoryProvider
strategy : RepositoryStrategy
hub : String
providerConfigs : List~ProviderConfig~
}
class RepositoryStrategyType {
<<enumeration>>
LEGACY
MULTI_REVISION
}
AssetManager --> RepositoryStrategyType
AssetManager "1" --o "1" RepositoryStrategy
AssetManager "1" --o "1" RepositoryProvider
AssetManager "1" --* "*" ProviderConfig
class RepositoryStrategy {
<<interface>>
}
class AbstractRepositoryStrategy {
<<abstract>>
project : String
provider : RepositoryProvider
root : File
}
class LegacyRepositoryStrategy {
localPath : File
}
class MultiRevisionRepositoryStrategy {
revision : String
bareRepo : File
commitPath : File
revisionSubdir : File
}
RepositoryStrategy <|-- AbstractRepositoryStrategy
AbstractRepositoryStrategy <|-- LegacyRepositoryStrategy
AbstractRepositoryStrategy <|-- MultiRevisionRepositoryStrategy
class RepositoryProvider {
<<abstract>>
}
RepositoryStrategy --> RepositoryProvider
RepositoryProvider <|-- AzureRepositoryProvider
RepositoryProvider <|-- BitbucketRepositoryProvider
RepositoryProvider <|-- BitbucketServerRepositoryProvider
RepositoryProvider <|-- GiteaRepositoryProvider
RepositoryProvider <|-- GithubRepositoryProvider
RepositoryProvider <|-- GitlabRepositoryProvider
RepositoryProvider <|-- LocalRepositoryProvider

130
nextflow/docs/developer/diagrams/nextflow.script.mmd Normal file
View File

@@ -0,0 +1,130 @@
classDiagram
%%
%% nextflow.script
%%
CmdRun --> ScriptRunner : run
class ScriptRunner {
scriptFile : ScriptFile
session : Session
}
ScriptRunner --* ScriptFile
ScriptRunner --> ScriptParser : execute
ScriptParser --> BaseScript : parse
class ScriptFile {
source : Path
main : Path
repository : String
revisionInfo : AssetManager.RevisionInfo
localPath : Path
projectName : String
}
class BaseScript {
meta : ScriptMeta
entryFlow : WorkflowDef
}
BaseScript --* ScriptBinding
BaseScript --* ScriptMeta
BaseScript --> IncludeDef : include
IncludeDef --> ScriptParser : load0
class ScriptBinding {
scriptPath : Path
args : List~String~
params : ParamsMap
configEnv : Map
entryName : String
}
class ScriptMeta {
scriptPath : Path
definitions : Map
imports : Map
module : boolean
}
ScriptMeta "1" --* "*" ComponentDef : definitions
ScriptMeta "1" --* "*" ComponentDef : imports
ComponentDef <|-- FunctionDef
ComponentDef <|-- ProcessDef
ComponentDef <|-- WorkflowDef
class FunctionDef {
target : Object
name : String
alias : String
}
class ProcessDef {
processName : String
simpleName : String
baseName : String
rawBody : Closure~BodyDef~
}
ProcessDef --* ProcessConfig
ProcessDef --* BodyDef
ProcessDef --* ChannelOut
class WorkflowDef {
name : String
body : BodyDef
declaredInputs : List~String~
declaredOutputs : List~String~
variableNames : Set~String~
}
WorkflowDef --* BodyDef
WorkflowDef --* WorkflowBinding
WorkflowDef --* ChannelOut
class ProcessConfig {
configProperties : Map
inputs : InputsList
outputs : OutputsList
}
ProcessConfig --* InputsList
ProcessConfig --* OutputsList
class BodyDef {
closure : Closure
source : String
type : ScriptType
isShell : boolean
}
class ChannelOut {
target : List~DataflowWriteChannel~
channels : Map
}
class WorkflowBinding {
vars : Map
}
class InputsList {
target : List~InParam~
}
InputsList "1" --* "*" InParam
class OutputsList {
target : List~OutParam~
}
OutputsList "1" --* "*" OutParam
%% InParam <|-- BaseInParam
%% BaseInParam <|-- EachInParam
%% BaseInParam <|-- EnvInParam
%% BaseInParam <|-- FileInParam
%% BaseInParam <|-- StdInParam
%% BaseInParam <|-- TupleInParam
%% BaseInParam <|-- ValueInParam
%% OutParam <|-- BaseOutParam
%% BaseOutParam <|-- EachOutParam
%% BaseOutParam <|-- EnvOutParam
%% BaseOutParam <|-- FileOutParam
%% BaseOutParam <|-- StdOutParam
%% BaseOutParam <|-- TupleOutParam
%% BaseOutParam <|-- ValueOutParam

12
nextflow/docs/developer/diagrams/nextflow.secret.mmd Normal file
View File

@@ -0,0 +1,12 @@
classDiagram
%%
%% nextflow.secret
%%
ConfigBuilder --> SecretsLoader : build
BaseScript --> SecretsLoader : run
BashWrapperBuilder --> SecretsLoader : build
SecretsLoader --> SecretsProvider : load
SecretsProvider --> Secret : getSecret
SecretsProvider <|-- LocalSecretsProvider
Secret <|-- SecretImpl

13
nextflow/docs/developer/diagrams/nextflow.trace.mmd Normal file
View File

@@ -0,0 +1,13 @@
classDiagram
%%
%% nextflow.trace
%%
Session --> TraceObserverFactory : init
TraceObserverFactory "1" --> "*" TraceObserver : create
TraceObserver <|-- AnsiLogObserver
TraceObserver <|-- GraphObserver
TraceObserver <|-- ReportObserver
TraceObserver <|-- TimelineObserver
TraceObserver <|-- TraceFileObserver
TraceObserver <|-- WorkflowStatsObserver

69
nextflow/docs/developer/diagrams/overview.mmd Normal file
View File

@@ -0,0 +1,69 @@
flowchart TB
subgraph Launcher
subgraph CmdRun
subgraph AssetManager
ScriptFile
end
subgraph ConfigBuilder
ConfigParser([ConfigParser])
ConfigBase([ConfigBase])
end
subgraph ScriptRunner
subgraph Session
ConfigMap
DAG
ExecutorFactory([ExecutorFactory])
subgraph TaskProcessor
TaskRun
end
subgraph Executor
subgraph TaskMonitor
TaskHandler
end
TaskBean
BashWrapperBuilder([BashWrapperBuilder])
end
TraceRecord
CacheFactory([CacheFactory])
CacheDB
TraceObserver([TraceObserver])
end
ScriptParser([ScriptParser])
BaseScript([BaseScript])
subgraph ScriptMeta
WorkflowDef([WorkflowDef])
ProcessDef([ProcessDef])
FunctionDef([FunctionDef])
end
IncludeDef([IncludeDef])
OpCall([OpCall])
end
ConfigParser --> ConfigBase
ConfigBase --> ConfigMap
ScriptFile --> ScriptParser
ScriptParser --> BaseScript
BaseScript --> WorkflowDef
BaseScript --> ProcessDef
BaseScript --> FunctionDef
BaseScript --> IncludeDef
IncludeDef --> ScriptParser
WorkflowDef --> OpCall
OpCall --> DAG
ProcessDef --> DAG
DAG --> TaskRun
TaskRun --> DAG
ExecutorFactory --> Executor
ConfigMap --> Executor
ProcessDef --> TaskProcessor
ConfigMap --> TaskProcessor
TaskRun --> TaskHandler
TaskRun --> TaskBean
TaskBean --> BashWrapperBuilder
BashWrapperBuilder --> TaskHandler
CacheFactory --> CacheDB
TaskHandler --> CacheDB
TaskHandler --> TraceRecord
TraceRecord --> CacheDB
TaskHandler --> TraceObserver
end
end

181
nextflow/docs/developer/index.md Normal file
View File

@@ -0,0 +1,181 @@
(contributing-page)=
# Overview
This section provides a high-level overview of the Nextflow source code for users who want to understand or contribute to it. Rather than a comprehensive API documentation, these docs simply provide a conceptual map to help you understand the key concepts of the Nextflow implementation, and to quickly find code sections of interest for further investigation.
Before you dive into code, be sure to check out the [CONTRIBUTING.md](https://github.com/nextflow-io/nextflow/blob/master/CONTRIBUTING.md) for Nextflow to learn about the many ways to contribute to the project.
## IntelliJ IDEA
The suggested development environment is [IntelliJ IDEA](https://www.jetbrains.com/idea/download/). Nextflow development with IntelliJ IDEA requires a recent version of the IDE (2019.1.2 or later).
After installing IntelliJ IDEA, use the following steps to use it with Nextflow:
1. Clone the Nextflow repository to a directory in your computer.
2. Open IntelliJ IDEA and go to **File > New > Project from Existing Sources...**.
3. Select the Nextflow project root directory in your computer and click **OK**.
4. Select **Import project from external model > Gradle** and click **Finish**.
5. After the import process completes, select **File > Project Structure...**.
6. Select **Project**, and make sure that the **SDK** field contains Java 11 (or later).
7. Go to **File > Settings > Editor > Code Style > Groovy > Imports** and apply the following settings:
* Use single class import
* Class count to use import with '*': `99`
* Names count to use static import with '*': `99`
* Imports layout:
* `import java.*`
* `import javax.*`
* *blank line*
* all other imports
* all other static imports
New files must include the appropriate license header boilerplate and the author name(s) and contact email(s) ([see for example](https://github.com/nextflow-io/nextflow/blob/e8945e8b6fc355d3f2eec793d8f288515db2f409/modules/nextflow/src/main/groovy/nextflow/Const.groovy#L1-L15)).
## Groovy
Nextflow is written in [Groovy](http://groovy-lang.org/), which is itself a programming language based on [Java](https://www.java.com/). Groovy is designed to be highly interoperable with Java -- Groovy programs compile to Java bytecode, and nearly any Java program is also a valid Groovy program. However, Groovy adds several language features (e.g. closures, list and map literals, optional typing, optional semicolons, meta-programming) and standard libraries (e.g. JSON and XML parsing) that greatly improve the overall experience of developing for the Java virtual machine.
Recommended resources for Groovy, from most reference-complete to most user-friendly, are listed below:
- [Groovy documentation](http://groovy-lang.org/documentation.html)
- [Groovy in Action](https://www.manning.com/books/groovy-in-action-second-edition)
- [Groovy: The Awesome Parts](https://www.slideshare.net/paulk_asert/awesome-groovy)
- [Groovy cheat sheet](http://www.cheat-sheets.org/saved-copy/rc015-groovy_online.pdf)
## Software Dependencies
Nextflow depends on a variety of libraries and frameworks, the most prominent of which are listed below:
- [AWS SDK for Java 1.x](https://aws.amazon.com/sdk-for-java/): AWS integration
- [Azure SDK for Java](https://learn.microsoft.com/en-us/azure/developer/java/sdk/): Azure integration
- [Google Cloud Client Libraries for Java](https://cloud.google.com/java/docs/reference): Google Cloud integration
- [GPars](http://gpars.org/1.2.1/guide/guide/dataflow.html): dataflow concurrency
- [Gradle](https://gradle.org/): build automation
- [JCommander](https://jcommander.org/): command line interface
- [JGit](https://www.eclipse.org/jgit/): Git integration
- [Kryo](https://github.com/EsotericSoftware/kryo): serialization
- [LevelDB](https://mvnrepository.com/artifact/org.iq80.leveldb/leveldb): key-value store for the cache database
- [Logback](https://logback.qos.ch/): application logging
- [PF4J](https://pf4j.org/): plugin extensions
- [Spock](https://spockframework.org/): unit testing framework
Any other integrations are likely implemented using a CLI (e.g. Conda, Docker, HPC schedulers) or REST API (e.g. Kubernetes).
## Class Diagrams
Each package has a class diagram, abridged and annotated for relevance and ease of use.
Each node is a class. Fields are selectively documented in order to show only the core data structures and the classes that "own" them. Methods are not explicitly documented, but they are mentioned in certain links where appropriate. Links are selectively documented in order to show only the most important classes and relationships.
Links between classes denote one of the following relationships:
- Inheritance (`A <|-- B`): `B` is a subclass of `A`
- Composition (`A --* B`): `A` contains `B`
- Instantiation (`A --> B : f`): `A` creates instance(s) of `B` at runtime via `A::f()`
See {ref}`packages-page` for the list of Nextflow packages.
```{warning}
Class diagrams are manually curated, so they might not always reflect the latest version of the source code.
```
## Building from source
If you are interested in modifying the source code, you only need Java 11 or later to build Nextflow from source. Nextflow uses the [Gradle](http://www.gradle.org/) build automation system, but you do not need to install Gradle to build Nextflow. In other words, if you can run Nextflow, then you can probably build it too!
To build locally from a branch (useful for testing PRs):
```bash
git clone -b <branch> git@github.com:nextflow-io/nextflow.git
cd nextflow
make compile
```
The build system will automatically download all of the necessary dependencies on the first run, which may take several minutes.
Once complete, you can run your local build of Nextflow using the `launch.sh` script in place of the `nextflow` command:
```bash
./launch.sh run <script> ...
```
Alternatively, you can build a self-contained executable with the following command:
```bash
make pack
```
It will create a binary in the `build/releases` directory which can be used in place of `nextflow`. This approach is useful when testing a pipeline that uses third-party plugins, which is not supported by `launch.sh`.
## Testing
To run the unit tests:
```bash
# run all tests
make test
# run individual test
make test module=<nextflow|plugins:nf-amazon|...> class=<package>.<class>.<method>
# refer to the Makefile for all build rules
```
When a test fails, it will give you a report that you can open in your browser to view the reason for each failed test. The **Standard output** tab is particularly useful as it shows the console output of each test.
To run the integration tests:
```bash
cd tests/checks
./qrun.sh
```
To run a specific integration test:
```bash
cd tests/checks
./qrun.sh <FOLDER>
```
To test the documentation snippets:
```bash
cd docs/snippets
./test.sh
```
Refer to the [build.yml](https://github.com/nextflow-io/nextflow/tree/master/.github/workflows/build.yml) configuration to see how to run other end-to-end tests locally.
## Debugging
### Groovy REPL
The `groovysh` command provides a command-line REPL that you can use to play around with Groovy code independently of Nextflow. The `groovyConsole` command provides a graphical REPL similar to `nextflow console`. These commands require a standalone Groovy distribution, which can be installed as described for Java on the {ref}`Installation <install-requirements>` page.
:::{note}
If you are using WSL, you must also install an X server for Windows, such as [VcXsrv](https://sourceforge.net/projects/vcxsrv/) or [Xming](http://www.straightrunning.com/XmingNotes/), in order to use these commands.
:::
### IntelliJ IDEA
:::{versionadded} 23.09.0-edge
:::
You can perform limited breakpoint debugging on a Nextflow script using IntelliJ IDEA.
1. Set a breakpoint in your Nextflow script by clicking on a line number.
2. Run `nextflow -remote-debug run <script>`
3. Select the **Run / Debug Configurations** dropdown, select **Edit Configurations...**, and create a new configuration of type **Remote JVM Debug**. Set the port that appeared in the terminal when you launched your Nextflow script. Click **OK**.
4. Select the green bug icon to begin the remote debug session. The Debug window will appear and allow you to step through and inspect your script as it runs.
Note that this approach can only be used to debug the *script* execution, which does not include the *pipeline* execution.

172
nextflow/docs/developer/nextflow.ast.md Normal file
View File

@@ -0,0 +1,172 @@
# `nextflow.ast`
The `nextflow.ast` package implements the Nextflow language extensions as AST transforms.
## Class Diagram
```{mermaid} diagrams/nextflow.ast.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The Nextflow scripting language is essentially Groovy with some extensions, implemented as transformations to the abstract syntax tree (AST). Every Nextflow script is syntactically (but not semantically) valid Groovy.
You can see the effect of Nextflow's AST transforms by using the Nextflow console:
1. Run `nextflow console` to open the console
2. Enter a Nextflow script
3. Execute the script
4. Go to **Script** > **Inspect AST**
Here is the example from {ref}`your-first-script`:
```nextflow
params.str = 'Hello world!'
process split_letters {
output:
path 'chunk_*'
script:
"""
printf '${params.str}' | split -b 6 - chunk_
"""
}
process convert_to_upper {
input:
path x
output:
stdout
script:
"""
cat $x | tr '[a-z]' '[A-Z]'
"""
}
workflow {
split_letters | flatten | convert_to_upper | view { it.trim() }
}
```
Here it is after being parsed and de-sugared by Groovy:
```groovy
params.str = 'Hello world!'
process( split_letters( {
output:
path('chunk_*')
script:
"""
printf '${params.str}' | split -b 6 - chunk_
"""
} ))
process( convert_to_upper( {
input:
path(x)
output:
stdout
script:
"""
cat $x | tr '[a-z]' '[A-Z]'
"""
} ))
workflow({
split_letters | flatten | convert_to_upper | view { it.trim() }
})
```
Here it is after being transformed by Nextflow (whitespace edited for readability):
```groovy
import static nextflow.Nextflow.*
import org.apache.commons.lang.StringUtils as StringUtils
import groovy.transform.Field as Field
import java.nio.file.Path as Path
import nextflow.Channel as Channel
import nextflow.util.Duration as Duration
import nextflow.util.MemoryUnit as MemoryUnit
import nextflow.io.ValueObject as ValueObject
import nextflow.Channel as channel
@groovy.transform.BaseScript
public class script1677225313239 extends nextflow.script.BaseScript {
public script1677225313239() {
nextflow.script.ScriptMeta.get(this).setDsl1ProcessNames(['split_letters', 'convert_to_upper'])
}
public script1677225313239(final groovy.lang.Binding context) {
super.setBinding(context)
nextflow.script.ScriptMeta.get(this).setDsl1ProcessNames(['split_letters', 'convert_to_upper'])
}
public static void main(final java.lang.String[] args) {
org.codehaus.groovy.runtime.InvokerHelper.runScript(script1677225313239, args)
}
@groovy.transform.Generated
protected java.lang.Object runScript() {
params.str = 'Hello world!'
this.process('split_letters', {
this._out_path('chunk_*')
new nextflow.script.BodyDef(
{
"printf '$params.str' | split -b 6 - chunk_"
},
'"""\n printf \'${params.str}\' | split -b 6 - chunk_\n """\n',
'script',
[
new nextflow.script.TokenValRef('params.str', 8, 13)
]
)
})
this.process('convert_to_upper', {
this._in_path(new nextflow.script.TokenVar('x'))
this._out_stdout()
new nextflow.script.BodyDef(
{
"cat $x | tr '[a-z]' '[A-Z]'"
},
'"""\n cat $x | tr \'[a-z]\' \'[A-Z]\'\n """\n',
'script',
[
new nextflow.script.TokenValRef('x', 19, 8)
]
)
})
this.workflow({
new nextflow.script.BodyDef(
{
split_letters | flatten | convert_to_upper | this.view({
it.trim()
})
},
' split_letters | flatten | convert_to_upper | view { it.trim() }\n',
'workflow',
[
new nextflow.script.TokenValRef('flatten', 24, 18),
new nextflow.script.TokenValRef('split_letters', 24, 3),
new nextflow.script.TokenValRef('convert_to_upper', 24, 28)
]
)
})
}
}
```

21
nextflow/docs/developer/nextflow.cache.md Normal file
View File

@@ -0,0 +1,21 @@
# `nextflow.cache`
The `nextflow.cache` package implements the cache database of previously executed tasks.
## Class Diagram
```{mermaid} diagrams/nextflow.cache.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The cache database uses [Kryo](https://github.com/EsotericSoftware/kryo) to serialize and deserialize task data. Each key-value pair in the cache database corresponds to a task. The key is the task hash, and the value consists of (1) the task `TraceRecord`, (2) the `TaskContext`, and (3) the task reference count.
The default cache store is backed by [LevelDB](https://mvnrepository.com/artifact/org.iq80.leveldb/leveldb) and is stored in `.nextflow/cache/<session-id>` relative to the launch directory.
The cloud cache store is backed by remote object storage such as Amazon S3, Azure Blob Storage, and Google Cloud Storage. It stores each task entry as a separate object.

17
nextflow/docs/developer/nextflow.cli.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.cli`
The `nextflow.cli` package implements the command line interface.
## Class Diagram
```{mermaid} diagrams/nextflow.cli.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The `Launcher` class is the entrypoint for Nextflow. It uses [JCommander](https://jcommander.org/) to parse the command-line arguments. Additionally, there is a class for each subcommand which implements the application logic of that command. By far the most complex command is `CmdRun`.

13
nextflow/docs/developer/nextflow.cloud.aws.md Normal file
View File

@@ -0,0 +1,13 @@
# `nextflow.cloud.aws`
The `nextflow.cloud.aws` package implements the AWS Batch executor.
## Class Diagram
```{mermaid} diagrams/nextflow.cloud.aws.mmd
```
```{note}
Some classes may be excluded from the above diagrams for brevity.
```

17
nextflow/docs/developer/nextflow.cloud.aws.nio.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.cloud.aws.nio`
The `nextflow.cloud.aws.nio` package implements the S3 filesystem.
## Class Diagram
```{mermaid} diagrams/nextflow.cloud.aws.nio.mmd
```
```{note}
Some classes may be excluded from the above diagrams for brevity.
```
## Notes
The S3 filesystem translates Java Path API calls into S3 API calls, which allows Nextflow to interact with S3 objects through the same interface for local files.

13
nextflow/docs/developer/nextflow.cloud.azure.md Normal file
View File

@@ -0,0 +1,13 @@
# `nextflow.cloud.azure`
The `nextflow.cloud.azure` package implements the Azure Batch executor.
## Class Diagram
```{mermaid} diagrams/nextflow.cloud.azure.mmd
```
```{note}
Some classes may be excluded from the above diagrams for brevity.
```

13
nextflow/docs/developer/nextflow.cloud.google.md Normal file
View File

@@ -0,0 +1,13 @@
# `nextflow.cloud.google`
The `nextflow.cloud.google` package implements the Google Batch executor.
## Class Diagram
```{mermaid} diagrams/nextflow.cloud.google.mmd
```
```{note}
Some classes may be excluded from the above diagrams for brevity.
```

19
nextflow/docs/developer/nextflow.config.md Normal file
View File

@@ -0,0 +1,19 @@
# `nextflow.config`
The `nextflow.config` package contains the implementation of the Nextflow configuration.
## Class Diagram
```{mermaid} diagrams/nextflow.config.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
Any command that parses Nextflow config files (`config`, `run`, etc) uses the `ConfigBuilder` to build a `ConfigMap` from a set of config files. The `ConfigBuilder` itself uses a `ConfigParser` to parse the config files.
The Nextflow configuration language is essentially Groovy with some extensions. These extensions are implemented in `ConfigBase` and `ConfigTransformImpl`.

19
nextflow/docs/developer/nextflow.container.md Normal file
View File

@@ -0,0 +1,19 @@
# `nextflow.container`
The `nextflow.container` package implements the integration with container runtimes.
## Class Diagram
```{mermaid} diagrams/nextflow.container.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The `ContainerBuilder` class is the base class for all container runtimes supported by Nextflow. It produces the container wrapper command for a given task run.
Executors that support containerized tasks insert this wrapper command into the task wrapper script (`.command.run`). Executors that are *container-native*, i.e. that launch the task wrapper itself inside a container, don't need to generate a container wrapper command.

19
nextflow/docs/developer/nextflow.dag.md Normal file
View File

@@ -0,0 +1,19 @@
# `nextflow.dag`
The `nextflow.dag` package implements the workflow DAG and renderers for several diagram formats.
## Class Diagram
```{mermaid} diagrams/nextflow.dag.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The workflow DAG defines the network of processes, channels, and operators that comprise a workflow. It is produced by the execution of the Nextflow script. See [nextflow.script](nextflow.script.md) for more details.
Implementations of the `DagRenderer` interface define how to render the workflow DAG to a particular diagram format. See {ref}`workflow-diagram` for more details.

21
nextflow/docs/developer/nextflow.executor.md Normal file
View File

@@ -0,0 +1,21 @@
# `nextflow.executor`
The `nextflow.executor` package defines the executor interface and implements several built-in executors.
## Class Diagram
```{mermaid} diagrams/nextflow.executor.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The `Executor` class is the base class for all Nextflow executors. The main purpose of an `Executor` is to submit tasks to an underlying compute environment, such as an HPC scheduler or cloud batch executor. It uses a `TaskMonitor` to manage the lifecycle of all tasks and a `TaskHandler` to manage each individual task. Most executors use the same polling monitor, but each executor implements its own task handler to customize it for a particular compute environment. See [nextflow.processor](nextflow.processor.md) for more details about these classes.
The built-in executors include the local executor (`LocalExecutor`) and the various grid executors (SLURM, PBS, LSF, etc), all of which extend `AbstractGridExecutor`. The `LocalExecutor` implements both "script" tasks (processes with a `script` or `shell` block) and "native" tasks (processes with an `exec` block).
The `BashWrapperBuilder` is used by executors to generate the wrapper script (`.command.run`) for a task, from a template script called `command-run.txt`, as well as the task configuration and the execution environment.

19
nextflow/docs/developer/nextflow.extension.md Normal file
View File

@@ -0,0 +1,19 @@
# `nextflow.extension`
The `nextflow.extension` package implements the channel operators and other extension methods.
## Class Diagram
```{mermaid} diagrams/nextflow.extension.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
Operators are implemented using the [GPars](http://gpars.org/1.2.1/guide/guide/dataflow.html) dataflow library. In general, an operator consumes one or more `DataflowReadChannel`s and produces one or more `DataflowWriteChannel`s. See {ref}`operator-page` for details about each operator.
Other notable classes include `Bolts` and `FilesEx`, which implement various extension methods used throughout the Nextflow codebase. If you see a method that doesn't appear to be implemented by the calling object, it may be implemented in one of these extension classes.

17
nextflow/docs/developer/nextflow.k8s.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.k8s`
The `nextflow.k8s` package implements the Kubernetes executor and the `kuberun` command.
## Class Diagram
```{mermaid} diagrams/nextflow.k8s.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The Kubernetes integration uses the K8s HTTP API to interact with K8s clusters, and relies on the `kubectl` command and `~/.kube/config` file for authentication.

21
nextflow/docs/developer/nextflow.md Normal file
View File

@@ -0,0 +1,21 @@
# `nextflow`
The `nextflow` package contains various top-level classes.
## Class Diagram
```{mermaid} diagrams/nextflow.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The `Nextflow` class implements several methods that are exposed to Nextflow scripts. See {ref}`stdlib-namespaces-global` for details.
The `Channel` class implements the channel factory methods, and it is exposed directly to Nextflow scripts. See {ref}`channel-factory` for details.
The `Session` class is the top-level representation of a Nextflow run, or "session". See [nextflow.script](nextflow.script.md) for more details about how a `Session` is created.

17
nextflow/docs/developer/nextflow.plugin.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.plugin`
The `nextflow.plugin` package implements the plugin manager.
## Class Diagram
```{mermaid} diagrams/nextflow.plugin.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The plugin system uses the [PF4J](https://pf4j.org/) library, which allows for extension classes to be loaded at runtime. Each plugin includes a manifest of extension classes, all of which extend or implement some base class in Nextflow. The `Plugins` class can be used to query the available extensions for a given base class. Extensions can be assigned a priority using the `@Priority` annotation, to ensure that certain extensions are used over others when available.

23
nextflow/docs/developer/nextflow.processor.md Normal file
View File

@@ -0,0 +1,23 @@
# `nextflow.processor`
The `nextflow.processor` package implements the execution and monitoring of tasks.
## Class Diagram
```{mermaid} diagrams/nextflow.processor.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
While the [`executor`](nextflow.executor.md) package defines how tasks are submitted to a particular execution backend (such as an HPC scheduler), the `processor` package defines how tasks are created and executed. As such, these packages work closely together, and in fact several components of the `Executor` interface, specifically the `TaskHandler` and `TaskMonitor`, are defined in this package.
The `TaskProcessor` is by far the largest and most complex class in this package. It implements both the dataflow operator for a given process as well as the task execution logic. In other words, it defines the mapping from an abstract process definition with input and output channels into concrete task executions.
A `TaskRun` represents a particular task execution. There is also `TaskBean`, which is a serializable representation of a task. Legends say that `TaskBean` was originally created to support a "daemon" mode in which Nextflow would run on both the head node and the worker nodes, so the Nextflow "head" would need to send tasks to the Nextflow "workers". This daemon mode was never completed, but echoes of it remain (see `CmdNode`, `DaemonLauncher`, and the `nf-ignite` plugin).
When a `TaskProcessor` receives a set of input values, it creates a `TaskRun` and submits it to an `Executor`, which in turn submits the task to a underlying execution backend. The executor's `TaskMonitor` then monitors the status of the task, and when it is completed, returns it to the task processor for finalization. If the task completed successfully, the task processor collects the task outputs and emits them on the corresponding output channels. If the task failed, the task processor will retry it if possible, or else return a task error to the workflow run.

63
nextflow/docs/developer/nextflow.scm.md Normal file
View File

@@ -0,0 +1,63 @@
# `nextflow.scm`
The `nextflow.scm` package defines the Git provider interface and implements several built-in Git providers. It also manages local pipeline repositories using a Strategy pattern to support different repository management approaches.
## Class Diagram
```{mermaid} diagrams/nextflow.scm.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Architecture overview
### Repository strategies
The `AssetManager` uses the **Strategy pattern** to support different ways of managing local pipeline installations:
- **`LegacyRepositoryStrategy`**: Traditional approach where each project gets a full git clone in `$NXF_HOME/{project}` directory. Only one revision can exist at a time per project.
- **`MultiRevisionRepositoryStrategy`**: Modern approach that allows multiple revisions to coexist efficiently by:
- Maintaining a shared bare repository in `$NXF_HOME/.repos/{project}/bare/`
- Creating lightweight clones for each commit in `$NXF_HOME/.repos/{project}/clones/{commitId}/`
- Sharing git objects between revisions to minimize disk space
### Strategy selection
The `AssetManager` automatically selects the appropriate strategy based on:
1. **Environment variable**: `NXF_SCM_LEGACY=true` forces legacy mode
2. **Repository status**: Detected by checking existing repository structure:
- `UNINITIALIZED`: No repository exists, use Multi-Revision (default)
- `LEGACY_ONLY`: Only legacy `.git` directory exists, use Legacy
- `BARE_ONLY`: Only bare repository exists, use Multi-Revision
- `HYBRID`: Both exist, prefer Multi-Revision
### Repository provider
The `RepositoryProvider` class is the base class for all Git providers. It defines how to authenticate with the provider, clone a Git repository, inspect branches and tags, etc. The provider is used by repository strategies to interact with remote Git services.
## Key components
### AssetManager
Central class that manages pipeline assets. Key responsibilities include:
- Project name resolution and validation
- Strategy selection and initialization
- Provider configuration and authentication
- Repository download and checkout operations
- Coordination between strategy and provider
### RepositoryStrategy
Interface defining for repository management operations:
- `download()`: Download or update a revision
- `checkout()`: Switch to a specific revision
- `drop()`: Delete local copies
- `getLocalPath()`: Get path to working directory
- `getGit()`: Access JGit repository instance

21
nextflow/docs/developer/nextflow.script.md Normal file
View File

@@ -0,0 +1,21 @@
# `nextflow.script`
The `nextflow.script` package implements the parsing and execution of Nextflow scripts.
## Class Diagram
```{mermaid} diagrams/nextflow.script.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The execution of a Nextflow pipeline occurs in two phases. In the first phase, Nextflow parses and runs the script (using the language extensions in [nextflow.ast](nextflow.ast.md) and [nextflow.extension](nextflow.extension.md)), which produces the workflow DAG. In the second phase, Nextflow executes the workflow.
```{note}
In DSL1, there was no separation between workflow construction and execution -- dataflow operators were executed as soon as they were constructed. DSL2 introduced lazy execution in order to separate process definition from execution, and thereby facilitate subworkflows and modules.
```

17
nextflow/docs/developer/nextflow.secret.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.secret`
The `nextflow.secret` package defines the secrets provider interface and implements the built-in local secrets store.
## Class Diagram
```{mermaid} diagrams/nextflow.secret.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The default secrets provider simply stores key-value pairs in a local JSON file.

17
nextflow/docs/developer/nextflow.trace.md Normal file
View File

@@ -0,0 +1,17 @@
# `nextflow.trace`
The `nextflow.trace` package defines the trace observer interface and implements several built-in trace observers.
## Class Diagram
```{mermaid} diagrams/nextflow.trace.mmd
```
```{note}
Some classes may be excluded from the above diagram for brevity.
```
## Notes
The `TraceObserver` interface defines a set of hooks into the workflow execution, such as when a workflow starts and completes, when a task starts and completes, and when an output file is published. The `Session` maintains a list of all observers and triggers each hook when the corresponding event occurs. Implementing classes can use these hooks to perform custom behaviors. In fact, this interface is used to implement several core features, including the various execution reports, DAG renderer, and the integration with Seqera Platform.

12
nextflow/docs/developer/packages.md Normal file
View File

@@ -0,0 +1,12 @@
(packages-page)=
# Packages
The following subpages correspond to packages in the Nextflow source code:
```{toctree}
:glob:
:maxdepth: 1
nextflow*
```

598
nextflow/docs/executor.md Normal file
View File

@@ -0,0 +1,598 @@
(executor-page)=
# Executors
In the Nextflow framework architecture, the *executor* is the component that determines the system where a pipeline process is run and supervises its execution.
The executor provides an abstraction between the pipeline processes and the underlying execution system. This allows you to write the pipeline functional logic independently from the actual processing platform.
In other words, you can write your pipeline script once and have it running on your computer, a cluster resource manager, or the cloud — simply change the executor definition in the Nextflow configuration file.
(awsbatch-executor)=
## AWS Batch
Nextflow supports the [AWS Batch](https://aws.amazon.com/batch/) service that allows job submission in the cloud without having to spin out and manage a cluster of virtual machines. AWS Batch uses Docker containers to run tasks, which greatly simplifies pipeline deployment.
The pipeline processes must specify the Docker image to use by defining the `container` directive, either in the pipeline script or the `nextflow.config` file.
To enable this executor, set `process.executor = 'awsbatch'` in the `nextflow.config` file.
The pipeline can be launched either on a local computer, or an EC2 instance. EC2 is suggested for heavy or long-running workloads. Additionally, an S3 bucket must be used as the pipeline work directory.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-accelerator`
- {ref}`process-arch` (only when using Fargate platform type for AWS Batch)
- {ref}`process-container`
- {ref}`process-containerOptions`
- {ref}`process-cpus`
- {ref}`process-disk` (only when using Fargate platform type for AWS Batch)
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-resourcelabels`
- {ref}`process-time`
The following {ref}`hints <process-hints>` are supported:
- `consumableResources`: Specify [AWS Batch consumable resources](https://docs.aws.amazon.com/batch/latest/userguide/resource-aware-scheduling.html) as a list of name-value pairs. For example:
```nextflow
hints consumableResources: ['my-license-a': 1, 'my-license-b': 2]
```
See {ref}`aws-batch` for more information.
(azurebatch-executor)=
## Azure Batch
Nextflow supports the [Azure Batch](https://azure.microsoft.com/en-us/services/batch/) service that allows job submission in the cloud without having to spin out and manage a cluster of virtual machines. Azure Batch uses Docker containers to run tasks, which greatly simplifies pipeline deployment.
The pipeline processes must specify the Docker image to use by defining the `container` directive, either in the pipeline script or the `nextflow.config` file.
To enable this executor, set `process.executor = 'azurebatch'` in the `nextflow.config` file.
The pipeline can be launched either on a local computer, or a cloud virtual machine. The cloud VM is suggested for heavy or long-running workloads. Additionally, an Azure Blob storage container must be used as the pipeline work directory.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-container`
- {ref}`process-containerOptions`
- {ref}`process-cpus`
- {ref}`process-disk`
- {ref}`process-machineType`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-resourcelabels`
- {ref}`process-time`
See {ref}`azure-batch` for more information.
(bridge-executor)=
## Bridge
:::{versionadded} 22.09.1-edge
:::
[Bridge](https://github.com/cea-hpc/bridge) is an abstraction layer to ease batch system and resource manager usage in heterogeneous HPC environments.
It is open source software that can be installed on top of existing classical job schedulers such as Slurm, LSF, or other schedulers. Bridge allows you to submit jobs, get information on running jobs, stop jobs, get information on the cluster system, etc.
For more details on how to install the Bridge system, see the [documentation](https://github.com/cea-hpc/bridge).
To enable the Bridge executor, set `process.executor = 'bridge'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
(flux-executor)=
## Flux Executor
:::{versionadded} 22.11.0-edge
:::
The `flux` executor allows you to run your pipeline script using the [Flux Framework](https://flux-framework.org).
Nextflow submits each process to the cluster as a separate job using the `flux submit` command.
To enable the Flux executor, set `process.executor = 'flux'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-queue`
- {ref}`process-time`
:::{note}
Flux does not support the `memory` directive.
:::
:::{note}
By default, Flux will send all output to the `.command.log` file. To send this output to stdout and stderr instead, set `flux.terminalOutput = true` in your config file.
:::
(google-batch-executor)=
## Google Cloud Batch
:::{versionadded} 22.07.1-edge
:::
[Google Cloud Batch](https://cloud.google.com/batch) is a managed computing service that allows the execution of containerized workloads in the Google Cloud Platform infrastructure.
Nextflow provides built-in support for the Cloud Batch API, which allows the seamless deployment of Nextflow pipelines in the cloud, offloading the pipeline process executions.
The pipeline processes must specify the Docker image to use by defining the `container` directive, either in the pipeline script or the `nextflow.config` file. Additionally, the pipeline work directory must be located in a Google Storage bucket.
To enable this executor, set `process.executor = 'google-batch'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-accelerator`
- {ref}`process-container`
- {ref}`process-containerOptions`
- {ref}`process-cpus`
- {ref}`process-disk`
- {ref}`process-machineType`
- {ref}`process-memory`
- {ref}`process-resourcelabels`
- {ref}`process-time`
See the {ref}`Google Cloud Batch <google-batch>` page for further configuration details.
(htcondor-executor)=
## HTCondor
:::{warning} *Experimental: may change in a future release.*
:::
The `condor` executor allows you to run your pipeline script by using the [HTCondor](https://research.cs.wisc.edu/htcondor/) resource manager.
Nextflow manages each process as a separate job that is submitted to the cluster using the `condor_submit` command.
The pipeline must be launched from a node where the `condor_submit` command is available, which is typically the cluster login node.
:::{note}
The HTCondor executor for Nextflow does not currently support HTCondor's ability to transfer input/output data to the corresponding job's compute node. Therefore, the data must be made accessible to the compute nodes through a shared file system directory from where the Nextflow workflow is executed (or specified via the `-w` option).
:::
To enable the HTCondor executor, set `process.executor = 'condor'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-disk`
- {ref}`process-memory`
- {ref}`process-time`
(hyperqueue-executor)=
## HyperQueue
:::{versionadded} 22.05.0-edge
:::
:::{versionchanged} 24.06.0-edge
HyperQueue 0.17.0 or later is required.
:::
:::{versionchanged} 25.01.0-edge
HyperQueue 0.20.0 or later is required.
:::
The `hyperqueue` executor allows you to run your pipeline script by using the [HyperQueue](https://github.com/It4innovations/hyperqueue) job scheduler.
Nextflow manages each process as a separate job that is submitted to the cluster using the `hq` command line tool.
The pipeline must be launched from a node where the `hq` command is available, which is typically the cluster login node.
To enable the HyperQueue executor, set `process.executor = 'hq'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-accelerator`
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-time`
(k8s-executor)=
## Kubernetes
The `k8s` executor allows you to run a pipeline on a [Kubernetes](http://kubernetes.io/) cluster.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-accelerator`
- {ref}`process-cpus`
- {ref}`process-disk`
- {ref}`process-memory`
- {ref}`process-pod`
- {ref}`process-resourcelabels`
- {ref}`process-time`
See the {ref}`Kubernetes <k8s-page>` page to learn how to set up a Kubernetes cluster to run Nextflow pipelines.
(local-executor)=
## Local
The `local` executor is used by default. It runs the pipeline processes on the computer where Nextflow is launched. The processes are parallelized by spawning multiple threads, taking advantage of the multi-core architecture of the CPU.
The `local` executor is useful for developing and testing a pipeline script on your computer, before switching to a cluster or cloud environment with production data.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-time`
- {ref}`process-container`
- {ref}`process-containerOptions`
:::{note}
While the `local` executor limits the number of concurrent tasks based on requested vs available resources, it does not enforce task resource requests. In other words, it is possible for a local task to use more CPUs and memory than it requested, in which case it may starve other tasks. An exception to this behavior is when using {ref}`container-docker` or {ref}`container-podman` containers, in which case the resource requests are enforced by the container runtime.
:::
The local executor supports two types of tasks:
- Script tasks (processes with a `script` or `shell` block) - executed via a Bash wrapper
- Native tasks (processes with an `exec` block) - executed directly in the JVM.
(lsf-executor)=
## LSF
The `lsf` executor allows you to run your pipeline script using a [Platform LSF](http://en.wikipedia.org/wiki/Platform_LSF) cluster.
Nextflow manages each process as a separate job that is submitted to the cluster using the `bsub` command.
The pipeline must be launched from a node where the `bsub` command is available, which is typically the cluster login node.
To enable the LSF executor, set `process.executor = 'lsf'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
:::{note}
LSF supports both *per-core* and *per-job* memory limits. Nextflow assumes that LSF works in the *per-core* mode, thus it divides the requested {ref}`process-memory` by the number of requested {ref}`process-cpus`.
When LSF is configured to work in the *per-job* memory limit mode, you must specify this limit with the `perJobMemLimit` option in the {ref}`config-executor` scope of your Nextflow config file.
See also the [Platform LSF documentation](https://www.ibm.com/support/knowledgecenter/SSETD4_9.1.3/lsf_config_ref/lsf.conf.lsb_job_memlimit.5.dita).
:::
(moab-executor)=
## Moab
:::{versionadded} 19.07.0
:::
:::{warning} *Experimental: may change in a future release.*
:::
The `moab` executor allows you to run your pipeline script using the [Moab](https://en.wikipedia.org/wiki/Moab_Cluster_Suite) resource manager by [Adaptive Computing](http://www.adaptivecomputing.com/).
Nextflow manages each process as a separate job that is submitted to the cluster using the `msub` command provided by the resource manager.
The pipeline must be launched from a node where the `msub` command is available, which is typically the cluster login node.
To enable the `Moab` executor, set `process.executor = 'moab'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
(nqsii-executor)=
## NQSII
The `nqsii` executor allows you to run your pipeline script using the [NQSII](https://www.rz.uni-kiel.de/en/our-portfolio/hiperf/nec-linux-cluster) resource manager.
Nextflow manages each process as a separate job that is submitted to the cluster using the `qsub` command provided by the scheduler.
The pipeline must be launched from a node where the `qsub` command is available, which is typically the cluster login node.
To enable the NQSII executor, set `process.executor = 'nqsii'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
(oar-executor)=
## OAR
:::{versionadded} 19.11.0-edge
:::
The `oar` executor allows you to run your pipeline script using the [OAR](https://oar.imag.fr) resource manager.
Nextflow manages each process as a separate job that is submitted to the cluster using the `oarsub` command.
The pipeline must be launched from a node where the `oarsub` command is available, which is typically the cluster login node.
To enable the OAR executor set `process.executor = 'oar'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
When specifying `clusterOptions` as a string, multiple options must be separated by semicolons to ensure that the job script is formatted correctly:
```groovy
clusterOptions = '-t besteffort;--project myproject'
```
:::{versionadded} 24.04.0
:::
The same behavior can now be achieved using a string list:
```groovy
clusterOptions = [ '-t besteffort', '--project myproject' ]
```
See {ref}`process-clusteroptions` for details.
(pbs-executor)=
## PBS/Torque
The `pbs` executor allows you to run your pipeline script using a resource manager from the [PBS/Torque](http://en.wikipedia.org/wiki/Portable_Batch_System) family of batch schedulers.
Nextflow manages each process as a separate job that is submitted to the cluster using the `qsub` command provided by the scheduler.
The pipeline must be launched from a node where the `qsub` command is available, which is typically the cluster login node.
To enable the PBS executor, set `process.executor = 'pbs'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
(pbspro-executor)=
## PBS Pro
The `pbspro` executor allows you to run your pipeline script using the [PBS Pro](https://www.pbspro.org/) resource manager.
Nextflow manages each process as a separate job that is submitted to the cluster using the `qsub` command provided by the scheduler.
The pipeline must be launched from a node where the `qsub` command is available, which is typically the cluster login node.
To enable the PBS Pro executor, set `process.executor = 'pbspro'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
(sge-executor)=
## SGE
The `sge` executor allows you to run your pipeline script using a [Sun Grid Engine](http://en.wikipedia.org/wiki/Oracle_Grid_Engine) cluster or a compatible platform ([Open Grid Engine](http://gridscheduler.sourceforge.net/), [Univa Grid Engine](http://www.univa.com/products/grid-engine.php), etc).
Nextflow manages each process as a separate grid job that is submitted to the cluster using the `qsub` command.
The pipeline must be launched from a node where the `qsub` command is available, which is typically the cluster login node.
To enable the SGE executor, set `process.executor = 'sge'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-penv`
- {ref}`process-queue`
- {ref}`process-time`
(seqera-executor)=
## Seqera
:::{versionadded} 26.04.0
:::
:::{warning}
*Preview feature: may change in a future release.*
:::
The `seqera` executor allows you to run your pipeline using the [Seqera](https://seqera.io) cloud infrastructure. It enables the seamless execution of Nextflow pipelines by offloading process executions to the Seqera scheduler service.
The pipeline processes must specify the Docker image to use by defining the `container` directive, either in the pipeline script or the `nextflow.config` file. Additionally, an S3 bucket must be used as the pipeline work directory.
To enable this executor, set `process.executor = 'seqera'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-arch`
- {ref}`process-container`
- {ref}`process-containerOptions`
- {ref}`process-cpus`
- {ref}`process-disk`
- {ref}`process-memory`
- {ref}`process-time`
The following {ref}`hints <process-hints>` are supported:
- `machineRequirement.capacityMode`
- `machineRequirement.diskAllocation`
- `machineRequirement.diskEncrypted`
- `machineRequirement.diskIops`
- `machineRequirement.diskMountPath`
- `machineRequirement.diskSize`
- `machineRequirement.diskThroughputMiBps`
- `machineRequirement.diskType`
- `machineRequirement.machineTypes`
- `machineRequirement.maxSpotAttempts`
- `machineRequirement.provisioning`
Each hint overrides the corresponding field of the `seqera.executor.machineRequirement` config scope on a per-process basis. Keys may be used as-is or with the `seqera/` prefix to restrict them to this executor.
For example, to override the provisioning mode for a single process:
```nextflow
process hello {
hints 'seqera/machineRequirement.provisioning': 'spotFirst'
script:
"""
your_command --here
"""
}
```
See {ref}`config-seqera` for the full config reference.
### Disk support
When the {ref}`process-disk` directive is specified, the Seqera executor provisions storage for the task container. There are two disk allocation strategies:
- **task** (default): A dedicated EBS volume is created for each task at launch time. This provides isolated, high-performance storage with configurable volume type, IOPS, throughput, and encryption.
- **node**: Uses the instance storage attached at the cluster level. This is shared across tasks running on the same node and does not support EBS-specific options.
#### Task allocation (EBS volumes)
By default, a gp3 volume with 325 MiB/s throughput is used (Fusion recommended settings). You can customize the EBS volume configuration:
```groovy
seqera {
executor {
machineRequirement {
diskAllocation = 'task' // Per-task EBS volume (default)
diskType = 'ebs/io1' // Use provisioned IOPS SSD
diskIops = 10000 // Required for io1/io2
diskThroughputMiBps = 500 // Throughput for gp3 volumes
diskEncrypted = true // Enable KMS encryption
diskMountPath = '/data' // Container mount path (default: /tmp)
}
}
}
```
Supported volume types: `ebs/gp3` (default), `ebs/gp2`, `ebs/io1`, `ebs/io2`, `ebs/st1`, `ebs/sc1`.
#### Node allocation (instance storage)
To use instance storage instead of per-task EBS volumes:
```groovy
seqera {
executor {
machineRequirement {
diskAllocation = 'node' // Use instance storage
}
}
}
```
:::{note}
When using `node` allocation, the EBS-specific options (`diskType`, `diskIops`, `diskThroughputMiBps`, `diskEncrypted`) are not applicable and will cause an error if specified.
:::
See the {ref}`seqera scope <config-seqera>` for the available configuration options.
(slurm-executor)=
## SLURM
The `slurm` executor allows you to run your pipeline script using the [SLURM](https://slurm.schedmd.com/documentation.html) resource manager.
Nextflow manages each process as a separate job that is submitted to the cluster using the `sbatch` command.
The pipeline must be launched from a node where the `sbatch` command is available, which is typically the cluster login node.
To enable the SLURM executor, set `process.executor = 'slurm'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-cpus`
- {ref}`process-memory`
- {ref}`process-queue`
- {ref}`process-time`
:::{note}
SLURM partitions can be specified with the `queue` directive.
:::
:::{note}
Nextflow does not provide direct support for SLURM multi-clusters. If you need to submit workflow executions to a cluster other than the current one, specify it with the `SLURM_CLUSTERS` variable in the launch environment.
:::
:::{versionadded} 23.07.0-edge
Some SLURM clusters require memory allocations to be specified with `--mem-per-cpu` instead of `--mem`. You can specify `executor.perCpuMemAllocation = true` in the Nextflow configuration to enable this behavior. Nextflow will automatically compute the memory per CPU for each task (by default 1 CPU is used).
:::
:::{versionadded} 25.12.0-edge
Since SLURM 24, `squeue` supports an `--only-job-state` option that ignores the partition (`-p`) or user (`-u`) filters. To enable this behavior, specify `executor.$slurm.onlyJobState = true` in your Nextflow configuration. If `SchedulerParameters=enable_job_state_cache` is enabled, you can expect improved Nextflow performance and reduced load on the SLURM controller. See [`enable_job_state_cache`](https://slurm.schedmd.com/slurm.conf.html#OPT_enable_job_state_cache) and [`--only-job-state`](https://slurm.schedmd.com/squeue.html#OPT_only-job-state) for more information.
:::
(tcs-executor)=
## TCS
The `tcs` executor allows you to run your pipeline script using a [Fujitsu Technical Computing Suite (TCS)](https://software.fujitsu.com/jp/manual/manualindex/p21000155e.html).
Nextflow manages each process as a separate job that is submitted to the cluster using the `pjsub` command.
The pipeline must be launched from a node where the `pjsub` command is available, which is typically the login node.
To enable the TCS executor, set `process.executor = 'tcs'` in the `nextflow.config` file.
Resource requests and other job characteristics can be controlled via the following process directives:
- {ref}`process-clusterOptions`
- {ref}`process-time`
:::{note}
Use `clusterOptions` to specify system-dependent options such as queue (resource group), CPU, and node. These options vary across target systems and are not standardized. They correspond to `-L` options in the arguments of the `pjsub` command and should be configured according to the requirements of the specific cluster environment.
For example:
```groovy
process {
executor = 'tcs'
time = '00:30:00'
clusterOptions = '-L rscgrp=a-batch -L vnode-core=4'
}
```
:::

37
nextflow/docs/fusion.md Normal file
View File

@@ -0,0 +1,37 @@
(fusion-page)=
# Fusion file system
:::{versionadded} 22.10.0
:::
:::{versionadded} 23.02.0-edge
Support for Google Cloud Storage.
:::
Cloud object stores such as AWS S3 are scalable and cost-effective, but they don't present a POSIX (Portable Operating System Interface). This means containerized applications must copy data to and from cloud storage for every task — a slow and inefficient process.
Fusion is a virtual, lightweight, distributed file system that bridges the gap between pipelines and cloud-native storage. Fusion enables seamless filesystem I/O to cloud object stores via a standard POSIX interface, resulting in simpler pipeline logic and faster, more efficient pipeline execution.
See [Fusion file system](https://docs.seqera.io/fusion) for more information about Fusion features.
:::{note}
Fusion requires a license for use in Seqera Platform compute environments or directly in Nextflow. Fusion can be trialed at no cost. See the [Fusion licensing documentation](https://docs.seqera.io/fusion/licensing) for more information.
:::
## Get started
Fusion integrates with Nextflow directly and does not require any installation or change in pipeline code. It only requires use of a container runtime or a container computing service, such as Kubernetes, AWS Batch, or Google Cloud Batch.
To enable Fusion in your Nextflow pipeline, add the following to your `nextflow.config` file:
```{code-block} nextflow
:class: copyable
fusion.enabled = true
wave.enabled = true
tower.accessToken = '<PLATFORM_ACCESS_TOKEN>'
```
Replace `<PLATFORM_ACCESS_TOKEN>` with your Platform access token.
See [Get started](https://docs.seqera.io/fusion/get-started) for more information and guides to get started with Fusion.

245
nextflow/docs/git.md Normal file
View File

@@ -0,0 +1,245 @@
(git-page)=
# Git
## Git configuration
The file `$HOME/.nextflow/scm` allows you to centralize the security credentials required to access private project repositories on Bitbucket, GitHub and GitLab source code management (SCM) platforms or to manage the configuration properties of private server installations (of the same platforms).
The configuration properties for each Git provider are defined inside the `providers` section. Properties for the same provider are grouped with a common name and delimited with curly brackets. For example:
```groovy
providers {
<provider-name> {
property = value
// ...
}
}
```
In the above template replace `<provider-name>` with one of the "default" servers (i.e. `bitbucket`, `github` or `gitlab`) or a custom identifier representing a private SCM server installation.
:::{versionadded} 20.10.0
A custom location for the SCM file can be specified using the `NXF_SCM_FILE` environment variable.
:::
The following configuration properties are supported for each provider configuration:
`providers.<provider>.user`
: User name required to access private repositories on the SCM server.
`providers.<provider>.password`
: User password required to access private repositories on the SCM server.
`providers.<provider>.token`
: *Required only for private Gitlab servers and supported for BitBucket*
: Private API access token.
`providers.<provider>.platform`
: *Required only for private SCM servers*
: Git provider name, either: `github`, `gitlab` or `bitbucket`.
`providers.<provider>.server`
: *Required only for private SCM servers*
: SCM server name including the protocol prefix e.g. `https://github.com`.
`providers.<provider>.endpoint`
: *Required only for private SCM servers*
: SCM API `endpoint` URL e.g. `https://api.github.com` (default: the same as `providers.<provider>.server`).
## Git providers
### Bitbucket
Create a `bitbucket` entry in the [SCM configuration file](#git-configuration) specifying your user name and either an API token or app password, as shown below:
```groovy
providers {
bitbucket {
user = 'me'
token = 'my-api-token'
}
}
```
:::{versionadded} 25.07.0-edge
API tokens are supported for BitBucket authentication. When both `token` and `password` are provided, the API token takes priority over the app password.
:::
:::{note}
API tokens are substitute passwords for a user account which you can use for scripts and integrating tools in order to avoid putting your real password into configuration files. Learn more about [app passwords](https://support.atlassian.com/bitbucket-cloud/docs/app-passwords/) and [API tokens](https://support.atlassian.com/bitbucket-cloud/docs/using-api-tokens/).
:::
### Bitbucket Server
[Bitbucket Server](https://confluence.atlassian.com/bitbucketserver) is a self-hosted Git repository and management platform.
:::{note}
Bitbucket Server uses a different API from the [Bitbucket](https://bitbucket.org/) Cloud service. Make sure to use the right configuration whether you are using the cloud service or a self-hosted installation.
:::
To access your local Bitbucket Server create an entry in the [SCM configuration file](#git-configuration) as shown below:
```groovy
providers {
mybitbucket {
platform = 'bitbucketserver'
server = 'https://your.bitbucket.host.com'
endpoint = 'https://your.bitbucket.host.com'
user = 'your-user'
password = 'your-password or your-token'
}
}
```
### GitHub
Create a `github` entry in the [SCM configuration file](#git-configuration) specifying your user name and access token as shown below:
```groovy
providers {
github {
user = 'your-user-name'
password = 'your-personal-access-token'
}
}
```
GitHub requires the use of a personal access token (PAT) in place of a password when accessing APIs. Learn more about PAT and how to create it at [this link](https://docs.github.com/en/github/authenticating-to-github/keeping-your-account-and-data-secure/creating-a-personal-access-token).
:::{versionadded} 23.01.0-edge
Nextflow automatically uses the `GITHUB_TOKEN` environment variable to authenticate access to the GitHub repository if no credentials are provided via the `scm` file. This is useful especially when accessing pipeline code from a GitHub Action. Read more about the token authentication in the [GitHub documentation](https://docs.github.com/en/actions/security-guides/automatic-token-authentication).
:::
### GitLab
Create a `gitlab` entry in the [SCM configuration file](#git-configuration) specifying the user name, password and your API access token that can be found in your GitLab [account page](https://gitlab.com/profile/account) (sign in required). For example:
```groovy
providers {
gitlab {
user = 'me'
password = 'my-secret'
token = 'YgpR8m7viH_ZYnC8YSe8'
}
}
```
:::{tip}
The GitLab *token* string can be used as the `password` value in the above setting. When doing that the `token` field can be omitted.
:::
### Gitea
[Gitea](https://gitea.io) is a Git repository server with GitHub-like GUI access. Since Gitea installation is quite easy, it is suitable for building a private development environment in your network. To access your Gitea server, you have to provide all the credential information below:
```groovy
providers {
mygitea {
server = 'http://your-domain.org/gitea'
endpoint = 'http://your-domain.org/gitea/api/v1'
platform = 'gitea'
user = 'your-user'
password = 'your-password'
token = 'your-api-token'
}
}
```
See [Gitea documentation](https://docs.gitea.io/en-us/api-usage/) about how to enable API access on your server and how to issue a token.
### Azure Repos
Nextflow has built-in support for [Azure Repos](https://azure.microsoft.com/en-us/services/devops/repos/), a Git source code management service hosted in the Azure cloud. To access your Azure Repos with Nextflow provide the repository credentials using the configuration snippet shown below:
```groovy
providers {
azurerepos {
user = 'your-user-name'
password = 'your-personal-access-token'
}
}
```
:::{tip}
The Personal access token can be generated in the repository `Clone Repository` dialog.
:::
(aws-codecommit)=
### AWS CodeCommit
:::{versionadded} 22.06.0-edge
:::
Nextflow supports [AWS CodeCommit](https://aws.amazon.com/codecommit/) as a Git provider to access and to share pipelines code.
To access your project hosted on AWS CodeCommit with Nextflow provide the repository credentials using the configuration snippet shown below:
```groovy
providers {
my_aws_repo {
platform = 'codecommit'
user = '<AWS ACCESS KEY>'
password = '<AWS SECRET KEY>'
}
}
```
In the above snippet replace `<AWS ACCESS KEY>` and `<AWS SECRET KEY>` with your AWS credentials, and `my_aws_repo` with a name of your choice.
:::{tip}
The `user` and `password` settings are optional. If omitted, the [AWS default credentials provider chain](https://docs.aws.amazon.com/sdk-for-java/v1/developer-guide/credentials.html) is used.
:::
Then the pipeline can be accessed with Nextflow as shown below:
```bash
nextflow run https://git-codecommit.eu-west-1.amazonaws.com/v1/repos/my-repo
```
In the above example replace `my-repo` with your own repository. Note also that AWS CodeCommit has different URLs depending on the region in which you are working.
:::{note}
The support for protocols other than HTTPS is not available at this time.
:::
## Private server configuration
Nextflow is able to access repositories hosted on private Bitbucket, GitHub, GitLab and Gitea server installations.
In order to use a private SCM installation you will need to set the server name and access credentials in your [SCM configuration file](#git-configuration) .
If, for example, the host name of your private GitLab server is `gitlab.acme.org`, you will need to have in the `$HOME/.nextflow/scm` file a configuration like the following:
```groovy
providers {
mygit {
server = 'http://gitlab.acme.org'
platform = 'gitlab'
user = 'your-user'
password = 'your-password'
token = 'your-api-token'
}
}
```
Then you will be able to run/pull a project with Nextflow using the following command line:
```bash
nextflow run acme/hello -hub mygit
```
Or, alternatively, using the Git clone URL:
```bash
nextflow run http://gitlab.acme.org/acme/hello.git
```
:::{note}
You must also specify the server API endpoint URL if it differs from the server base URL. For example, for GitHub Enterprise V3, add `endpoint = 'https://git.your-domain.com/api/v3'`.
:::
:::{warning}
When accessing a private SCM installation over `https` from a server that uses a custom SSL certificate, you may need to import the certificate into your local Java keystore. See [Import the Certificate as a Trusted Certificate](https://docs.oracle.com/javase/tutorial/security/toolsign/rstep2.html) for more information.
:::

Some files were not shown because too many files have changed in this diff Show More