Files
ma/nextflow/adr/20251114-module-system.md
2026-04-29 23:01:54 +02:00

931 lines
32 KiB
Markdown

# Module System for Nextflow
- Authors: Paolo Di Tommaso
- Status: approved
- Date: 2025-01-06
- Tags: modules, dsl, registry, versioning, architecture
- Version: 2.7
## Updates
### Version 2.7 (2026-03-09)
- **Renamed `.checksum` to `.module-info`**: Leaves room for additional properties in the future
- **Removed `@` prefix from module scopes**: Local modules are distinguished from remote modules by presence/absence of `./` prefix
- **Removed version pinning from config**: Installed module versions are now inferred from the `meta.yml` of each module in the `modules/` directory instead of being declared in `nextflow.config`
### Version 2.6 (2026-01-28)
- **Removed module parameters**: Module parameters specification moved to separate spec document.
### Version 2.5 (2026-01-23)
- **Module parameters**: Replaced structured tool arguments with general module parameters defined in `meta.yml`
- **Simplified tools section**: Removed `args` property from tools; tool arguments now configured via module parameters
- **Simplified `requires` block**: Removed `plugins`, `modules`, and `subworkflows` sub-properties; `requires` now only contains `nextflow` version constraint
- **Process modules focus**: Removed sub-workflow references; spec is now focused on process modules only
### Version 2.4 (2026-01-15)
- **Removed transitive dependency resolution**: Module dependencies are explicit only; no automatic transitive resolution
- **Removed `freeze` command**: No longer needed without transitive dependency management
- **Simplified model**: Each module explicitly declares its dependencies in `nextflow.config`
### Version 2.3 (2026-01-15)
- **Resolution Rules table**: Added clear table specifying behavior for each combination of local state and declared version
- **Local modification protection**: Locally modified modules (checksum mismatch) are NOT overridden unless `-force` flag is used
- **Simplified storage model**: Single version per module locally (`modules/@scope/name/` without version in path)
- **`.checksum` file**: Registry checksum cached locally for fast integrity verification without network calls
### Version 2.2 (2025-01-06) — *Superseded by v2.5*
- **Structured tool arguments**: Added `args` property to `tools` section for type-safe argument configuration
- **New implicit variables**: `tools.<toolname>.args.<argname>` returns formatted flag+value; `tools.<toolname>.args` returns all args concatenated
- **Deprecation**: All `ext.*` custom directives (e.g., `ext.args`, `ext.args2`, `ext.args3`, `ext.prefix`, `ext.suffix`) deprecated in favor of structured tool arguments
- *Note: Tool arguments replaced by module parameters in v2.5*
### Version 2.1 (2024-12-11)
- **Unified dependencies**: Consolidated `components`, `dependencies`, and `requires` into single `requires` field
- **Unified version syntax**: `[scope/]name[@constraint]` format across plugins and modules
- **Deprecation**: `components` field deprecated (use top-level `modules` instead)
## Context and Problem Statement
Nextflow supports local script inclusion via `include` directive but lacks standardized mechanisms for package management, versioning, and distribution of reusable process definitions. This limits code reuse and reproducibility across the ecosystem.
Discussion/request goes back to at least 2019, see GitHub issues [#1376](https://github.com/nextflow-io/nextflow/issues/1376), [#1463](https://github.com/nextflow-io/nextflow/issues/1463) and [#4122](https://github.com/nextflow-io/nextflow/issues/4112).
## Decision
Implement a module system with four core capabilities:
1. **Remote module inclusion** via registry
2. **Semantic versioning** with dependency resolution
3. **Unified Nextflow Registry** (rebrand existing Nextflow registry)
4. **First-class CLI support** (install, publish, search, list, remove, run)
## Core Capabilities
### 1. Remote Module Inclusion
**DSL Syntax**:
```groovy
// Include from registry (scoped module name without `./` prefix)
include { BWA_ALIGN } from 'nf-core/bwa-align'
// Existing file-based includes remain supported
include { MY_PROCESS } from './modules/my-process.nf'
```
**Module Naming**: Scoped modules `scope/name` (e.g., `nf-core/salmon`, `myorg/custom`). Local paths supported for backwards compatibility. No nested paths with the module are allowed - each module must have a `main.nf` as the entry point.
**Version Resolution**: Installed module versions are inferred from the `meta.yml` of each module in the `modules/` directory. If a module is not present locally, the latest available version is downloaded from the registry.
**Resolution Order**:
1. Check local `modules/scope/name/` exists
2. Verify integrity against `.module-info` file
3. Apply resolution rules (see below)
**Resolution Rules**:
| Local State | Action |
|-------------|--------|
| Missing | Download latest from registry |
| Exists, checksum valid | Use local module (version from `meta.yml`) |
| Exists, checksum mismatch | **Warn**: locally modified, will NOT replace unless `-force` is used |
**Key Behaviors**:
- **Local modification**: When the local module content was manually changed (checksum mismatch with `.module-info`), Nextflow warns and does NOT override to prevent accidental loss of local changes
- **Force flag**: Use `-force` with `nextflow module install` to override locally modified modules
**Resolution Timing**: Modules resolved at workflow parse time (after plugin resolution at startup).
**Local Storage**: Downloaded modules stored in `modules/scope/name/` directory in project root (not global cache). Each module must contain a `main.nf` file as the required entry point. It is intended that module source code will be committed to the pipeline git repository.
### 2. Semantic Versioning and Configuration
**Version Format**: MAJOR.MINOR.PATCH
- **MAJOR**: Breaking changes to process signatures, inputs, or outputs
- **MINOR**: New processes, backward-compatible enhancements
- **PATCH**: Bug fixes, documentation updates
**Registry Configuration** (`nextflow.config`):
```groovy
registry {
url = 'https://registry.nextflow.io' // Default registry
// allow the use of multiple registry url for resolving module
// across custom registries, e.g.
// url = [ 'https://custom.registry.com', 'https://registry.nextflow.io' ]
auth {
'registry.nextflow.io' = '${NXF_REGISTRY_TOKEN}'
'npm.myorg.com' = '${MYORG_TOKEN}'
}
}
```
**Module Spec** (`meta.yml`):
```yaml
name: nf-core/bwa-align
version: 1.2.4 # This module's version
requires:
nextflow: ">=24.04.0"
```
**Version Constraints** (unified `name@constraint` syntax):
- `name`: Any version (latest)
- `name@1.2.3`: Exact version
- `name@>=1.2.3`: Greater or equal
- `name@>=1.2.3,<2.0.0`: Range (comma-separated)
**Version Notation Consistency**:
Modules use the same version constraint syntax already supported by both `nextflowVersion` and plugins:
| Notation | Meaning | nextflowVersion | Plugins | Modules |
| :---- | :---- | :---- | :---- | :---- |
| 1.2.3 | Exact version | ✓ | ✓ | ✓ |
| >=1.2.3 | Greater or equal | ✓ | ✓ | ✓ |
| <=1.2.3 | Less or equal | ✓ | ✓ | ✓ |
| >1.2.3 | Greater than | ✓ | ✓ | ✓ |
| <1.2.3 | Less than | ✓ | ✓ | ✓ |
| >=1.2, <2.0 | Range (comma) | ✓ | ✓ | ✓ |
| !=1.2.3 | Not equal | ✓ | - | - |
| 1.2+ | >=1.2.x <2.0 | ✓ | - | - |
| 1.2.+ | >=1.2.0 <1.3.0 | ✓ | - | - |
| ~1.2.3 | >=1.2.3 <1.3.0 | - | ✓ | - |
Using comparison operators (`>=`, `<`) with comma-separated ranges provides the same expressive power as
npm-style `^` and `~` notation while maintaining consistency with existing Nextflow version constraint syntax.
This avoids introducing new notation that would require additional parser support.
**Module Resolution**:
Installed module versions are inferred from the `meta.yml` file for each module in the `modules/` directory.
### 3. Unified Nextflow Registry
**Architecture Decision**: Extend existing Nextflow registry at `registry.nextflow.io` to host both plugins and modules.
**Current Plugin API** (reference: https://registry.nextflow.io/openapi/):
```
GET /api/v1/plugins # List/search plugins
GET /api/v1/plugins/{pluginId} # Get plugin + all releases
GET /api/v1/plugins/{pluginId}/{version} # Get specific release
GET /api/v1/plugins/{pluginId}/{version}/download/{fileName} # Download artifact
POST /api/v1/plugins/release # Create draft release
POST /api/v1/plugins/release/{releaseId}/upload # Upload artifact
```
**Module API** (reference: https://github.com/seqeralabs/plugin-registry/pull/266):
```
GET /api/modules?query=<text> # Search modules (semantic search)
GET /api/modules/{name} # Get module + latest release
GET /api/modules/{name}/releases # List all releases
GET /api/modules/{name}/{version} # Get specific release
GET /api/modules/{name}/{version}/download # Download module bundle
POST /api/modules/{name} # Publish module version (authenticated)
```
Note: The `{name}` parameter includes the namespace prefix (e.g., "nf-core/fastqc").
**Registry URL**: `registry.nextflow.io`
**Artifact Types**:
- **Plugins**: JAR files with JSON metadata, resolved at startup
- **Modules**: Source archives (.nf + meta.yml), resolved at parse time
**Benefits**:
- Reuses existing infrastructure (HTTP service, S3 storage, authentication)
- Consistent API patterns for both artifact types
- Operational simplicity (one service vs. two)
- Internal module API already partially implemented
### 4. First-Class CLI Support
**Commands**:
```bash
nextflow module run scope/name # Run a module directly without a wrapper script
nextflow module search <query> # Search registry
nextflow module install scope/name # Install a module
nextflow module list # Show installed vs configured
nextflow module remove scope/name # Remove from config + local cache
nextflow module publish scope/name # Publish to registry (requires api key)
```
**General Notes**:
- All commands respect the `registry.url` configuration for custom registries
#### `nextflow module run scope/name`
Run a module directly without requiring a wrapper workflow script. This command enables standalone execution of any module by automatically mapping command-line arguments to the module's process inputs. If the module is not available locally, it is automatically installed before execution.
**Arguments**:
- `scope/name`: Module identifier to run (required)
**Options**:
- `-version <ver>`: Run a specific version (default: latest or configured version)
- `--<input_name> <value>`: Map value to the corresponding module process input channel
- All standard `nextflow run` options (e.g., `-profile`, `-work-dir`, `-resume`, etc.)
**Behavior**:
1. Checks if module is installed locally; if not, downloads from registry
2. Parses the module's `main.nf` to identify the main process and its input declarations
3. Validates command-line arguments against the process input declarations
4. Generates an implicit workflow that wires CLI arguments to process inputs
5. Executes the workflow using standard Nextflow runtime
**Input Mapping**:
- Named arguments (`--reads`, `--reference`) are mapped to corresponding process inputs
- File paths are automatically converted to files for process file inputs
- Multiple values can be provided for inputs expecting collections
- Required inputs without defaults must be provided; optional inputs use declared defaults
**Example**:
```bash
# Run BWA alignment module with input files
nextflow module run nf-core/bwa-align \
--reads 'samples/*_{1,2}.fastq.gz' \
--reference genome.fa
# Run a specific version with Nextflow options
nextflow module run nf-core/fastqc -version 1.0.0 \
--input 'data/*.fastq.gz' \
-profile docker \
-resume
# Run with work directory and output specification
nextflow module run nf-core/salmon \
--reads reads.fq \
--index salmon_index \
-work-dir /tmp/work \
-output-dir results/
```
---
#### `nextflow module search <query>`
Search the Nextflow registry for available modules matching the specified query. The search operates against module names, descriptions, tags, and author information. Results are displayed with module name, latest version, description, and download statistics.
**Arguments**:
- `<query>`: Search term (required) - matches against module metadata
**Options**:
- `-limit <n>`: Maximum number of results to return (default: 10)
- `-json`: Output results in JSON format for programmatic use
**Example**:
```bash
nextflow module search bwa
nextflow module search "alignment" -limit 50
```
---
#### `nextflow module install <scope/name>`
Download and install a module to the local `modules/` directory.
**Arguments**:
- `<scope/name>`: Module identifier.
**Options**:
- `-version <ver>`: Install a specific version (default: latest)
- `-force`: Overwrite any local changes
**Behavior**:
1. If `-version` not specified, queries registry for the latest available version
2. Checks if local module exists and verifies integrity against `.module-info` file
3. If local module is unmodified and version differs: replaces with requested version
4. If local module was modified (checksum mismatch): warns and aborts unless `-force` is used
5. Downloads the module archive from the registry
6. Extracts to `modules/scope/name/` directory
7. Stores `.module-info` file from registry's X-Checksum response header
**Example**:
```bash
nextflow module install nf-core/bwa-align # Install specific module (latest)
nextflow module install nf-core/salmon -version 1.2.0
```
---
#### `nextflow module list`
Display the status of all modules, comparing what is configured in `nextflow.config` against what is actually installed in the `modules/` directory.
**Options**:
- `-json`: Output in JSON format
- `-outdated`: Only show modules with available updates
**Output columns**:
- Module name (`scope/name`)
- Installed version (from `modules/scope/name/meta.yml`)
- Latest available version (from registry)
- Status indicator (up-to-date, outdated, missing)
**Example**:
```bash
nextflow module list
nextflow module list -outdated
```
---
#### `nextflow module remove scope/name`
Remove a module from the local `modules/` directory.
**Arguments**:
- `scope/name`: Module identifier to remove (required)
**Options**:
- `-keep-files`: Remove `.module-info` file but keep local files
**Behavior**:
1. Removes the module directory from `modules/scope/name/`
**Example**:
```bash
nextflow module remove nf-core/bwa-align
nextflow module remove myorg/custom -keep-files
```
---
#### `nextflow module publish scope/name`
Publish a module to the Nextflow registry, making it available for others to install. Requires authentication via API key and appropriate permissions for the target scope.
**Arguments**:
- `scope/name`: Module identifier to publish (required)
**Options**:
- `-registry <url>`: Target registry URL (default: `registry.nextflow.io`)
- `-tag <tag>`: Additional tags for discoverability
- `-dry-run`: Validate without publishing
**Behavior**:
1. Validates `meta.yml` schema and required fields (name, version, description)
2. Verifies that `main.nf` exists and is valid Nextflow syntax
3. Verifies that `README.md` documentation is present
4. Authenticates with registry using configured credentials
5. Creates a release draft and uploads the module archive
6. Publishes the release, making it available for installation
**Requirements**:
- Valid `meta.yml` with name, version, and description
- `main.nf` entry point file
- `README.md` documentation
- Authentication token configured in `registry.auth` or `NXF_REGISTRY_TOKEN`
- Write permission for the target scope
**Example**:
```bash
nextflow module publish myorg/my-process
nextflow module publish myorg/my-process -dry-run
```
## Module Structure
**Directory Layout**:
Everything within the module directory should be uploaded. Module bundle should not exceed 1MB (uncompressed). Typically this is expected to look something like this:
```
my-module/
├── main.nf # Required: entry point for module
├── meta.yml # Required: Module spec (version, metadata, I/O specs)
├── README.md # Required: Module description
└── tests/ # Optional tests
```
**Module Spec extension** (`meta.yml`):
```yaml
name: nf-core/bwa-align
version: 1.2.4 # This module's version
description: Align reads using BWA-MEM
authors:
- nf-core community
license: MIT
requires:
nextflow: ">=24.04.0"
```
**Local Storage Structure**:
```
project-root/
├── nextflow.config
├── main.nf
└── modules/ # Local module cache
├── nf-core/
│ ├── bwa-align/
│ │ ├── .module-info # Cached registry checksum
│ │ ├── meta.yml
│ │ └── main.nf # Required entry point
│ └── samtools/view/
│ ├── .module-info
│ ├── meta.yml
│ └── main.nf # Required entry point
└── myorg/
└── custom-process/
├── .module-info
├── meta.yml
└── main.nf # Required entry point
```
**Module Integrity Verification**:
- On install: `.module-info` file created from registry's X-Checksum response header
- On run: Local module checksum compared against `.module-info` file
- If match: Proceed without network call
- If mismatch: Report warning (module may have been locally modified)
## Implementation Strategy
**Phase 1**: Module schema, local module loading, validation tools
**Phase 2**: Extend Nextflow registry for modules, implement caching, add `install` and `search` commands
**Phase 3**: Extend DSL parser for `from module` syntax
**Phase 4**: Implement `publish` command with authentication and `run` command
**Phase 5**: Advanced features (search UI, language server integration, ontology validation)
## Technical Details
**Module Resolution Flow**:
1. Parse `include` statements → extract module names (e.g., `nf-core/bwa-align`)
2. For each module:
a. Check local `modules/scope/name/` exists
- If exists → read installed version from `modules/scope/name/meta.yml`
- If missing → download latest version from registry
b. Verify local module integrity against `.module-info` file
- Checksum mismatch → warn and do NOT override (local changes detected)
3. On download: store module to `modules/scope/name/` with `.module-info` file
4. Read `meta.yml` file: Validates Nextflow requirement → Fail if not fulfilled
5. Parse module's `main.nf` file → make processes available
**Security**:
- SHA-256 checksum verification on download (stored in `.module-info` file)
- Integrity verification on run (local checksum vs `.module-info` file)
- Authentication required for publishing
- Support for private registries
**Integration with Plugin System**:
- Both plugins and modules query same registry
- Single authentication system
- Separate cache locations: `$NXF_HOME/plugins/` (global) vs `modules/` (per-project)
## Comparison: Plugins vs. Modules
| Aspect | Plugins | Modules |
|--------|---------|---------|
| Purpose | Extend runtime | Reusable processes |
| Format | JAR files | Source code (.nf) |
| Resolution | Startup | Parse time |
| Metadata | JSON spec | YAML spec |
| Naming | `nf-amazon` | `nf-core/salmon` |
| Cache Location | `$NXF_HOME/plugins/` | `modules/scope/name/` |
| Version Config | `plugins {}` in config | `meta.yml` in `modules/` directory |
| Registry Path | `/api/v1/plugins/` | `/api/modules/{name}` |
## Rationale
**Why unified registry?**
- Reuses battle-tested infrastructure (HTTP API, S3, auth)
- Single discovery experience for ecosystem
- Lower operational overhead
- Type-specific handling maintains separation of concerns
**Why infer versions from `meta.yml` instead of pinning in a separate file?**
- Simple: install a version once and it is captured in the module files
- Reproducibility via committing the `modules/` directory (including `meta.yml`) to the project git repository
- Reduces configuration burden: no need to keep config in sync with installed state
**Why parse-time resolution?**
- Modules are source code, not compiled artifacts
- Allows inspection/modification for reproducibility
- Enables dependency analysis before execution
**Why scoped modules?**
- Organization namespacing prevents name collisions (`nf-core/salmon` vs `myorg/salmon`)
- Clear ownership and provenance of modules
- Supports private registries per scope
- Industry-standard pattern (NPM, Terraform, others)
- Enables ecosystem organization by maintainer/organization
**Why semantic versioning?**
- Clear compatibility guarantees
- Industry standard (npm, cargo, Go modules)
## Consequences
**Positive**:
- Enables ecosystem-wide code reuse
- Reproducible workflows via committing the `modules/` directory (including `meta.yml`) to the project git repository
- Centralized discovery and distribution via unified registry
- Minimal operational overhead (single registry for both plugins and modules)
- Module scoping enables organization namespaces and private registries
- Local `modules/` directory provides project isolation
- No version duplication: installed `meta.yml` is the single source of truth
- Simple module structure: each module has single `main.nf` entry point
**Negative**:
- Registry becomes critical infrastructure (requires HA setup)
- Type-specific handling adds registry complexity
- Parse-time resolution adds latency to workflow startup
- Local `modules/` directory duplicates storage across projects (unlike global cache)
**Neutral**:
- Modules and plugins conceptually distinct but share infrastructure
- Different resolution timing supported by same API
## Links
- Related: [Plugin Spec ADR](20250922-plugin-spec.md)
- Inspired by: [Go Modules](https://go.dev/ref/mod), [npm](https://docs.npmjs.com), [Cargo](https://doc.rust-lang.org/cargo/)
- Related: [nf-core modules](https://nf-co.re/modules)
---
## Appendix A: Module Schema Specification
This appendix defines the JSON schema for module `meta.yml` files. The schema maintains backward compatibility with existing nf-core module metadata patterns while supporting the new Nextflow module system features.
**Schema File:** [module-spec-schema.json](module-spec-schema.json)
**Published URL:** `https://registry.nextflow.io/schemas/module-spec/v1.0.0`
### Field Reference
#### Core Fields (Existing nf-core Pattern)
These fields are already widely adopted in the nf-core community and remain fully supported:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `name` | string | Yes | Module identifier |
| `description` | string | Yes | Brief description of module functionality |
| `keywords` | array[string] | Recommended | Discovery and categorization keywords |
| `authors` | array[string] | Recommended | Original authors (GitHub handles) |
| `maintainers` | array[string] | Recommended | Current maintainers |
| `tools` | array[object] | Conditional | Software tools wrapped by the module |
| `input` | array/object | Recommended | Input channel specifications |
| `output` | object/array | Recommended | Output channel specifications |
#### Extension Fields (Nextflow Module System)
These fields extend the schema to support the new Nextflow module system:
| Field | Type | Required | Description |
|-------|------|----------|-------------|
| `version` | string | Registry | Semantic version (MAJOR.MINOR.PATCH) |
| `license` | string | Registry | SPDX license identifier for module code |
| `requires` | object | Optional | Runtime requirements |
| `requires.nextflow` | string | Optional | Nextflow version constraint |
### Detailed Field Specifications
#### `name`
The module name must be a fully qualified scoped identifier in `scope/name` format:
```yaml
name: nf-core/fastqc
name: nf-core/bwa-mem
name: myorg/custom-aligner
```
**Naming Rules:**
- Format: `scope/name` (e.g., `nf-core/salmon`, `myorg/custom`)
- Scope: lowercase alphanumeric with hyphens (organization/owner identifier)
- Name: lowercase alphanumeric with underscores/hyphens (module identifier)
- Pattern: `^[a-z0-9][a-z0-9-]*/[a-z][a-z0-9_-]*$`
#### `version`
Semantic version following [SemVer 2.0.0](https://semver.org/):
```yaml
version: "1.0.0"
version: "2.3.1"
version: "1.0.0-beta.1"
```
**Version Semantics:**
- **MAJOR:** Breaking changes to process signatures, inputs, or outputs
- **MINOR:** New processes, backward-compatible enhancements
- **PATCH:** Bug fixes, documentation updates
**Requirement:** Mandatory for registry-published modules (scoped names in `scope/name` format).
#### `requires`
Specifies runtime requirements for the module.
```yaml
requires:
nextflow: ">=24.04.0"
```
**`requires.nextflow`** - Nextflow version constraint:
```yaml
requires:
nextflow: ">=24.04.0" # minimum version
nextflow: ">=24.04.0,<25.0.0" # version range
```
#### `tools`
Documents the software tools wrapped by the module:
```yaml
tools:
- bwa:
description: BWA aligner
homepage: http://bio-bwa.sourceforge.net/
license: ["GPL-3.0-or-later"]
identifier: biotools:bwa
```
**Tool Properties:**
| Property | Required | Description |
|----------|----------|-------------|
| `description` | Yes | Tool description |
| `homepage` | One of these | Tool homepage URL |
| `documentation` | One of these | Documentation URL |
| `tool_dev_url` | One of these | Development/source URL |
| `doi` | One of these | Publication DOI |
| `arxiv` | No | arXiv identifier |
| `license` | Recommended | SPDX license(s) |
| `identifier` | Recommended | bio.tools identifier |
| `manual` | No | User manual URL |
#### `input` and `output`
The schema supports a simplified syntax for inputs and outputs. Tooling such as the `module` CLI and module registry supports both the module spec and nf-core syntax for backwards compatibility:
**Module Spec:**
```yaml
input:
- - name: meta
type: map
description: Sample metadata
- name: reads
type: file
description: Input FastQ files
ontologies:
- edam: "http://edamontology.org/format_1930"
- name: index
type: directory
description: Reference index
output:
- - name: meta
type: map
description: Sample metadata
- name: bam
type: file
description: Aligned BAM file
pattern: "*.bam"
- name: versions
type: file
description: Software versions
```
**nf-core module (legacy):**
```yaml
input:
- - meta:
type: map
description: Sample metadata
- reads:
type: file
description: Input FastQ files
ontologies:
- edam: "http://edamontology.org/format_1930"
- - index:
type: directory
description: Reference index
output:
bam:
- - meta:
type: map
description: Sample metadata
- "*.bam":
type: file
description: Aligned BAM file
pattern: "*.bam"
versions:
- versions.yml:
type: file
description: Software versions
```
**Module Input/Output Properties:**
| Property | Type | Description |
|----------|------|-------------|
| `type` | string | Data type: `map`, `file`, `directory`, `string`, `integer`, `float`, `boolean`, `list`, `val` |
| `description` | string | Human-readable description |
| `pattern` | string | File glob pattern or value pattern |
| `optional` | boolean | Whether input is optional (default: false) |
| `default` | any | Default value if not provided |
| `enum` | array | List of allowed values |
| `ontologies` | array | EDAM or other ontology annotations |
### Migration Guide
#### From nf-core Module to Registry Module
**Before (nf-core local):**
```yaml
name: bwa_mem
description: Align reads using BWA-MEM
keywords:
- alignment
- bwa
tools:
- bwa:
description: BWA software
homepage: http://bio-bwa.sourceforge.net/
license: ["GPL-3.0-or-later"]
identifier: biotools:bwa
authors:
- "@drpatelh"
maintainers:
- "@drpatelh"
input:
# ... existing input spec
output:
# ... existing output spec
```
**After (Registry-ready):**
```yaml
name: nf-core/bwa-mem # Added scope prefix
version: "1.0.0" # Added version
description: Align reads using BWA-MEM
keywords:
- alignment
- bwa
license: MIT # Added module license
requires: # Added requirements
nextflow: ">=24.04.0"
tools:
- bwa:
description: BWA software
homepage: http://bio-bwa.sourceforge.net/
license: ["GPL-3.0-or-later"]
identifier: biotools:bwa
authors:
- "@drpatelh"
maintainers:
- "@drpatelh"
input:
# ... modified as shown above
output:
# ... modified as shown above
```
#### Schema Validation
Use the schema reference in your `meta.yml`:
```yaml
# yaml-language-server: $schema=https://registry.nextflow.io/schemas/module-spec/v1.0.0
name: nf-core/my-module
version: "1.0.0"
# ...
```
### Compatibility Matrix
| Feature | nf-core Current | Nextflow Module System |
|---------|-----------------|------------------------|
| Simple names | Yes | Yes (local only) |
| Scoped names | No | Yes (registry) |
| Version field | No | Yes (required for registry) |
| `tools` section | Yes | Yes |
| `components` | Yes | Deprecated |
| `requires` | No | Yes (Nextflow version constraint) |
| I/O specifications | Yes | Yes |
| Ontologies | Yes | Yes |
### Unsupported nf-core Attributes
The following attributes from the nf-core meta schema are **not supported** in the Nextflow module system:
| Attribute | Reason | Alternative |
|-----------|--------|-------------|
| `extra_args` | Not adopted in practice by nf-core modules | To be defined |
| `components` | No longer supported | Module dependencies are managed via `nextflow.config` |
### Complete Examples
#### Minimal nf-core Module
```yaml
name: fastqc
description: Run FastQC on sequenced reads
keywords:
- quality control
- qc
- fastq
tools:
- fastqc:
description: FastQC quality metrics
homepage: https://www.bioinformatics.babraham.ac.uk/projects/fastqc/
license: ["GPL-2.0-only"]
identifier: biotools:fastqc
authors:
- "@drpatelh"
maintainers:
- "@drpatelh"
input:
- - meta:
type: map
description: Metadata map
- reads:
type: file
description: List of input FastQ files
output:
html:
- "*.html":
type: file
description: FastQC HTML report
versions:
- versions.yml:
type: file
description: Software versions
```
#### Full Registry Module
```yaml
name: nf-core/bwa-align
version: "1.2.4"
description: Align reads to reference genome using BWA-MEM algorithm
keywords:
- alignment
- mapping
- bwa
- bam
- fastq
license: MIT
requires:
nextflow: ">=24.04.0"
tools:
- bwa:
description: |
BWA is a software package for mapping DNA sequences
against a large reference genome.
homepage: http://bio-bwa.sourceforge.net/
documentation: https://bio-bwa.sourceforge.net/bwa.shtml
doi: 10.1093/bioinformatics/btp324
license: ["GPL-3.0-or-later"]
identifier: biotools:bwa
authors:
- "@nf-core"
maintainers:
- "@drpatelh"
- "@maxulysse"
input:
- - name: meta
type: map
description: Sample metadata map (e.g., [ id:'sample1', single_end:false ])
- name: reads
type: file
description: Input FastQ files
ontologies:
- edam: "http://edamontology.org/format_1930"
- - name: meta2
type: map
description: Reference metadata
- name: index
type: directory
description: BWA index directory
ontologies:
- edam: "http://edamontology.org/data_3210"
output:
- - name: meta
type: map
description: Sample metadata
- name: bam
type: file
description: Aligned BAM file
pattern: "*.bam"
ontologies:
- edam: "http://edamontology.org/format_2572"
- name: versions
type: file
description: Software versions
pattern: "versions.yml"
```