1.7 KiB
1.7 KiB
Amazon Web Services plugin for Nextflow
Summary
The Amazon Web Services (AWS) plugin provides support for AWS, including AWS Batch as a compute executor, S3 as a file system, and Fusion file system for high-performance data operations.
Get started
To use this plugin, add it to your nextflow.config:
plugins {
id 'nf-amazon'
}
Configure your AWS credentials using environment variables, AWS CLI profiles, or IAM roles. Then set up the executor and work directory:
process.executor = 'awsbatch'
process.queue = '<YOUR BATCH QUEUE>'
workDir = 's3://<YOUR BUCKET>/work'
aws {
region = 'us-east-1'
batch {
cliPath = '/home/ec2-user/miniconda/bin/aws'
}
}
Examples
Basic AWS Batch configuration
plugins {
id 'nf-amazon'
}
process.executor = 'awsbatch'
process.queue = 'my-batch-queue'
workDir = 's3://my-bucket/work'
aws {
region = 'eu-west-1'
batch {
cliPath = '/home/ec2-user/miniconda/bin/aws'
jobRole = 'arn:aws:iam::123456789:role/MyBatchJobRole'
}
}
Using Fusion file system
fusion {
enabled = true
}
wave {
enabled = true
}
process.executor = 'awsbatch'
workDir = 's3://my-bucket/work'
S3 storage options
aws {
client {
maxConnections = 20
connectionTimeout = 10000
storageEncryption = 'AES256'
}
region = 'us-east-1'
}