Nextflow，限制谷歌的并发worker数答案

【问题标题】：Nextflow, Limit the number of concurrent workers in GoogleNextflow，限制谷歌的并发worker数
【发布时间】：2021-12-14 09:39:27
【问题描述】：

我想知道在使用 Google Life Sciences 执行程序时是否有办法限制同时运行的工作人员的数量。我在 nextflow.config 文件中使用了以下内容，但显然这并不重要。

executor {
    queueSize = 100
}

有 100 多个工作人员同时运行（进程名称旁边大括号中的数字显示为 108）。

【问题讨论】：

标签： nextflow

【解决方案1】：

像您所做的那样设置executor.queueSize = 100 应该完全限制执行程序将处理的并行任务的数量。但是，我认为您遇到的问题实际上是由于以下原因：

渠道保证商品以与之前相同的顺序交付它们已被发送 - 但是 - 因为该过程是在一个并行方式，不能保证它们在与收到的订单相同。

即使 queueSize 设置为 1 也是如此。结果可能出乎意料，但我们可以通过以下方式轻松测试：

nextflow.config的内容：

executor {
    queueSize = 1
}

main.nf的内容：

nextflow.enable.dsl=2

process test {

    tag { "myval: ${myval}" }

    input:
    val myval

    """
    sleep 1
    """
}

workflow {

    myvals = Channel.of( 'A'..'Z' )

    test( myvals )
}

运行：

nextflow run -ansi-log false main.nf

结果：

N E X T F L O W  ~  version 21.04.3
Launching `main.nf` [ecstatic_visvesvaraya] - revision: d34a24fe4a
[b9/21ef9b] Submitted process > test (myval: B)
[39/783aaf] Submitted process > test (myval: H)
[b5/aeae8b] Submitted process > test (myval: A)
[7a/e36e72] Submitted process > test (myval: D)
[de/25f001] Submitted process > test (myval: I)
[72/f913f2] Submitted process > test (myval: C)
[43/a2c78e] Submitted process > test (myval: L)
[5b/7c0434] Submitted process > test (myval: F)
[25/884e7c] Submitted process > test (myval: E)
[16/4c3b41] Submitted process > test (myval: G)
[62/c7bee1] Submitted process > test (myval: Q)
[71/cdbd37] Submitted process > test (myval: J)
[e6/634461] Submitted process > test (myval: N)
[37/03dd88] Submitted process > test (myval: S)
[fd/70867c] Submitted process > test (myval: K)
[cf/fb7e83] Submitted process > test (myval: T)
[56/3d6d41] Submitted process > test (myval: M)
[1e/81ad89] Submitted process > test (myval: O)
[db/66a292] Submitted process > test (myval: R)
[d5/212940] Submitted process > test (myval: Z)
[a8/1a33ab] Submitted process > test (myval: P)
[7e/60daa7] Submitted process > test (myval: U)
[d5/4d19c4] Submitted process > test (myval: V)
[13/8404ff] Submitted process > test (myval: W)
[22/adb044] Submitted process > test (myval: X)
[65/21a22b] Submitted process > test (myval: Y)

【讨论】：