【问题标题】:How to create a background single-thread FIFO job queue in node.js如何在 node.js 中创建后台单线程 FIFO 作业队列
【发布时间】:2016-05-06 11:05:18
【问题描述】:

我是一名 JS/node 初学者,我正在努力了解 Javascript 中的“并发”主题。我认为我现在对回调相当满意,但我认为这不是我的方案中要走的路。基本上我有经常性的昂贵任务 (worker),我需要在 main process 继续其工作时一一处理。这是一个最小的测试。

/*
 * We have a worker function  that does some expensive task, e.g., an
 * I/O task or something else.
 */
worker = function ( jobId ) {

    // It will take something between 1 and 2 seconds.
    var runtime = Math.floor((Math.random() * 1000) + 1000);
    console.log("started job #" + jobId + " (" + runtime + " ms)");

    // Then the worker will do something for a while ...
    setTimeout(
        function() {
            // .. and at some point it'll be finished.
            console.log("finished job #" + jobId);
        }, runtime
    );

};

/*
 * We obviously have a main process that meanwhile does other stuff
 * like processed user interactions. In this case we call this until
 * some artificial tickets are used.
 */
mainprocess = function (tickets) {

    // Simulate some processing time ..
    var runtime = Math.floor((Math.random() * 500));
    setTimeout(
        function() {
            console.log("main process #" + tickets + " (" + runtime + " ms)");
            if (tickets > 0) {
                tickets--;
                mainprocess(tickets);
            }
        }, runtime
    );
}

// At some point in the code we create workers and we want to make sure
// they're processed in the *order of creation* and *one after another* 
// without blocking the main process ...
for ( var i = 1; i <= 10; i++) {
    worker(i);
};

// ... and the some other stuff will happen for a while!
mainprocess(10);

// ..

代码当前输出类似..

started job #1 (1751 ms)
started job #2 (1417 ms)
...
started job #9 (1050 ms)
started job #10 (1864 ms)
main process #10 (142 ms)
main process #9 (228 ms)
main process #8 (149 ms)
main process #7 (88 ms)
main process #6 (410 ms)
finished job #9
finished job #5
main process #5 (265 ms)
finished job #2
main process #4 (270 ms)
finished job #7
finished job #3
finished job #1
...
main process #1 (486 ms)
main process #0 (365 ms)

我不太清楚,怎么改代码,让主进程继续执行,而工作线程按创建顺序执行(目前只按正确顺序启动) 一个接一个(目前都是并行的)。所需的输出将是..

started job #1 (1384 ms)
main process #10 (268 ms)
main process #9 (260 ms)
main process #8 (216 ms)
main process #7 (93 ms)
main process #6 (160 ms)
main process #5 (269 ms)
main process #4 (44 ms)
finished job #1
started job #2 (1121 ms)
main process #3 (172 ms)
main process #2 (170 ms)
main process #1 (437 ms)
finished job #2
started job #3 (1585 ms)
main process #0 (460 ms)
finished job #3
started job #4 (1225 ms)
finished job #4
started job #5 (1300 ms)
finished job #5

任何帮助将不胜感激。

【问题讨论】:

    标签: javascript node.js concurrency


    【解决方案1】:

    好的,我想通了。扩展和注释代码使用 javascript 的内置承诺,但您可以使用 Q 或 Bluebird 或任何其他与节点兼容的承诺库来实现相同的目标。请注意,jquery $.Deferred 对象在节点环境中不可用。

    /*
     * We have a worker function  that does some expensive task, e.g., an
     * I/O task or something else.
     */
    worker = function ( jobId ) {
    
        // we need to put it into a new promise object
        return new Promise(function(resolve, reject) {
    
            // It will take something between 1 and 2 seconds.
            var runtime = Math.floor((Math.random() * 1000) + 1000);
            console.log("started job #" + jobId + " (" + runtime + " ms)");
    
            // Then the worker will do something for a while ...
            setTimeout(
                function() {
                    // .. and at some point it'll be finished.
                    console.log("finished job #" + jobId);
                     // .. now we have to resolve the promise!!
                    resolve("resolved job #" + jobId);
                }, runtime
            );
    
        });
    };
    
    
    /*
     * We obviously have a main process that meanwhile does other stuff
     * like processed user interactions. In this case we call this until
     * some artificial tickets are used.
     */
    mainprocess = function (tickets) {
    
        // Simulate some processing time ..
        var runtime = Math.floor((Math.random() * 500));
        setTimeout(
            function() {
                console.log("main process #" + tickets + " (" + runtime + " ms)");
                if (tickets > 0) {
                    tickets--;
                    mainprocess(tickets);
                }
            }, runtime
        );
    }
    
    // create a sequence with a resolved promise
    var sequence = Promise.resolve();
    
    // At some point in the code we create workers and we want to make sure
    // they're processed in the *order of creation* and *one after another*
    // without blocking the main process ...
    for ( var i = 1; i <= 10; i++) {
        // create an IIFE so that the current "i" gets its own
        // closure when it will be used later (otherwise all job ids
        // would be "11" on invokation of the worker).
        (function() {
            var jobId = i;
            // add a new promise after the previous promise resolved
            sequence = sequence.then(
                function(result) {
                    // handle result later
                    return worker(jobId);
                    // return just added the next promise to the chain!
                },
                function(err) {
                    // handle error later
                }
            );
        })(); // END IIFE
    };
    
    // ... and the some other stuff will happen for a while!
    mainprocess(10);
    
    // ..
    

    输出如你所愿:

    started job #1 (1384 ms)
    main process #10 (268 ms)
    main process #9 (260 ms)
    main process #8 (216 ms)
    main process #7 (93 ms)
    main process #6 (160 ms)
    main process #5 (269 ms)
    main process #4 (44 ms)
    finished job #1
    started job #2 (1121 ms)
    main process #3 (172 ms)
    main process #2 (170 ms)
    main process #1 (437 ms)
    finished job #2
    started job #3 (1585 ms)
    main process #0 (460 ms)
    finished job #3
    started job #4 (1225 ms)
    finished job #4
    started job #5 (1300 ms)
    finished job #5
    ...
    

    我必须对以下提供见解的文章表示高度赞扬:

    【讨论】:

      【解决方案2】:

      如果您希望主线程先运行,然后按创建顺序运行工作线程,请将 for 循环放在调用 worker(i) 的设置超时函数中,并给出大约 1000 毫秒的恒定延迟.然后还应该给已完成的作业一个 1000 毫秒的恒定延迟,以便它们按照创建的顺序执行。 查看链接:jsfiddle.net/som99/weq87xnd/2/

      【讨论】:

      • 我不太喜欢使用持续延迟的想法。在实际应用程序中,您无法确保 worker 函数运行固定的时间。如果不是,它将浪费时间,如果是,则会中断该过程。我认为有条件的解决方案应该更合适,因此我目前尝试掌握 Promise 的窍门。
      • 但是在下面提供的解决方案中,主线程没有作为后台运行。实际应用中的场景使工作线程或辅助线程在主线程还活着的情况下运行在前台。如果主线程先完成,工作线程或服务线程如何运行?
      • 主“线程”是处理用户交互等的线程。工作“线程”处理一项昂贵的任务。没有前景或背景或活着。一切都在事件循环中的单个线程中运行。这里有两个不同的任务似乎并行运行。只需运行示例或查看输出。当所有工作人员都完成并且主进程没有剩余票时,程序终止。
      猜你喜欢
      • 1970-01-01
      • 2010-11-29
      • 2014-08-24
      • 1970-01-01
      • 1970-01-01
      • 2017-11-22
      • 2013-11-15
      • 2010-10-04
      • 1970-01-01
      相关资源
      最近更新 更多