如何克隆可读流答案

【问题标题】：How to clone a readable stream如何克隆可读流
【发布时间】：2020-06-08 11:54:29
【问题描述】：

我有一个流，我试图将同一个流提交到两个不同的目的地。第一个目的地是 AWS S3，第二个目的地是通过 http 请求到其他后端。

const document = fs.createReadStream(process.cwd() + "/test/resources/" + "id/document.jpg");

const s3Response = await submitToS3(document);

const backendResponse = await submitToBackend(document);

据我了解，流只能读取一次。如何将相同的流发送到两个不同的目的地。

我想过克隆流，但只是创建一个新变量并将流分配给该变量是行不通的。

【问题讨论】：

标签： node.js node-streams

【解决方案1】：

你可以查看这个 npm 模块：https://www.npmjs.com/package/readable-stream-clone

npm install readable-stream-clone

const fs = require("fs");
const ReadableStreamClone = require("readable-stream-clone");
 
const readStream = fs.createReadStream('text.txt');
 
const readStream1 = new ReadableStreamClone(readStream);
const readStream2 = new ReadableStreamClone(readStream);
 
const writeStream1 = fs.createWriteStream('sample1.txt');
const writeStream2 = fs.createWriteStream('sample2.txt');
 
readStream1.pipe(writeStream1)
readStream2.pipe(writeStream2)

【讨论】：

【解决方案2】：

Raghavendra 的回答暗示了一个很好的潜在方向。您可以将来自this answer 的多个管道与来自this answer 的S3 管道实现结合起来。

对于submitToBackend 部分，不确定您的实现究竟是什么样的，但假设您可以通过管道传输到某种 HTTP 请求...

例子：

var fs = require("fs");
const request = require("request");
const AWS = require("aws-sdk");
const s3 = new AWS.S3();

const rs = fs.createReadStream(process.cwd() + "/test/resources/" + "id/document.jpg");

function uploadFromStream(s3) {
  var pass = new stream.PassThrough();

  var params = {Bucket: BUCKET, Key: KEY, Body: pass};
  s3.upload(params, function(err, data) {
    console.log(err, data);
  });

  return pass;
}

rs.pipe(uploadFromStream(s3));

// Just guessing at your submitToBackend implementation:
const backendWs = request.post("http://example.com/docs");

// However it works, if you can get to a stream.Writable, you can now pipe the same stream.Readable:
rs.pipe(backendWs);

【讨论】：

submitToBackend 是一个异步函数，我需要等待响应
我已经更新了我的问题以反映我的功能是承诺。
啊，谢谢，我明白了，但是createReadStream 本身并没有返回一个承诺，而是一个fs.ReadStream 的实例。也许这是唯一的问题？
不，我有一个流来了，createReadStream 是正确的。然后流被传递给 s3 和 http 请求
是的，对不起，我应该等待 fs.createReadStream，我已经更新了

【解决方案3】：

var fs = require("fs");
var ReadableStreamClone = require("readable_stream");

var readStream = fs.createReadStream('text1.txt');

var readStream1 = new ReadableStreamClone(readStream);
var readStream2 = new ReadableStreamClone(readStream);

var writeStream1 = fs.createWriteStream('testsample1.txt');
var writeStream2 = fs.createWriteStream('testsample2.txt');

readStream1.pipe(writeStream1)
readStream2.pipe(writeStream2)

【讨论】：