带有 etags 的 s3 流式传输（节点）

Question

我们正在使用 aws-sdk 中的 createReadStream() 从节点中的 S3 流式传输内容。我们想添加 etag 支持。如果我们从客户端添加“If-None-Match”标头，s3 会抛出 NotModified 作为我似乎无法处理的错误。

retrieveFile = function(req, res) {
    var s3 = new AWS.S3(); 
    var params = {
        Bucket: bucket,
        key: key
    };
    if (req.get('If-None-Match')) {
        params.IfNoneMatch = req.get('If-None-Match');
    }
    return s3.getObject(params).on('httpHeaders', function(statusCode, headers) {
        if (headers.etag) {
            res.set('etag', headers.etag);
        }
        if (headers['content-length']) {
            return res.set('content-length', headers['content-length']);
        }
    }).createReadStream().pipe(res);
};

我尝试过监听流上的事件，并对从 getObject 返回的请求使用回调。我可以通过这种方式收到错误消息，但 aws-sdk 中的其他内容似乎正在杀死我的进程。

/Projects/my-app/node_modules/aws-sdk/lib/request.js:31
            throw err;
            ^

NotModified: null
  at Request.extractError (/Projects/my-app/node_modules/aws-sdk/lib/services/s3.js:519:35)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
  at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
  at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
  at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
  at Request.emit (/Projects/my-app/node_modules/aws-sdk/lib/request.js:673:14)
  at Request.transition (/Projects/my-app/node_modules/aws-sdk/lib/request.js:22:10)
  at AcceptorStateMachine.runTo (/Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:14:12)
  at /Projects/my-app/node_modules/aws-sdk/lib/state_machine.js:26:10
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:38:9)
  at Request.<anonymous> (/Projects/my-app/node_modules/aws-sdk/lib/request.js:675:12)
  at Request.callListeners (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
  at callNextListener (/Projects/my-app/node_modules/aws-sdk/lib/sequential_executor.js:95:12)
  at IncomingMessage.onEnd (/Projects/my-app/node_modules/aws-sdk/lib/event_listeners.js:244:11)
  at emitNone (events.js:91:20)
  at IncomingMessage.emit (events.js:185:7)
  at endReadableNT (_stream_readable.js:974:12)
  at _combinedTickCallback (internal/process/next_tick.js:74:11)
  at process._tickDomainCallback (internal/process/next_tick.js:122:9)

Answer 1

似乎错误是在流中引发的，如果您正在侦听 AWS.Request，您将收到错误，但仍未捕获流错误。

考虑一下：

s3.getObject({
    Bucket: 'foo',
    Key: 'bar'
}).on('error', function (err) {
    console.log('Error event!');
}).createReadStream();

这将显示“错误事件！”但它也会抛出错误并退出该过程。那是因为 createReadStream 创建的流也收到错误。

考虑一下：

s3.getObject({
    Bucket: 'lalaland',
    Key: 'blabaliets'
}).on('error', function (err) {
    console.log('Error event!');
}).createReadStream().on('error', function (err) {
    console.log('Error event on stream!');
});

它将在 AWS.Request 对象上侦听错误，但也在流对象上。这将首先打印“错误事件！”，然后是“流中的错误事件！”并且该过程不会退出。

请注意：

s3.getObject({
    Bucket: 'lalaland',
    Key: 'blabaliets'
}).createReadStream().on('error', function (err) {
    console.log('Error event on stream!');
});

将仅打印“流中的错误事件！”而不是退出。这最后一个选项可能是您想要的：只听流上的错误而不是请求对象上的错误。

回归初心；此时（在流上的 on 错误中）您想要检查它是否确实是 NotModified 错误并执行res.status(304).end() 之类的操作。