【问题标题】:AWS C++ S3 SDK PutObjectRequest unable to connect to endpointAWS C++ S3 SDK PutObjectRequest 无法连接到终端节点
【发布时间】:2016-12-14 00:35:36
【问题描述】:

在使用 AWS C++ SDK 时,我遇到了一个问题,即尝试执行 PutObjectRequest 时,当上传超过 ~ 400 KB 时会抱怨它“无法连接到端点”。

Aws::Client::ClientConfiguration clientConfig;
clientConfig.scheme = Aws::Http::Scheme::HTTPS;
clientConfig.region = Aws::Region::US_EAST_1;

Aws::S3::S3Client s3Client(clientConfig);

Aws::S3::Model::PutObjectRequest putObjectRequest;
putObjectRequest.SetBucket("mybucket");
putObjectRequest.SetKey("mykey");

typedef boost::iostreams::basic_array_source<char> Device;
boost::iostreams::stream_buffer<Device> stmbuf(compressedData, dataSize);
std::iostream *stm = new std::iostream(&stmbuf);

putObjectRequest.SetBody(std::shared_ptr<Aws::IOStream>(stm));
putObjectRequest.SetContentLength(dataSize);

Aws::S3::Model::PutObjectOutcome outcome = s3Client.PutObject(putObjectRequest);

只要我的数据小于 ~400KB,它就会上传到 S3 上的一个文件中,但除此之外,它就无法连接到端点。我应该能够在一个 PutObjectRequest 中最多上传 5GB。

有什么想法吗?

编辑:

响应@JonathanHenson 的评论,AWS 日志反复显示此超时错误:

[DEBUG] 2016-08-04 13:42:03 AWSClient [0x700000081000] Request Successfully signed
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] Making request to https://s3.amazonaws.com/mybucket/myfile
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] Including headers:
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] content-length: 3151261
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] content-type: binary/octet-stream
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] host: s3.amazonaws.com
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] user-agent: aws-sdk-cpp/0.13.9 Darwin/15.6.0 x86_64
[DEBUG] 2016-08-04 13:42:03 CurlHandleContainer [0x700000081000] Attempting to acquire curl connection.
[DEBUG] 2016-08-04 13:42:03 CurlHandleContainer [0x700000081000] Returning connection handle 0x10b09cc00
[DEBUG] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] Obtained connection handle 0x10b09cc00
[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] HTTP/1.1 100 Continue

[TRACE] 2016-08-04 13:42:03 CurlHttpClient [0x700000081000] 

[ERROR] 2016-08-04 13:42:06 CurlHttpClient [0x700000081000] Curl returned error code 28
[DEBUG] 2016-08-04 13:42:06 CurlHandleContainer [0x700000081000] Releasing curl handle 0x10b09cc00
[DEBUG] 2016-08-04 13:42:06 CurlHandleContainer [0x700000081000] Notifying waiting threads.
[DEBUG] 2016-08-04 13:42:06 AWSClient [0x700000081000] Request returned error. Attempting to generate appropriate error codes from response
[WARN] 2016-08-04 13:42:06 AWSClient [0x700000081000] Request failed, now waiting 12800 ms before attempting again.
[DEBUG] 2016-08-04 13:42:19 InstanceProfileCredentialsProvider [0x700000081000] Checking if latest credential pull has expired.

【问题讨论】:

  • 你能给我发一份日志文件吗?我很乐意对此进行调查。
  • 感谢@JonathanHenson 的评论。我编辑了问题以在日志文件中显示错误。我可以通过电子邮件与您联系吗?
  • 我看到您在 Apple 设备上执行此操作。这是通过wifi吗?如果是这样,您可以通过以太网尝试吗?
  • 另外,尝试设置接收超时,而不是连接超时。奇怪的是,它在 3 秒后超时。
  • 我面临同样的错误。对此的答案将不胜感激

标签: c++ amazon-s3 aws-sdk


【解决方案1】:

最终为我解决这个问题的是设置请求超时。请求超时需要足够长才能完成整个传输。如果您在互联网连接速度较慢的情况下传输大文件,请确保请求超时时间足够长以允许传输这些文件。

Aws::Client::ClientConfiguration clientConfig;
clientConfig.scheme = Aws::Http::Scheme::HTTPS;
clientConfig.region = Aws::Region::US_EAST_1;
clientConfig.connectTimeoutMs = 30000;
clientConfig.requestTimoutMs = 600000;

【讨论】:

  • 这是 curl 的一个已知问题,因为我错误地认为超时选项是正常的套接字读取超时,但事实并非如此。我正在修复我们的 libcurl 代码,使其表现得像正常的读取超时,这应该不再是问题了。
【解决方案2】:

将你的配置文件调整到下面。看看它会起作用。

Aws::Client::ClientConfiguration clientConfig;
clientConfig.scheme = Aws::Http::Scheme::HTTPS;
clientConfig.region = Aws::Region::US_EAST_1;
clientConfig.connectTimeoutMs = 30000;

Aws::S3::S3Client s3Client(clientConfig);

Aws::S3::Model::PutObjectRequest putObjectRequest;
putObjectRequest.SetBucket("mybucket");
putObjectRequest.SetKey("mykey");

typedef boost::iostreams::basic_array_source<char> Device;
boost::iostreams::stream_buffer<Device> stmbuf(compressedData, dataSize);
std::iostream *stm = new std::iostream(&stmbuf);

putObjectRequest.SetBody(std::shared_ptr<Aws::IOStream>(stm));
putObjectRequest.SetContentLength(dataSize);

Aws::S3::Model::PutObjectOutcome outcome = s3Client.PutObject(putObjectRequest);

【讨论】:

  • 不幸的是,即使使用clientConfig.connectTimeoutMs = 30000; 设置超时,它仍然会出现无法连接到端点的错误
  • 连接超时无论如何都是错误的设置,如果在发送过程中超时,这是实际的接收超时。
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2019-06-02
相关资源
最近更新 更多