【问题标题】:Downloading files >3Gb from S3 fails with "SocketTimeoutException: Read timed out"从 S3 下载 >3Gb 的文件失败并显示“SocketTimeoutException:读取超时”
【发布时间】:2015-03-27 12:40:15
【问题描述】:

AWS Java 开发工具包 1.9.3

在 Java 服务器应用程序中从 AWS S3 下载许多大文件 (~3Gb) 时,我不时收到 SocketTimeoutException,如下所示:

Caused by: com.amazonaws.AmazonClientException: Unable to store object contents to disk: Read timed out
 at com.amazonaws.services.s3.internal.ServiceUtils.downloadObjectToFile(ServiceUtils.java:270)
 at com.amazonaws.services.s3.internal.ServiceUtils.retryableDownloadS3ObjectToFile(ServiceUtils.java:344)
 at com.amazonaws.services.s3.transfer.TransferManager$2.call(TransferManager.java:737)
 ... 4 more
Caused by: java.net.SocketTimeoutException: Read timed out
 at java.net.SocketInputStream.socketRead0(Native Method)
 at java.net.SocketInputStream.read(SocketInputStream.java:152)
 at java.net.SocketInputStream.read(SocketInputStream.java:122)
 at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
 at sun.security.ssl.InputRecord.readV3Record(InputRecord.java:554)
 at sun.security.ssl.InputRecord.read(InputRecord.java:509)
 at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
 at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:884)
 at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
 at org.apache.http.impl.io.AbstractSessionInputBuffer.read(AbstractSessionInputBuffer.java:198)
 at org.apache.http.impl.io.ContentLengthInputStream.read(ContentLengthInputStream.java:178)
 at org.apache.http.conn.EofSensorInputStream.read(EofSensorInputStream.java:137)
 at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
 at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:151)
 at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
 at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
 at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
 at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:151)
 at java.security.DigestInputStream.read(DigestInputStream.java:161)
 at com.amazonaws.services.s3.internal.DigestValidationInputStream.read(DigestValidationInputStream.java:59)
 at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:73)
 at java.io.FilterInputStream.read(FilterInputStream.java:107)
 at com.amazonaws.services.s3.internal.ServiceUtils.downloadObjectToFile(ServiceUtils.java:265)
 ... 6 more

我使用 4 个线程从 S3 下载文件,所有线程都使用相同的 TransferManager 实例。如前所述,我使用的是 AWS SDK 1.9.3,并且只是不时地遇到此类异常。代码如下:

...
Download d = transferManager.download(currentBucket, remoteFileName, new File(tmpName));
d.waitForCompletion();
...

是SDK的问题吗?也许是 v1.9.3?

除了为ClientConfiguration 增加socketTimeout 之外,还有其他解决方案吗?

【问题讨论】:

  • 我遇到了同样的问题。我正在使用 AWS S3 Android SDK 2.1.7,我的文件大约有 10MB。你有答案来解决它吗?
  • 除了更改超时之外,这个问题仍然没有解决方案。
  • 嗨@Yury,你能告诉我如何更改超时吗?
  • 嗨@Caaarlos。以下是我可以在 AWS SDK 1.9.3 中执行的操作:ClientConfiguration config = new ClientConfiguration(); config.setConnectionTimeout(connectionTimeout); config.setSocketTimeout(readTimeout); AmazonS3 s3 = new AmazonS3Client(credentials, config);
  • 感谢 Yury,您的解决方案现在有效。

标签: java amazon-s3 socket-timeout-exception


【解决方案1】:

以下是我在 AWS SDK 1.9.3 中可以做的事情:

ClientConfiguration config = new ClientConfiguration(); 
config.setConnectionTimeout(connectionTimeout);
config.setSocketTimeout(readTimeout); 
AmazonS3 s3 = new AmazonS3Client(credentials, config);

【讨论】:

  • 默认超时时间是多少?
  • 在 1.10.x 中,socket 和连接超时的默认超时时间都是 50 秒。
  • 我很想看到这个的 v2 版本!
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 2011-09-28
  • 2017-06-27
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
相关资源
最近更新 更多