Amazon S3 服务 ConnectionPoolTimeoutException：等待来自池的连接超时

Question

当我在 Amazon 上使用 s3 时，我不断收到丑陋的 ConnectionPoolTimeoutException。

问题是由于应用程序的前端，我无法在前端完成之前关闭打开的 s3 对象，所以我实现了这个解决方案：

@Autowired
private AmazonS3 s3client; // credentials are set properly.

private static List<S3Object> openedObjects = new ArrayList<S3Object>();

// initialize bucket :
private String bucketName = "myShinyNewBucket";
private synchronized boolean initBucket(){
    try{
        Boolean exists = null;
        try{
            exists = s3client.doesBucketExist(bucketName);
        }catch(Exception e1){
            System.out.println("\n\n\tToo many opened objects ; closing...\n\n");
            deleteOpenedS3Objects();
            exists = s3client.doesBucketExist(bucketName);
        }
        if(exists!=null){
            if(!exists){
                s3client.createBucket(new CreateBucketRequest(bucketName));
            }
            return true;
        }
    }
    catch(Exception e){
        System.out.println("\n\n\tFailed to initialize bucket.\n");
        e.printStackTrace();
    }
    return false;
}

private synchronized void deleteOpenedS3Objects(){
    System.out.println("\n\tClosing opened objects...");
    try{
        for(int i=0 ; i<openedObjects.size() ; i++){
            openedObjects.get(i).close();
            openedObjects.remove(i);
        }
    }catch(Exception e1){
        System.out.println("\tCould not close all opened s3 objects, only the first "+i);
    }
    System.out.println("\tTrying again :\n\n");
}

// GET :
public final String getFromAWS(final String amazonName){
    S3Object s3object = null;

    if(initBucket()){
        try{
            try{
                s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
            }catch(AmazonClientException e){
                deleteOpenedS3Objects();
                s3object = s3client.getObject(new GetObjectRequest(bucketName, amazonName));
            }
            openedObjects.add(s3object);
            return s3object.getObjectContent().getHttpRequest().getURI().toString();
        }catch(Exception e1){
            if (((AmazonS3Exception)e1).getStatusCode() == HttpStatus.SC_NOT_FOUND){
                System.out.println("\n\nNo such object in bucket.\n");
            }
            else{
                System.out.println("\n\n\tCould not read bject from bucket.\n\n");
                e1.printStackTrace();
            }
        }
    }
    return null;
}

然而，异常仍在发生。

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:286) ~[httpclient-4.5.1.jar!/:4.5.1]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
        at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
        at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
        at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
...

只有当我在控制台中执行 ctrl+c 时，它才会到达关闭打开的 s3 连接的部分：

... Caused by: java.lang.InterruptedException: Operation interrupted
        at org.apache.http.pool.PoolEntryFuture.await(PoolEntryFuture.java:142) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool.getPoolEntryBlocking(AbstractConnPool.java:306) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool.access$000(AbstractConnPool.java:64) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:192) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.AbstractConnPool$2.getPoolEntry(AbstractConnPool.java:185) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.pool.PoolEntryFuture.get(PoolEntryFuture.java:107) ~[httpcore-4.4.4.jar!/:4.4.4]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.leaseConnection(PoolingHttpClientConnectionManager.java:276) ~[httpclient-4.5.1.jar!/:4.5.1]
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager$1.get(PoolingHttpClientConnectionManager.java:263) ~[httpclient-4.5.1.jar!/:4.5.1]
        at sun.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) ~[na:na]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_91]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_91]
        at com.amazonaws.http.conn.ClientConnectionRequestFactory$Handler.invoke(ClientConnectionRequestFactory.java:70) ~[aws-java-sdk-core-1.11.8.jar!/:na]
        at com.amazonaws.http.conn.$Proxy188.get(Unknown Source) ~[na:na]
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:190) ~[httpclient-4.5.1.jar!/:4.5.1]
        ... 148 common frames omitted


        Too many opened objects. // <-- This is where it catches it.



        Closing opened objects...
        Could not close all opened s3 objects.
        Trying again :




        Failed to initialize bucket.

再次，不幸的是，在我离开 s3-client 类中的函数之前，我无法关闭打开的 s3objects。我唯一的希望是等到 TimeoutException 发生，抓住它，然后关闭所有打开的对象并重试。 但是，我似乎无法在正确的地方捕捉到它。

请帮忙。

谢谢。

Answer 1

每当我们处理任何类型的文件读取或写入时，我们都必须使用try with resources。这将自动关闭使用的资源，即使我们遇到任何异常。

在我的情况下：我在从 S3 读取批量操作时得到 404，我在编写脚本时通过记录错误并忘记关闭连接时忽略了它最后阻塞。

示例 sn-p：

static String readFirstLineFromFile(String path) throws IOException {
    try (BufferedReader br =
                   new BufferedReader(new FileReader(path))) {
        return br.readLine();
    }
}

注意：

1. 尝试使用 JDK 8 及更高版本支持的资源。
2. try() 中声明的所有资源都必须实现AutoCloseable 接口。

Answer 2

我认为解决方案是在我的@ControllerAdvice 类中捕获并处理丑陋的 TimeoutException。我已经这样做了，到目前为止我还没有在应用程序中发生这种情况。确定后我会发布确认信息。