S3：为给定密钥生成预签名 URL。 [密钥可能/不存在]

Question

在 S3 存储桶中不存在密钥时获取消息。我正在检索该存储桶中的所有对象并将这些键与给定的搜索键匹配。如果可用则返回 URL 字符串，否则返回消息“指定的键不存在”。

他们在访问密钥时是否有任何其他提高性能的方法，这在 S3 存储桶中不可用。

这是我的代码：

public class S3Objects {
    static Properties props = new Properties();
    static InputStream resourceAsStream;
    static {
        ClassLoader classLoader = new S3Objects().getClass().getClassLoader();
        resourceAsStream = classLoader.getResourceAsStream("aws.properties");
        try {
            props.load(resourceAsStream);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    public static void main(String[] args) throws IOException, AmazonServiceException, AmazonClientException, InterruptedException {
        AWSCredentials awsCreds = new 
                        BasicAWSCredentials(props.getProperty("accessKey"), props.getProperty("secretKey"));
                        // PropertiesCredentials(resourceAsStream);
        AmazonS3 s3Client = new AmazonS3Client( awsCreds );

        String s3_BucketName = props.getProperty("bucketname");
        String folderPath_fileName = props.getProperty("path");

        //uploadObject(s3Client, s3_BucketName, folderPath_fileName);
        //downloadObject(s3Client, s3_BucketName, folderPath_fileName);
        //getSignedURLforS3File(s3Client, s3_BucketName, folderPath_fileName);
        String url = getSingnedURLKey(s3Client, s3_BucketName, folderPath_fileName);
        System.out.println("Received response:"+url);
    }
    //  <MaxKeys>1000</MaxKeys>
    private static String getSingnedURLKey(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName) {
        String folderPath = folderPath_fileName.substring(0,folderPath_fileName.lastIndexOf("/"));      
        ObjectListing folderPath_Objects = s3Client.listObjects(s3_BucketName, folderPath);

        List<S3ObjectSummary> listObjects = folderPath_Objects.getObjectSummaries();
        for(S3ObjectSummary object : listObjects){
            if(object.getKey().equalsIgnoreCase(folderPath_fileName)){
                return getSignedURLforS3File(s3Client, s3_BucketName, folderPath_fileName);
            }
        }
        return "The specified key does not exist.";
    }

    //  providing pre-signed URL to access an object w/o any AWS security credentials.
   //   Pre-Signed URL = s3_BucketName.s3.amazonaws.com/folderPath_fileName?AWSAccessKeyId=XX&Expires=XX&Signature=XX
    public static String getSignedURLforS3File(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName){
        GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(s3_BucketName, folderPath_fileName, HttpMethod.GET);
        request.setExpiration( new Date(System.currentTimeMillis() + 1000 * 60 * 15) ); // Default 15 min

        String url = s3Client.generatePresignedUrl( request ).toString();
        System.out.println("Pre-Signed URL = " + url);
        return url;
    }

    public static void uploadObject(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName) 
            throws AmazonServiceException, AmazonClientException, InterruptedException{
        TransferManager tm = new TransferManager(s3Client);

        PutObjectRequest putObjectRequest = 
                new PutObjectRequest(s3_BucketName, folderPath_fileName, new File("newImg.jpg"));
        Upload myUpload = tm.upload( putObjectRequest );
        myUpload.waitForCompletion();//block the current thread and wait for your transfer to complete.
        tm.shutdownNow();            //to release the resources once the transfer is complete.
    }
   //   When accessing a key which is not available in S3, it throws an exception The specified key does not exist.
    public static void downloadObject(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName) throws IOException{
        GetObjectRequest request = new GetObjectRequest(s3_BucketName,folderPath_fileName);
        try{
            S3Object s3object = s3Client.getObject( request );
            System.out.println("Content-Type: " + s3object.getObjectMetadata().getContentType());
            S3ObjectInputStream objectContent = s3object.getObjectContent();

            FileUtils.copyInputStreamToFile(objectContent, new File("targetFile.jpg"));
        }catch(AmazonS3Exception s3){
            System.out.println("Received error response:"+s3.getMessage());
        }
    }

}

aws.properties

accessKey   =XXXXXXXXX
secretKey   =XXXXXXXXX

bucketname  =examplebucket
path        =/photos/2006/February/sample.jpg

请告诉我他们是否有任何其他方法可以减少所有密钥的迭代次数并获得一些消息“密钥不存在”。

在请求生成预签名 URL 的密钥时。如果

• Key Present « 返回签名的 URL。
• 密钥不存在 «消息作为密钥不可用。

Answer 1

给定键，使用getObjectMetadata 快速确定对象是否存在。如果成功，则该对象存在。如果没有，请检查错误以确认它不是需要重试的临时错误。如果没有，则没有这样的密钥。

按照您的操作迭代对象不仅不会扩展，而且成本也会大大增加，因为列表请求的每个请求的价格高于获取对象或获取其元数据，这应该非常快.此操作向 S3 发送 HTTP HEAD 请求，该请求仅在对象存在时返回 200 OK。

但是，从设计的角度来看，我认为该服务不应该真正关心对象是否存在。为什么您会收到对不存在的对象的请求？谁在问这个？那应该是调用者的问题——如果你为一个不存在的对象生成一个签名的 URL，当调用者尝试使用这个 URL 时，请求将失败并出现错误……但是为一个对象生成一个签名的 URL不存在的对象是一个完全有效的操作。可以在对象实际上传之前对 URL 进行签名，并且只要 URL 没有过期，一旦对象被创建，它仍然可以工作，如果它是稍后创建的。

Answer 2

Amazon S3：检查 Key Exists 并生成 PresignedUrl

使用 getObjectMetadata():AmazonS3Client 这对于仅获取对象元数据很有用，并且可以避免在获取对象数据时浪费带宽。

private static String getPreSignedURLAsString_GetMetaData(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName) {
    GetObjectMetadataRequest request = new GetObjectMetadataRequest(s3_BucketName,folderPath_fileName);
    try{
        ObjectMetadata objectMetadata = s3Client.getObjectMetadata( request );
        System.out.println("Key Exists in S3");
        return getSignedURLforS3File(s3Client, s3_BucketName, folderPath_fileName);
    }catch(AmazonS3Exception s3){
        System.out.println("Received error response:"+s3.getMessage());
    }
    return "Key Not Found.";
}

使用 doesObjectExist():AmazonS3Client 它在内部调用 getObjectMetadata(bucketName, objectName); - [aws-java- sdk-1.10.68]

private static String getPreSignedURLAsString_DoesKeyExists(AmazonS3 s3Client, String s3_BucketName, String folderPath_fileName) {
    try{
        boolean doesObjectExist = s3Client.doesObjectExist(s3_BucketName, folderPath_fileName);
        if (doesObjectExist) {
            System.out.println("Key Exists in S3");
            return getSignedURLforS3File(s3Client, s3_BucketName, folderPath_fileName);
        }
    }catch(AmazonS3Exception s3){
        System.out.println("Received error response:"+s3.getMessage());
    }
    return "Key Not Found.";
}

必需的罐子[aws-java{sdk, sdk-core, sdk-s3}, fasterxml.jackson{databind, core, annotations}, joda-time, http{clent, core}]