Jmeter-如何将 S3 中的文件从一个存储桶复制到另一个存储桶

Question

我需要将大文件从 AWS S3 中的一个存储桶（源）上传到同一个 S3 中的不同存储桶（目标）。我们有一个脚本，它将从目标中选择文件并移动文件。

我尝试了下面的代码，虽然通过了采样器，但没有复制文件。然后我尝试从本地上传到 S3。由于我需要使用高达 15GB 的大文件进行测试，因此上传到 S3 需要花费大量时间。 有没有办法从 S3 复制到 S3。

import com.amazonaws.AmazonServiceException;
import com.amazonaws.SdkClientException;
import com.amazonaws.auth.profile.ProfileCredentialsProvider;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.s3.model.CopyObjectRequest;
import java.io.IOException;
public class CopyObjectSingleOperation {
    public static void main(String[] args) throws IOException {
        Regions clientRegion = Regions.DEFAULT_REGION;
        String bucketName = "*** Bucket name ***";
        String sourceKey = "*** Source object key *** ";
        String destinationKey = "*** Destination object key ***";
        try {
            AmazonS3 s3Client = AmazonS3ClientBuilder.standard()
                    .withCredentials(new ProfileCredentialsProvider())
                    .withRegion(clientRegion)
                    .build();
            // Copy the object into a new object in the same bucket.
            CopyObjectRequest copyObjRequest = new CopyObjectRequest(bucketName, sourceKey, bucketName, destinationKey);
            s3Client.copyObject(copyObjRequest);
        } catch (AmazonServiceException e) {
            // The call was transmitted successfully, but Amazon S3 couldn't process 
            // it, so it returned an error response.
            e.printStackTrace();
        } catch (SdkClientException e) {
            // Amazon S3 couldn't be contacted for a response, or the client  
            // couldn't parse the response from Amazon S3.
            e.printStackTrace();
        }
    }
}

Answer 1

您可以使用以下选项之一；

1. S3 Batch Operations job

2. AWS CLI aws s3 sync 例如aws s3 sync s3://source-examplebucket s3://destination-examplebucket

3. AWS CLI aws s3 cp 例如aws s3 cp s3://source-examplebucket/ s3://destination-examplebucket/ --recursive

4. AWS DataSync service