NSURLSession 和亚马逊 S3 上传答案

【问题标题】：NSURLSession and amazon S3 uploadsNSURLSession 和亚马逊 S3 上传
【发布时间】：2013-10-29 00:33:38
【问题描述】：

我有一个应用程序正在将图像上传到亚马逊 S3。我一直在尝试将其从使用 NSURLConnection 切换到 NSURLSession 以便在应用程序处于后台时可以继续上传！我似乎遇到了一些问题。 NSURLRequest 被创建并传递给 NSURLSession 但亚马逊发回 403 - 禁止响应，如果我将相同的请求传递给 NSURLConnection 它会完美地上传文件。

这是创建响应的代码：

NSString *requestURLString = [NSString stringWithFormat:@"http://%@.%@/%@/%@", BUCKET_NAME, AWS_HOST, DIRECTORY_NAME, filename];
NSURL *requestURL = [NSURL URLWithString:requestURLString];
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:requestURL
                                                       cachePolicy:NSURLRequestReloadIgnoringLocalAndRemoteCacheData
                                                   timeoutInterval:60.0];
// Configure request
[request setHTTPMethod:@"PUT"];
[request setValue:[NSString stringWithFormat:@"%@.%@", BUCKET_NAME, AWS_HOST] forHTTPHeaderField:@"Host"];
[request setValue:[self formattedDateString] forHTTPHeaderField:@"Date"];
[request setValue:@"public-read" forHTTPHeaderField:@"x-amz-acl"];
[request setHTTPBody:imageData];

然后这标志着响应（我认为这来自另一个 SO 答案）：

NSString *contentMd5  = [request valueForHTTPHeaderField:@"Content-MD5"];
NSString *contentType = [request valueForHTTPHeaderField:@"Content-Type"];
NSString *timestamp   = [request valueForHTTPHeaderField:@"Date"];

if (nil == contentMd5)  contentMd5  = @"";
if (nil == contentType) contentType = @"";

NSMutableString *canonicalizedAmzHeaders = [NSMutableString string];

NSArray *sortedHeaders = [[[request allHTTPHeaderFields] allKeys] sortedArrayUsingSelector:@selector(caseInsensitiveCompare:)];

for (id key in sortedHeaders)
{
    NSString *keyName = [(NSString *)key lowercaseString];
    if ([keyName hasPrefix:@"x-amz-"]){
        [canonicalizedAmzHeaders appendFormat:@"%@:%@\n", keyName, [request valueForHTTPHeaderField:(NSString *)key]];
    }
}

NSString *bucket = @"";
NSString *path   = request.URL.path;
NSString *query  = request.URL.query;

NSString *host  = [request valueForHTTPHeaderField:@"Host"];

if (![host isEqualToString:@"s3.amazonaws.com"]) {
    bucket = [host substringToIndex:[host rangeOfString:@".s3.amazonaws.com"].location];
}

NSString* canonicalizedResource;

if (nil == path || path.length < 1) {
    if ( nil == bucket || bucket.length < 1 ) {
        canonicalizedResource = @"/";
    }
    else {
        canonicalizedResource = [NSString stringWithFormat:@"/%@/", bucket];
    }
}
else {
    canonicalizedResource = [NSString stringWithFormat:@"/%@%@", bucket, path];
}

if (query != nil && [query length] > 0) {
    canonicalizedResource = [canonicalizedResource stringByAppendingFormat:@"?%@", query];
}

NSString* stringToSign = [NSString stringWithFormat:@"%@\n%@\n%@\n%@\n%@%@", [request HTTPMethod], contentMd5, contentType, timestamp, canonicalizedAmzHeaders, canonicalizedResource];

NSString *signature = [self signatureForString:stringToSign];

[request setValue:[NSString stringWithFormat:@"AWS %@:%@", self.S3AccessKey, signature] forHTTPHeaderField:@"Authorization"];

那么如果我使用这行代码：

[NSURLConnection connectionWithRequest:request delegate:self];

它可以工作并上传文件，但如果我使用：

NSURLSessionUploadTask *task = [self.session uploadTaskWithRequest:request fromFile:[NSURL fileURLWithPath:filePath]];
[task resume];

我得到了禁止的错误..!?

有没有人尝试用这个上传到 S3 并遇到类似的问题？我想知道这是否与会话暂停和恢复上传的方式有关，或者它对请求做了一些有趣的事情..？

一种可能的解决方案是将文件上传到我控制的临时服务器，并在完成后将其转发到 S3...但这显然不是一个理想的解决方案！

非常感谢任何帮助！

谢谢！

【问题讨论】：

@GeogeGreen 我需要将大型视频上传到 s3 存储桶，很可能是 5GB，我可以使用 NSURLSession 来完成吗，因为我读到的是后台会话不会执行很长时间

标签： iphone ios cocoa-touch amazon-s3 nsurlsession

【解决方案1】：

我还不太了解NSURLSessionUploadTask，但我可以告诉你我将如何调试它。

我会使用Charles 之类的工具来查看我的应用程序发出的 HTTP(S) 请求。问题可能是NSURLSessionUploadTask 忽略了您设置的标头，或者它使用的 HTTP 方法与 Amazon 的 S3 对文件上传的预期不同。这可以通过拦截代理轻松验证。

此外，当 Amazon S3 返回类似 403 的错误时，它实际上会发回一个 XML 文档，其中包含有关该错误的更多信息。也许NSURLSession 有一个可以检索响应正文的委托方法？如果没有，Charles 肯定会给你更多的见解。

【讨论】：

太棒了，这真的很有帮助。在我签署请求后，Apple 正在添加一个额外的标头字段！
@GeorgeGreen 你能提供更多信息吗？你最终是如何克服这个问题的？
@GeorgeGreen 我也很感兴趣。
@GeorgeGreen 你能详细说明一下吗？
来自“使用文件上传正文内容”的文档：“会话对象根据数据对象的大小计算 Content-Length 标头。如果您的应用没有为Content-Type 头，会话也提供一个。”这可能会弄乱您的 s3 签名。

【解决方案2】：

我只是花了一些时间，终于成功了。最好的方法是使用 AWS 库创建带有签名标头的请求，然后复制请求。复制请求至关重要，因为否则 NSURLSessionTask 会失败。在下面的代码示例中，我使用了 AFNetworking 和子类 AFHTTPSessionManager，但此代码也适用于 NSURLSession。

    @implementation MyAFHTTPSessionManager
    {

    }

    static MyAFHTTPSessionManager *sessionManager = nil;
    + (instancetype)manager {
        if (!sessionManager)
            sessionManager = [[MyAFHTTPSessionManager alloc] init];
        return sessionManager;
    }

    - (id)init {
        NSURLSessionConfiguration *sessionConfiguration = [NSURLSessionConfiguration          backgroundSessionConfiguration:toutBackgroundSessionNameAF];
        sessionConfiguration.timeoutIntervalForRequest = 30;
        sessionConfiguration.timeoutIntervalForResource = 300;
        self = [super initWithSessionConfiguration:sessionConfiguration];
        if (self)
        {
        }
        return self;
    }

    - (NSURLSessionDataTask *)POSTDataToS3:(NSURL *)fromFile
                               Key:(NSString *)key
                         completionHandler:(void (^)(NSURLResponse *response, id responseObject, NSError *error))completionHandler
    {
        S3PutObjectRequest *s3Request = [[S3PutObjectRequest alloc] initWithKey:key inBucket:_s3Bucket];
        s3Request.cannedACL = [S3CannedACL publicReadWrite];
        s3Request.securityToken = [CTUserDefaults awsS3SessionToken];
        [s3Request configureURLRequest];
        NSMutableURLRequest *request = [_s3Client signS3Request:s3Request];
        // For some reason, the signed S3 request comes back with '(null)' as a host.
        NSString *urlString = [NSString stringWithFormat:@"%@/%@/%@", _s3Client.endpoint, _s3Bucket, [key stringWithURLEncoding]] ;
        request.URL = [NSURL URLWithString:urlString];
        // Have to create a new request and copy all the headers otherwise the NSURLSessionDataTask will fail (since request get a pointer back to AmazonURLRequest which is a subclass of NSMutableURLRequest)
        NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc] initWithURL:[NSURL URLWithString:urlString]];
        [request2 setHTTPMethod:@"PUT"];
        [request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];
        NSURLSessionDataTask *task = [self uploadTaskWithRequest:request2
                                                fromFile:fromFile
                                                progress:nil 
                                       completionHandler:completionHandler];
        return task;
    }

    @end

另一个很好的资源是苹果示例代码here并寻找“简单后台传输”

【讨论】：

【解决方案3】：

我根据 Zeev Vax 的回答使它工作。我想就我遇到的问题提供一些见解并提供一些小的改进。

例如构建一个普通的 PutRequest

S3PutObjectRequest* putRequest = [[S3PutObjectRequest alloc] initWithKey:keyName inBucket:bucketName];

putRequest.credentials = credentials;
putRequest.filename = theFilePath;

现在我们需要做一些 S3Client 通常为我们做的工作

// set the endpoint, so it is not null
putRequest.endpoint = s3Client.endpoint;

// if you are using session based authentication, otherwise leave it out
putRequest.securityToken = messageTokenDTO.securityToken;

// sign the request (also computes md5 checksums etc.)
NSMutableURLRequest *request = [s3Client signS3Request:putRequest];

现在将所有内容复制到新请求中。亚马逊使用他们自己的 NSUrlRequest 类，这会导致异常

NSMutableURLRequest* request2 = [[NSMutableURLRequest alloc]initWithURL:request.URL];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];

现在我们可以开始实际的传输了

NSURLSession* backgroundSession = [self backgroundSession];
_uploadTask = [backgroundSession uploadTaskWithRequest:request2 fromFile:[NSURL fileURLWithPath:theFilePath]];
[_uploadTask resume];

这是创建后台会话的代码：

- (NSURLSession *)backgroundSession {
    static NSURLSession *session = nil;
    static dispatch_once_t onceToken;
    dispatch_once(&onceToken, ^{

        NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfiguration:@"com.example.my.unique.id"];
        session = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
    });

    return session;
}

我花了一段时间才弄清楚会话/任务委托需要处理身份验证挑战（我们实际上是对 s3 的身份验证）。所以只要实现

- (void)URLSession:(NSURLSession *)session didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *))completionHandler {
    NSLog(@"session did receive challenge");
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}

【讨论】：

您将如何使用预签名身份验证来做到这一点？客户端只有 accessKey 和签名，这假设你有 key 和 secret。
我之前实际上是使用FederationToken来生成临时凭证（docs.aws.amazon.com/STS/latest/APIReference/…）。我从未使用过预签名的 url，但据我了解文档，它是对生成的 url 的简单 url 请求。无需为此使用 AWS 开发工具包方法，只需创建一个 NSUrlSessionUploadTask，根本不需要任何 AWS 集成。
它工作得很好。奇怪的是，我无法为上传的文件添加前缀-即s3.amazonaws.com/NSURLessionUploadTest/image.jpg 工作正常，但s3.amazonaws.com/NSURLessionUploadTest/Prefix/image.jpg 失败..有人遇到过这个吗？
因为这是 S3 的任何其他字符串，结果应该是相同的，有或没有前缀。也许是存储桶策略阻止您上传。究竟是什么错误？第二个想法：请在开始上传任务之前打印出request2的url，可能斜杠正在以某种方式更改。
如果互联网连接中断，我该如何恢复

【解决方案4】：

这是我运行任务的代码：

AmazonS3Client *s3Client = [[AmazonS3Client alloc] initWithAccessKey:accessKey withSecretKey:secretKey];
S3PutObjectRequest *s3PutObjectRequest = [[S3PutObjectRequest alloc] initWithKey:[url lastPathComponent] inBucket:bucket];
s3PutObjectRequest.cannedACL = [S3CannedACL publicRead];
s3PutObjectRequest.endpoint = s3Client.endpoint;
s3PutObjectRequest.contentType = fileMIMEType([url absoluteString]);
[s3PutObjectRequest configureURLRequest];

NSMutableURLRequest *request = [s3Client signS3Request:s3PutObjectRequest];
NSMutableURLRequest *request2 = [[NSMutableURLRequest alloc]initWithURL:request.URL];
[request2 setHTTPMethod:request.HTTPMethod];
[request2 setAllHTTPHeaderFields:[request allHTTPHeaderFields]];

NSURLSessionUploadTask *task = [[self backgroundURLSession] uploadTaskWithRequest:request2 fromFile:url];
[task resume];

我开源了我上传的S3后台https://github.com/genadyo/S3Uploader/

【讨论】：

不鼓励仅链接的答案。请在您的答案中包含您解决方案的突出方面，或删除此答案并发表评论。
@Genady 无论如何，如果互联网连接断开，我可以继续上传？？
有趣的问题，我还没试过。
@Genady Okrain 目前如果连接断开，上传过程将停止，你认为是否有机会从停止的地方恢复上传？？
仅适用于 API v1，但目前 v2 可用，所以它不是实际的

【解决方案5】：

这里的答案有些过时，我花了很多时间尝试在 Swift 和新的 AWS SDK 中完成这项工作。所以这里是如何在 Swift 中使用新的AWSS3PreSignedURLBuilder（在 2.0.7+ 版本中可用）：

class S3BackgroundUpload : NSObject {

    // Swift doesn't support static properties yet, so have to use structs to achieve the same thing.
    struct Static {
        static var session : NSURLSession?
    }

    override init() {
        super.init()

        // Note: There are probably safer ways to store the AWS credentials.
        let configPath = NSBundle.mainBundle().pathForResource("appconfig", ofType: "plist")
        let config = NSDictionary(contentsOfFile: configPath!)
        let accessKey = config.objectForKey("awsAccessKeyId") as String?
        let secretKey = config.objectForKey("awsSecretAccessKey") as String?
        let credentialsProvider = AWSStaticCredentialsProvider .credentialsWithAccessKey(accessKey!, secretKey: secretKey!)

        // AWSRegionType.USEast1 is the default S3 endpoint (use it if you don't need specific endpoints such as s3-us-west-2.amazonaws.com)
        let configuration = AWSServiceConfiguration(region: AWSRegionType.USEast1, credentialsProvider: credentialsProvider)

        // This is setting the configuration for all AWS services, you can also pass in this configuration to the AWSS3PreSignedURLBuilder directly.
        AWSServiceManager.defaultServiceManager().setDefaultServiceConfiguration(configuration)

        if Static.session == nil {
            let configIdentifier = "com.example.s3-background-upload"

            var config : NSURLSessionConfiguration
            if NSURLSessionConfiguration.respondsToSelector("backgroundSessionConfigurationWithIdentifier:") {
                // iOS8
                config = NSURLSessionConfiguration.backgroundSessionConfigurationWithIdentifier(configIdentifier)
            } else {
                // iOS7
                config = NSURLSessionConfiguration.backgroundSessionConfiguration(configIdentifier)
            }

            // NSURLSession background sessions *need* to have a delegate.
            Static.session = NSURLSession(configuration: config, delegate: self, delegateQueue: nil)
        }
    }

    func upload() {
        let s3path = "/some/path/some_file.jpg"
        let filePath = "/var/etc/etc/some_file.jpg"

        // Check if the file actually exists to prevent weird uncaught obj-c exceptions.
        if NSFileManager.defaultManager().fileExistsAtPath(filePath) == false {
            NSLog("file does not exist at %@", filePath)
            return
        }

        // NSURLSession needs the filepath in a "file://" NSURL format.
        let fileUrl = NSURL(string: "file://\(filePath)")

        let preSignedReq = AWSS3GetPreSignedURLRequest()
        preSignedReq.bucket = "bucket-name"
        preSignedReq.key = s3path
        preSignedReq.HTTPMethod = AWSHTTPMethod.PUT                   // required
        preSignedReq.contentType = "image/jpeg"                       // required
        preSignedReq.expires = NSDate(timeIntervalSinceNow: 60*60)    // required

        // The defaultS3PreSignedURLBuilder uses the global config, as specified in the init method.
        let urlBuilder = AWSS3PreSignedURLBuilder.defaultS3PreSignedURLBuilder()

        // The new AWS SDK uses BFTasks to chain requests together:
        urlBuilder.getPreSignedURL(preSignedReq).continueWithBlock { (task) -> AnyObject! in

            if task.error != nil {
                NSLog("getPreSignedURL error: %@", task.error)
                return nil
            }

            var preSignedUrl = task.result as NSURL
            NSLog("preSignedUrl: %@", preSignedUrl)

            var request = NSMutableURLRequest(URL: preSignedUrl)
            request.cachePolicy = NSURLRequestCachePolicy.ReloadIgnoringLocalCacheData

            // Make sure the content-type and http method are the same as in preSignedReq
            request.HTTPMethod = "PUT"
            request.setValue(preSignedReq.contentType, forHTTPHeaderField: "Content-Type")

            // NSURLSession background session does *not* support completionHandler, so don't set it.
            let uploadTask = Static.session?.uploadTaskWithRequest(request, fromFile: fileUrl)

            // Start the upload task:
            uploadTask?.resume()

            return nil
        }
    }
}

extension S3BackgroundUpload : NSURLSessionDelegate {

    func URLSession(session: NSURLSession, dataTask: NSURLSessionDataTask, didReceiveData data: NSData) {
        NSLog("did receive data: %@", NSString(data: data, encoding: NSUTF8StringEncoding))
    }

    func URLSession(session: NSURLSession, task: NSURLSessionTask, didCompleteWithError error: NSError?) {
        NSLog("session did complete")
        if error != nil {
            NSLog("error: %@", error!.localizedDescription)
        }
        // Finish up your post-upload tasks.
    }
}

【讨论】：

我想使用 V2 api 将视频上传到 S3 存储桶，它应该支持暂停和恢复功能，可以使用此代码的 objc 版本 sn-p 上传

【解决方案6】：

对于后台上传/下载，您需要使用带有后台配置的 NSURLSession。从 AWS SDK 2.0.7 开始，您可以使用预签名请求：

PreSigned URL Builder** - SDK 现在包括对预签名的支持 Amazon Simple Storage Service (S3) URL。您可以使用这些 URL 使用 NSURLSession 类执行后台传输。

初始化后台 NSURLSession 和 AWS 服务

- (void)initBackgroundURLSessionAndAWS
{
    NSURLSessionConfiguration *configuration = [NSURLSessionConfiguration backgroundSessionConfigurationWithIdentifier:AWSS3BackgroundSessionUploadIdentifier];
    self.urlSession = [NSURLSession sessionWithConfiguration:configuration delegate:self delegateQueue:nil];
    AWSServiceConfiguration *configuration = [AWSServiceConfiguration configurationWithRegion:DefaultServiceRegionType credentialsProvider:credentialsProvider];
    [AWSServiceManager defaultServiceManager].defaultServiceConfiguration = configuration;
    self.awss3 = [[AWSS3 alloc] initWithConfiguration:configuration];
}

实现上传文件功能

- (void)uploadFile
{
    AWSS3GetPreSignedURLRequest *getPreSignedURLRequest = [AWSS3GetPreSignedURLRequest new];
    getPreSignedURLRequest.bucket = @"your_bucket";
    getPreSignedURLRequest.key = @"your_key";
    getPreSignedURLRequest.HTTPMethod = AWSHTTPMethodPUT;
    getPreSignedURLRequest.expires = [NSDate dateWithTimeIntervalSinceNow:3600];
    //Important: must set contentType for PUT request
    getPreSignedURLRequest.contentType = @"your_contentType";

    [[[AWSS3PreSignedURLBuilder defaultS3PreSignedURLBuilder] getPreSignedURL:getPreSignedURLRequest] continueWithBlock:^id(BFTask *task) {
        if (task.error)
        {
            NSLog(@"Error BFTask: %@", task.error);
        }
        else
        {
            NSURL *presignedURL = task.result;
            NSLog(@"upload presignedURL is: \n%@", presignedURL);

            NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:presignedURL];
            request.cachePolicy = NSURLRequestReloadIgnoringLocalCacheData;
            [request setHTTPMethod:@"PUT"];
            [request setValue:contentType forHTTPHeaderField:@"Content-Type"];

//          Background NSURLSessions do not support the block interfaces, delegate only.
            NSURLSessionUploadTask *uploadTask = [self.session uploadTaskWithRequest:request fromFile:@"file_path"];

            [uploadTask resume];
        }
        return nil;
    }];
}

NSURLSession 代理函数：

- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error
{
    if (error)
    {
        NSLog(@"S3 UploadTask: %@ completed with error: %@", task, [error localizedDescription]);
    }
    else
    {
//      AWSS3GetPreSignedURLRequest does not contain ACL property, so it has to be set after file was uploaded
        AWSS3PutObjectAclRequest *aclRequest = [AWSS3PutObjectAclRequest new];
        aclRequest.bucket = @"your_bucket";
        aclRequest.key = @"yout_key";
        aclRequest.ACL = AWSS3ObjectCannedACLPublicRead;

        [[self.awss3 putObjectAcl:aclRequest] continueWithBlock:^id(BFTask *bftask) {
            dispatch_async(dispatch_get_main_queue(), ^{
                if (bftask.error)
                {
                    NSLog(@"Error putObjectAcl: %@", [bftask.error localizedDescription]);
                }
                else
                {
                    NSLog(@"ACL for an uploaded file was changed successfully!");
                }
            });
            return nil;
        }];
    }
}

【讨论】：

【解决方案7】：

最近亚马逊已将 AWS api 更新为 2.2.4。这次更新的特别之处在于，它支持后台上传，你不必使用NSURLSession来上传视频它非常简单，你可以使用下面的源代码块来测试它，我已经用我的测试过旧版本，它比以前的版本快 30 - 40 %

在 AppDelegate.m didFinishLaunchingWithOptions 方法中 // ~GM~ 为 AWS V2 配置设置认知

AWSStaticCredentialsProvider *staticProvider = [[AWSStaticCredentialsProvider alloc] initWithAccessKey:@"xxxx secretKey:@"xxxx"];  

AWSServiceConfiguration *configuration = [[AWSServiceConfiguration alloc] initWithRegion:AWSRegionUSWest2                                                                 credentialsProvider:staticProvider];

AWSServiceManager.defaultServiceManager.defaultServiceConfiguration = configuration;

在handleEventsForBackgroundURLSession方法中

[AWSS3TransferUtility interceptApplication:application
       handleEventsForBackgroundURLSession:identifier
                         completionHandler:completionHandler];

在上传类中

NSURL *fileURL = // The file to upload.

AWSS3TransferUtilityUploadExpression *expression = [AWSS3TransferUtilityUploadExpression new];
expression.uploadProgress = ^(AWSS3TransferUtilityTask *task, int64_t bytesSent, int64_t totalBytesSent, int64_t totalBytesExpectedToSend) {
    dispatch_async(dispatch_get_main_queue(), ^{
        // Do something e.g. Update a progress bar.
    });
};

AWSS3TransferUtilityUploadCompletionHandlerBlock completionHandler = ^(AWSS3TransferUtilityUploadTask *task, NSError *error) {
    dispatch_async(dispatch_get_main_queue(), ^{
        // Do something e.g. Alert a user for transfer completion.
        // On failed uploads, `error` contains the error object.
    });
};

AWSS3TransferUtility *transferUtility = [AWSS3TransferUtility defaultS3TransferUtility];
[[transferUtility uploadFile:fileURL
                      bucket:@"YourBucketName"
                         key:@"YourObjectKeyName"
                 contentType:@"text/plain"
                  expression:expression
            completionHander:completionHandler] continueWithBlock:^id(AWSTask *task) {
    if (task.error) {
        NSLog(@"Error: %@", task.error);
    }
    if (task.exception) {
        NSLog(@"Exception: %@", task.exception);
    }
    if (task.result) {
        AWSS3TransferUtilityUploadTask *uploadTask = task.result;
        // Do something with uploadTask.
    }

    return nil;
}];

【讨论】：