在 .NET 中将文件保存到具有目录结构的 S3 存储桶

Question

我正在尝试使用 AWSSDK.S3 nuget 包将文件保存到 S3 存储桶。 我的代码在 F# 中，但 C# 中的答案也可以。

我有一系列要按文件夹组织的文件。据我了解，S3 存储桶中并没有真正的文件夹结构，但文件夹是文件名中的“键”。我可能是错的，但这是我目前的理解。

代码获取加密货币数据并每天制作一个文件，每枚硬币。这一直有效，直到我想为每个硬币制作一个文件夹。

文件名的格式如下：“coinname-YYMMDD.csv”，例如：“BTCUSDT-20200820.csv”。 我想要这样的结构：

coin1/
    coin1-20200819.csv
    coin1-20200820.csv
coin2/
    coin2-20200819.csv
    coin2-20200820.csv

从文档中，我需要将文件夹设置为'key'，所以这里是代码：

// upload a file
let uploadFile (filename: string) =

    // create the key
    let coinName = Path.GetFileName(filename).Split('-').[0]
    let keyName  = Path.Combine(coinName, filename)

    // create the send request
    let putObjectRequest = Amazon.S3.Model.PutObjectRequest()
    putObjectRequest.FilePath     <- filename
    putObjectRequest.Key          <- keyName
    putObjectRequest.BucketName   <- bucketName
    putObjectRequest.CannedACL    <- S3CannedACL.PublicRead
    putObjectRequest.StorageClass <- S3StorageClass.IntelligentTiering
    putObjectRequest.ContentType  <- "text/csv"

    // send the object
    let response =
        s3client.PutObjectAsync(putObjectRequest)
        |> Async.AwaitTask
        |> Async.RunSynchronously

    // DEBUG OUTPUT
    printfn "trying to save %s" keyName
    printfn "object request: %s" (putObjectRequest |> JsonConvert.SerializeObject)
    printfn "response: %s" (response |> JsonConvert.SerializeObject)

    // handle the response
    match response.HttpStatusCode with
    | HttpStatusCode.OK -> Ok()
    | _                 -> let message = sprintf "error uploading file %s to S3: %s" filename (response.ToString())
                           Error(message)

调试输出显示如下：

试图保存 /app/leechdata/ATOMUSDT/ATOMUSDT-20200819.csv 对象 要求： {"CannedACL":{"Value":"public-read"},"InputStream":null,"FilePath":"/app/leechdata/ATOMUSDT/ATOMUSDT-20200819.csv","ContentBody":null,"AutoCloseStream ":true,"AutoResetStreamPosition":true,"UseChunkEncoding":true,"BucketName":"sunnybot-cap ture","Headers":{"Count":1,"Keys":["Content-Type"],"CacheControl":null,"ContentDisposition":null,"ContentEncoding":null,"ContentLength":-1 ,"ContentMD5":null,"ContentType":"text/csv","ExpiresUtc":null,"Expires":null},"元数据":{"Count":0,"Keys":[]},"钥匙” :"/app/leechdata/ATOMUSDT/ATOMUSDT-20200819.csv","ServerSideEncryptionMethod":null,"ServerSideEncryptionCustomerMethod":null,"ServerSideEncryptionCustomerProvidedKey":null,"ServerSideEncryptionCustomerProvidedKeyMD5":null,"ServerSideEncryptionKeyManageme ntServiceKeyId":null,"ServerSideEncryptionKeyManagementServiceEncryptionContext":null,"StorageClass":{"Value":"INTELLIGENT_TIERING"},"ObjectLockLegalHoldStatus":null,"ObjectLockMode":null,"ObjectLockRetainUntilDate":"0001-01-01T00:00: 00","网站重定向 Location":null,"StreamTransferProgress":null,"ContentType":"text/csv","MD5Digest":null,"RequestPayer":null,"TagSet":[],"Grants":[]} 回复： {"过期":null,"ServerSideEncryptionMethod":null,"ETag":""6a614845e94ffa3fc1bfb8aad40bdb5b"","VersionId":null,"ServerSideEncryptionKeyManagementServiceKeyId":null,"ServerSideEncryptionCustomerMethod":null,"ServerSideEncryptionCustomerProv idedKeyMD5":null,"ServerSideEncryptionKeyManagementServiceEncryptionContext":null,"RequestCharged":null,"ResponseMetadata":{"RequestId":"81892F3E28F07562","Metadata":{"x-amz-id-2":"/mTevk7iATTIKlvIs6eGGc3QK9+rHdzgPu6SzZU36eNfEzyz6Eaz2TchBjX +eZJ N8="}},"ContentLength":0,"HttpStatusCode":200}

所以，看起来一切都很好，但文件没有出现！ 如果我不指定 Key 字段，所有文件都会正确上传，没有文件夹。

我错过了什么？

Answer 1

要将文件上传到 S3，我建议使用TransferUtility，您的密钥应包含“文件夹”和“文件名”，例如coin1/coin1-20200819.csv。 TransferUtility.UploadAsync 方法需要 3 个参数：

• 文件的路径，例如@"C:\coin1-20200819.csv"
• 您的 aws 中的存储桶名称，例如crypto
• 键名，包括存储桶中的文件夹和文件名，例如"coin1/coin1-20200819.csv" 您上传文件的代码是这样的

let fileTransferUtil = new TransferUtility( s3client )  // create a transfer utility
fileTransferUtil.UploadAsync(@"C:\coin1-20200819.csv", "crypto", "coin1/coin1-20200819.csv" ) |> Async.AwaitTask |> Async.RunSynchronously