通过 WebApi 将媒体批量上传到 Azure Blob 存储

Question

我的网络应用目前允许用户使用以下方式一次上传一个媒体：

var fd = new FormData(document.forms[0]);
fd.append("media", blob); // blob is the image/video
$.ajax({
    type: "POST",
    url: '/api/media',
    data: fd
})

媒体随后被发布到 WebApi 控制器：

    [HttpPost, Route("api/media")]
    public async Task<IHttpActionResult> UploadFile()
    {
        if (!Request.Content.IsMimeMultipartContent("form-data"))
        {
            throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
        }

        string mediaPath = await _mediaService.UploadFile(User.Identity.Name, Request.Content);

        return Ok(mediaPath);
    }

然后按照以下方式做一些事情：

 public async Task<string> UploadFile(string username, HttpContent content)
 {
   var storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
   CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
   CloudBlobContainer imagesContainer = blobClient.GetContainerReference("container-" + user.UserId);
   var provider = new AzureStorageMultipartFormDataStreamProvider(imagesContainer);
   await content.ReadAsMultipartAsync(provider);
   var filename = provider.FileData.FirstOrDefault()?.LocalFileName;
   // etc
 }

这对于个人上传非常有用，但是我该如何修改它以通过返回上传文件名数组的单个流操作支持多个文件的批量上传？这方面的文档/示例似乎很少。

public class AzureStorageMultipartFormDataStreamProvider : MultipartFormDataStreamProvider
{
    private readonly CloudBlobContainer _blobContainer;
    private readonly string[] _supportedMimeTypes = { "images/png", "images/jpeg", "images/jpg", "image/png", "image/jpeg", "image/jpg", "video/webm" };

    public AzureStorageMultipartFormDataStreamProvider(CloudBlobContainer blobContainer) : base("azure")
    {
        _blobContainer = blobContainer;
    }

    public override Stream GetStream(HttpContent parent, HttpContentHeaders headers)
    {
        if (parent == null) throw new ArgumentNullException(nameof(parent));
        if (headers == null) throw new ArgumentNullException(nameof(headers));

        if (!_supportedMimeTypes.Contains(headers.ContentType.ToString().ToLower()))
        {
            throw new NotSupportedException("Only jpeg and png are supported");
        }

        // Generate a new filename for every new blob
        var fileName = Guid.NewGuid().ToString();

        CloudBlockBlob blob = _blobContainer.GetBlockBlobReference(fileName);

        if (headers.ContentType != null)
        {
            // Set appropriate content type for your uploaded file
            blob.Properties.ContentType = headers.ContentType.MediaType;
        }

        this.FileData.Add(new MultipartFileData(headers, blob.Name));

        return blob.OpenWrite();
    }
}

Answer 1

假设您的AzureStorageMultipartFormDataStreamProvider 类似于this blog 中提到的同一个类，如果请求中有多个文件，它实际上已经在处理多个文件。

因此，您需要做的就是将您的 UploadFile 更改为返回 IEnumerable<string> 并将您的控制器更改为具有 mediaPath 本身。

所以您的 MediaService 将：

var filenames = provider.FileData.Select(x => x.LocalFileName).ToList(); ;
return filenames;

你的控制器会：

var mediaPaths = await _mediaService.UploadFile(User.Identity.Name, Request.Content);
return Ok(mediaPaths);

Answer 2

由于您没有使用 AzureStorageMultipartFormDataStreamProvider 类发布相关代码。

所以我创建了一个继承自 MultipartFileStreamProvider 的自定义 AzureStorageMultipartFormDataStreamProvider，以启用 Web api 上传批量上传多个文件。

在 AzureStorageMultipartFormDataStreamProvider 中，我们可以覆盖 ExecutePostProcessingAsync 方法。

在这个方法中，我们可以获取上传文件的数据，然后我们可以将这些数据上传到天蓝色的存储中。

更多细节，你可以参考下面的代码。总控制器。

 public class UploadingController : ApiController
    {
        public Task<List<FileItem>> PostFile()
        {
            if (!Request.Content.IsMimeMultipartContent("form-data"))
            {
                throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
            }

            var multipartStreamProvider = new AzureStorageMultipartFormDataStreamProvider(GetWebApiContainer());
            return Request.Content.ReadAsMultipartAsync<AzureStorageMultipartFormDataStreamProvider>(multipartStreamProvider).ContinueWith<List<FileItem>>(t =>
            {
                if (t.IsFaulted)
                {
                    throw t.Exception;
                }

                AzureStorageMultipartFormDataStreamProvider provider = t.Result;
                return provider.Files;
            });
        }


        public static CloudBlobContainer GetWebApiContainer(string containerName = "webapi-file-container")
        {
            // Retrieve storage account from connection-string
            CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
               "your connection string");

            // Create the blob client 
            CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

            CloudBlobContainer container = blobClient.GetContainerReference(containerName);

            // Create the container if it doesn't already exist
            container.CreateIfNotExists();

            // Enable public access to blob
            var permissions = container.GetPermissions();
            if (permissions.PublicAccess == BlobContainerPublicAccessType.Off)
            {
                permissions.PublicAccess = BlobContainerPublicAccessType.Blob;
                container.SetPermissions(permissions);
            }

            return container;
        }
    }


    public class FileItem
    {
        /// <summary>
        /// file name
        /// </summary>
        public string Name { get; set; }
        /// <summary>
        /// size in bytes
        /// </summary>
        public string SizeInMB { get; set; }
        public string ContentType { get; set; }
        public string Path { get; set; }
        public string BlobUploadCostInSeconds { get; set; }
    }


    public class AzureStorageMultipartFormDataStreamProvider : MultipartFileStreamProvider
    {
        private CloudBlobContainer _container;
        public AzureStorageMultipartFormDataStreamProvider(CloudBlobContainer container)
            : base(Path.GetTempPath())
        {
            _container = container;
            Files = new List<FileItem>();
        }

        public List<FileItem> Files { get; set; }

        public override Task ExecutePostProcessingAsync()
        {
            // Upload the files to azure blob storage and remove them from local disk
            foreach (var fileData in this.FileData)
            {
                var sp = new Stopwatch();
                sp.Start();
                string fileName = Path.GetFileName(fileData.Headers.ContentDisposition.FileName.Trim('"'));
                CloudBlockBlob blob = _container.GetBlockBlobReference(fileName);
                blob.Properties.ContentType = fileData.Headers.ContentType.MediaType;

                //set the number of blocks that may be simultaneously uploaded
                var requestOption = new BlobRequestOptions()
                {
                    ParallelOperationThreadCount = 5,
                    SingleBlobUploadThresholdInBytes = 10 * 1024 * 1024 ////maximum for 64MB,32MB by default
                };

                //upload a file to blob
                blob.UploadFromFile(fileData.LocalFileName, options: requestOption);
                blob.FetchAttributes();

                File.Delete(fileData.LocalFileName);
                sp.Stop();
                Files.Add(new FileItem
                {
                    ContentType = blob.Properties.ContentType,
                    Name = blob.Name,
                    SizeInMB = string.Format("{0:f2}MB", blob.Properties.Length / (1024.0 * 1024.0)),
                    Path = blob.Uri.AbsoluteUri,
                    BlobUploadCostInSeconds = string.Format("{0:f2}s", sp.ElapsedMilliseconds / 1000.0)
                });
            }
            return base.ExecutePostProcessingAsync();
        }
    }

结果如下：

Answer 3

在一个请求中从 Web API 获取所有文件的 SAS 令牌后，我会检查将媒体直接上传到 blob 存储。使用 Promise 和来自客户端的 http get 上传文件，这将并行上传。 这将是您正确的设计和方法。这也将提高您的上传速度并减少延迟。