【问题标题】:how do I unzip one large zip file(100gb+) present in a blob container to another blob container ,I get System.OutOfMemoryException如何将 blob 容器中存在的一个大 zip 文件(100gb+)解压缩到另一个 blob 容器,我得到 System.OutOfMemoryException
【发布时间】:2019-02-28 07:02:49
【问题描述】:
 StorageCredentials creden = new StorageCredentials(AccountNameAzure, AccessKeyAzure);
                CloudStorageAccount storageAccount = new CloudStorageAccount(creden, useHttps: true);
                CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
                //CloudBlobContainer cont = blobClient.GetContainerReference(SourceContainerName);

                //Get a reference to the container where we have our zip file as well as reference to the file itself.

                // Retrieve reference to a zip file container.
                CloudBlobContainer container = blobClient.GetContainerReference(SourceContainerName);

                // Retrieve reference to the blob - zip file which we wanted to extract 
                Console.WriteLine("SourceContainerName:" + SourceContainerName + "\n");
                CloudBlockBlob blockBlob = container.GetBlockBlobReference("ABCD.zip");
                //Now also get a reference to the container where you wanted to extract the files to and create the container in Storage Account if it is not exists already.

                //Retrieve reference to a container where you wanted to extract the zip file. 
                Console.WriteLine("TargetContainerName:"+ TargetContainerName);
                var extractcontainer = blockBlob.ServiceClient.GetContainerReference(TargetContainerName);
                extractcontainer.CreateIfNotExists();
                // As we have both source and target container references are setup, let’s download the zip file blob into a memory stream and and pass it on to ZipArchive class which is from System.IO.Compression namespace.

                // ZipArchive will take this memory stream as input and will provide a collection of entries property where in each entry represents an individual file in it.
                Console.WriteLine("Starting Unzip: "+DateTime.Now);
                // Save blob(zip file) contents to a Memory Stream.
                using (var zipBlobFileStream = new MemoryStream())
                {
                    blockBlob.DownloadToStream(zipBlobFileStream);
                    zipBlobFileStream.Flush();
                    zipBlobFileStream.Position = 0;
                    //use ZipArchive from System.IO.Compression to extract all the files from zip file
                    using (var zip = new ZipArchive(zipBlobFileStream))
                    {
                        //Each entry here represents an individual file or a folder
                        foreach (var entry in zip.Entries)
                        {
                            Console.WriteLine(entry.FullName+"\n");
                            Console.WriteLine("File unzip start: " + DateTime.Now+"\n");
                            //creating an empty file (blobkBlob) for the actual file with the same name of file
                            var blob = extractcontainer.GetBlockBlobReference(entry.FullName);
                            using (var stream = entry.Open())
                            {
                                //check for file or folder and update the above blob reference with actual content from stream
                                if (entry.Length > 0)
                                    blob.UploadFromStream(stream);
                            }
                            Console.WriteLine("File unzip end: " + DateTime.Now + "\n");
                        }
                    }
                }

【问题讨论】:

    标签: azure azure-storage azure-blob-storage unzip memorystream


    【解决方案1】:

    还没有找到解决办法。 而是使用 (var zipBlobFileStream = new FileStream(Path.GetTempFileName(),FileMode.Create,FileAccess.ReadWrite,FileShare.None,4096,FileOptions.DeleteOnClose)) 并将文件流下载到临时文件夹中

    【讨论】:

    • 下载到本地 vm 的 temp 。欢迎任何其他解决方案
    猜你喜欢
    • 2020-12-06
    • 1970-01-01
    • 2016-11-29
    • 2020-03-29
    • 2021-04-29
    • 2015-07-16
    • 2020-03-01
    • 2020-05-07
    • 2020-07-21
    相关资源
    最近更新 更多