Azure 文件共享同步

Question

我正在寻找在 Azure 存储帐户中跨两个文件共享同步文件的最佳方式。 到目前为止，看起来 AZCopy 是最好的选择，但在特定情况下或过滤器的情况下它将不起作用。

例如：

• 我想复制在某个日期（上周、一个月）之后添加的文件 等）。
• 跳过一些文件夹/子文件夹

使用 Powershell 也是一种选择，但同步大量文件可能会很慢。

您是否有一些基于 AZCopy 的脚本或包装器可以帮助覆盖按文件夹和文件类型进行过滤？

谢谢 伊霍尔

Answer 1

据我所知，Azure 本身无法保持两个 Azure 文件共享同步。

作为一种解决方法，您需要在 Azure 上构建一个 VM，并将这两个共享挂载在上面。然后你只需要找到一个可以保持两个文件夹同步的解决方案。在 Windows 和 Linux 上都有多种解决方案。

Answer 2

这里是同步一些文件夹的脚本：

function Copy-AzureFileStorageWithPS{
<#
.SYNOPSIS
Copy a file share from a from one storage account to another using PowerShell

.DESCRIPTION
This function will copy a file share or specific folder inside of file share from one storage account to another

.PARAMETER SourceStorageAccountName

.PARAMETER DestStorageAccountName

.PARAMETER SourceResourceGroupName

.PARAMETER DestResourceGroupName

.PARAMETER FoldersToSkip

.PARAMETER FoldersToCopy

.EXAMPLE
. .\Copy-AzureFileStorageWithPS -SourceStorageAccountName  "some" -DestStorageAccountName  "other" -SourceResourceGroupName  "RG" -DestResourceGroupName  "OtherRg" 
#>

[CmdletBinding()]
param(
    [Parameter (Mandatory = $true)]
    [string] $SourceStorageAccountName,

    [Parameter (Mandatory = $true)]
    [string] $DestStorageAccountName,

    [Parameter (Mandatory = $true)]
    [string] $SourceResourceGroupName,

    [Parameter (Mandatory = $true)]
    [string] $DestResourceGroupName,

    [Parameter (Mandatory = $true)]
    [string] $FoldersToSkip = "",

    [Parameter (Mandatory = $true)]
    [string] $FoldersToCopy = ""
)

$exclusions = $FoldersToSkip.ToLower().Split(',').Trim()
$copyList = $FoldersToCopy.ToLower().Split(',').Trim()

$SourceStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $SourceStorageAccountName -ResourceGroupName $SourceResourceGroupName)[0].Value
$DestStorageAccountKey = (Get-AzureRmStorageAccountKey -Name $DestStorageAccountName -ResourceGroupName $DestResourceGroupName)[0].Value

$SourceContext = New-AzureStorageContext -StorageAccountName $SourceStorageAccountName -StorageAccountKey $SourceStorageAccountKey
$DestContext = New-AzureStorageContext -StorageAccountName $DestStorageAccountName -StorageAccountKey $DestStorageAccountKey

#Get all shares from Source SA
$shares  = Get-AzureStorageshare -Context $SourceContext

foreach($share in $shares){
    $destShare = Get-AzureStorageshare -Context $DestContext -Name $share.Name -ErrorAction SilentlyContinue

    if(-Not $destShare)
    {
        $destShare = New-AzureStorageShare $share.Name -Context $DestContext            
        Write-Host "Successfully created share $($share.Name) in Account $($DestContext.StorageAccountName)"
    }

    $shareContent = Get-AzureStorageFile -Share $share     

    foreach($content in $shareContent)
    {
         if(-Not $exclusions.Contains($content.Name.ToLower())){
            if($copyList.Length -ne 0){
                if($copyList.Contains($content.Name.ToLower())){
                    Write-Output "Starting copy of $($content.Name) from $($content.Uri.AbsoluteUri) to $($destShare.Uri.AbsoluteUri+"/"+$content.Name)"
                    Copy-AonAzureFolders -SourceShare $share -SourceFolder $content -DestShare $destShare -SourceContext $SourceContext -DestContext $DestContext
                }
            }
            else
            {
                Write-Output "Starting copy of $($content.Name) from $($content.Uri.AbsoluteUri) to $($destShare.Uri.AbsoluteUri+"/"+$content.Name)"
                Copy-AonAzureFolders -SourceShare $share -SourceFolder $content -DestShare $destShare -SourceContext $SourceContext -DestContext $DestContext
            }
        }
        else{
                Write-Output "Excluding folder $($content.Name) from copying to $($destShare.Name)"
        }
    }
}
}


function Copy-AonAzureFolders
{
[CmdletBinding()]
param(
    [Parameter (Mandatory = $true)]
    [Microsoft.WindowsAzure.Storage.File.CloudFileShare] $SourceShare,

    [Parameter (Mandatory = $true)]
    [Microsoft.WindowsAzure.Storage.File.CloudFileDirectory] $SourceFolder,

    [Parameter (Mandatory = $true)]
    [Microsoft.WindowsAzure.Storage.File.CloudFileShare] $DestShare,

    [Parameter (Mandatory = $true)]
    [Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext] $SourceContext,

    [Parameter (Mandatory = $true)]
    [Microsoft.WindowsAzure.Commands.Common.Storage.AzureStorageContext] $DestContext
)
$pathWithoutShare = $($SourceFolder.Uri.LocalPath).Replace("/$($SourceShare.Name)","")

$shareFolders = Get-AzureStorageFile -Share $SourceShare -Path $pathWithoutShare | Get-AzureStorageFile | Where-Object {$_.GetType().Name -eq "CloudFileDirectory"}

$folderExist = Get-AzureStorageFile -Share $DestShare -Path $pathWithoutShare -ErrorAction SilentlyContinue
if(-Not $folderExist){
    $newFolder = New-AzureStorageDirectory -Share $DestShare -Path $pathWithoutShare
}

foreach($folder in $shareFolders)
{
    #Recursive call    
    Copy-AonAzureFolders -SourceShare $SourceShare -SourceFolder $folder -DestShare $DestShare -SourceContext $SourceContext -DestContext $DestContext       
}

$shareFiles = Get-AzureStorageFile -Share $SourceShare -Path $($pathWithoutShare+"/") | Get-AzureStorageFile | Where-Object {$_.GetType().Name -eq "CloudFile"}
foreach($file in $shareFiles)
{
    $fullFilePath = $pathWithoutShare + "/" + $($file.Name)

    Write-Host "-SrcShareName $($SourceShare.Name) -SrcFilePath $fullFilePath -DestShareName $($DestShare.Name) -DestFilePath $fullFilePath"
    # copy a file to the new directory
    $copyfile = Start-AzureStorageFileCopy -SrcShareName $($SourceShare.Name) -SrcFilePath $fullFilePath `
                                           -DestShareName $($DestShare.Name) -DestFilePath $fullFilePath -Context $SourceContext -DestContext $DestContext -Force

    $status = $copyfile | Get-AzureStorageFileCopyState
    while ($status.Status -eq "Pending")
    {
        $status = $copyfile | Get-AzureStorageFileCopyState
        Start-Sleep -Milliseconds 150
    }
    $copyfile
}    
}

希望对你有所帮助 但是很慢

Answer 3

一个用 Python 编写但可以使用 Docker 启动的实用程序：

https://github.com/dokkur/azure-fileshare-sync

运行：

AZURE_STORAGE_ACCOUNT=account\
AZURE_STORAGE_ACCESS_KEY=key\
python -m src.azure_sync ./local/path sharename[/remote/path]