Azure Batch Pool 启动任务以从 Blob FileShare 下载资源文件

Question

我是 Azure 新手，我正在尝试在这里构建一个简单的 Azure 批处理案例。

我正在堆积批处理池启动任务...

我在美国东部创建了一个批处理帐户和一个存储帐户，然后我在带有容器的存储帐户中创建了一个通用的文件共享。我手动更新了一个名为 Test.txt 的文件。

我要做的是在启动任务时让批处理池下载这个文件。

代码如下：

string storageConnectionString = String.Format("DefaultEndpointsProtocol=https;AccountName={0};AccountKey={1}", StorageAccountName, StorageAccountKey);

// Retrieve the storage account
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(storageConnectionString);
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
SharedAccessFilePolicy fileShareConstraint = new SharedAccessFilePolicy
{
    SharedAccessExpiryTime = DateTime.UtcNow.AddHours(48),
    Permissions = SharedAccessFilePermissions.Read,
};
var fileShare = fileClient.GetShareReference(inputContainerName);
var rootDir = fileShare.GetRootDirectoryReference();
var testFile = rootDir.GetFileReference("test.txt");
var sasUrl = fileShare.GetSharedAccessSignature(fileShareConstraint);
var fileUrl = string.Format("{0}{1}", testFile.StorageUri.PrimaryUri, sasUrl);

var list = new List<ResourceFile>();
var testResourceFile = new ResourceFile(fileUrl, "test.txt");

list.Add(testResourceFile );
await CreatePoolAsync(batchClient, PoolId, list);

然后是 CreatePoolAsync 方法：

private static async Task CreatePoolAsync(BatchClient batchClient, string poolId, IList<ResourceFile> resourceFiles)
{

    Console.WriteLine("Creating pool [{0}]...", poolId);

    // Create the unbound pool. Until we call CloudPool.Commit() or CommitAsync(), no pool is actually created in the
    // Batch service. This CloudPool instance is therefore considered "unbound," and we can modify its properties.

    //if(await batchClient.PoolOperations.GetPoolAsync(poolId) == null)
    //{

    CloudPool pool = batchClient.PoolOperations.CreatePool(
        poolId: poolId,
        targetDedicated: 1,                                                         // 3 compute nodes
        virtualMachineSize: "small",                                                // single-core, 1.75 GB memory, 225 GB disk
        cloudServiceConfiguration: new CloudServiceConfiguration(osFamily: "4"));   // Windows Server 2012 R2
    pool.MaxTasksPerComputeNode = 2;

    // Create and assign the StartTask that will be executed when compute nodes join the pool.
    // In this case, we copy the StartTask's resource files (that will be automatically downloaded
    // to the node by the StartTask) into the shared directory that all tasks will have access to.
    pool.StartTask = new StartTask
    {
        // Specify a command line for the StartTask that copies the task application files to the
        // node's shared directory. Every compute node in a Batch pool is configured with a number
        // of pre-defined environment variables that can be referenced by commands or applications
        // run by tasks.

        // Since a successful execution of robocopy can return a non-zero exit code (e.g. 1 when one or
        // more files were successfully copied) we need to manually exit with a 0 for Batch to recognize
        // StartTask execution success.

        CommandLine = "cmd /c (robocopy %AZ_BATCH_TASK_WORKING_DIR% %AZ_BATCH_NODE_SHARED_DIR%) ^& IF %ERRORLEVEL% LEQ 1 exit 0",
        //CommandLine = "cmd /c net use E: \\krisblob2.file.core.windows.net\\krisfilecontiner1 /u:krisblob2 aqTFKyPqcpeI3BrEnlx8RTBAmDaN5FK+mxpBtdgn3v6IT+IbPgDhVU4ojRA1wAmMpYPEHQ9Gzh/A1mAHtxNs+A==",
        //CommandLine = $@"cmd /c net use Z: \\{StorageAccountName}.file.core.windows.net\krisfilecontiner1 /u:{StorageAccountName} {StorageAccountKey}",
        //CommandLine = "cmd /c %AZ_BATCH_TASK_WORKING_DIR%\\WinPcap_4_1_3.exe /passive",
        ResourceFiles = resourceFiles,
        WaitForSuccess = true
    };
}     

inputcontainer 是我在文件共享中为容器指定的名称。

当我运行代码时，启动任务总是失败并出现错误：

BlobDownloadMisc 错误消息

在执行过程中遇到其他错误 下载指定的 Azure Blob 详细信息之一 HTTP 标头之一的格式不正确。 请求ID：944807de-001a-00bb-73ae-4ac746000000 时间：2016-11-30T02:04:59.8679984Z

谁能帮我解决这个问题？

谢谢！

Answer 1

Azure Batch resource files 只能来自 Azure Blob 存储，不能来自 Azure 文件存储。

您需要 net use 挂载共享并手动复制文件，或者将文件移动到 Azure Blob 存储。