【问题标题】:deploy cloudera on azure FAILED, error with JSON / ARM template function 'copyIndex'在 azure 上部署 cloudera 失败,JSON / ARM 模板函数“copyIndex”出错
【发布时间】:2018-12-16 09:29:26
【问题描述】:

我正在尝试使用下面的链接在 azure 上安装 cloudera https://github.com/Azure/azure-quickstart-templates/tree/master/cloudera-on-centos 它给了我错误 “消息”:“部署模板验证失败:'493'行和'9'列的模板资源'master-node'无效:此位置不需要模板函数'copyIndex'。该函数只能在指定副本的资源中使用。有关使用详情,请参阅https://aka.ms/arm-copy。有关使用详情,请参阅https://aka.ms/arm-template-expressions

代码如下所示

        "name": "master-node",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/shared-resources"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[concat(variables('scriptsUri'), '/master-node.json')]",
                    "contentVersion": "1.0.0.1",
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
                    "parameters": {
    "vnetID": {
      "type": "string"
    },
    "resourceAPIVersion": {
      "type": "string"
    },
    "dnsNamePrefix": {
      "type": "string",
      "defaultValue": "azuredevcluster"
    },
    "scriptsUri": {
      "type": "string"
    },
    "storageAccount": {
      "type": "object"
    },
    "vmCount": {
      "type": "int"
    },
    "vmSpec": {
      "type": "secureObject"
    },
    "networkSpec": {
      "type": "object"
    },
    "clusterSpec": {
      "type": "object"
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
                      "variables": {
    "singleQuote": "'",
    "masterIP": "[parameters('networkSpec').masterIP]",
    "workerIP": "[parameters('networkSpec').workerIP]",
    "vmName": "[concat(parameters('dnsNamePrefix'), '-mn')]",
    "nicName": "[concat(variables('vmName'), '-nic')]",
    "publicIPAddressName": "[concat(variables('vmName'), '-publicIP')]",
    "storageAccountName": "[parameters('storageAccount').prefix]",
    "securityGroupName": "[concat(parameters('dnsNamePrefix'), '-mn-sg')]"
  },
                    "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "name": "[variables('securityGroupName')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "SSH",
            "properties": {
              "description": "Allows SSH traffic",
              "protocol": "Tcp",
              "sourcePortRange": "*",
              "destinationPortRange": "22",
              "sourceAddressPrefix": "*",
              "destinationAddressPrefix": "*",
              "access": "Allow",
              "priority": 100,
              "direction": "Inbound"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[concat(variables('publicIPAddressName'), copyIndex())]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "publicIPLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "publicIPAllocationMethod": "Dynamic",
        "dnsSettings": {
          "domainNameLabel": "[concat(parameters('dnsNamePrefix'), '-mn', copyIndex())]"
        }
      }
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(variables('nicName'), copyIndex())]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'), copyIndex())]",
        "[concat('Microsoft.Network/networkSecurityGroups/', variables('securityGroupName'))]"
      ],
      "copy": {
        "name": "masterNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Static",
              "privateIPAddress": "[concat(parameters('networkSpec').ipOctet01, add(parameters('networkSpec').ipOctet2, div(copyIndex(parameters('networkSpec').ipOctet3), 256)), '.', mod(copyIndex(parameters('networkSpec').ipOctet3), 256))]",
              "subnet": {
                "id": "[concat(parameters('vnetID'), '/subnets/', parameters('networkSpec').virtualNetworkSubnetName)]"
              },
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(variables('publicIPAddressName'), copyIndex()))]"
              }
            }
          }
        ],
        "networkSecurityGroup": {
          "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('securityGroupName'))]"
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2017-03-30",
      "name": "[concat(variables('vmName'), copyIndex())]",
      "plan": "[parameters('vmSpec').imageInfo.plan]",
      "location": "[parameters('location')]",
      "copy":{
        "name": "masterNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex())]"
      ],
      "properties": {
        "availabilitySet": {
          "id": "[resourceId('Microsoft.Compute/availabilitySets', parameters('vmSpec').masterNodeASName)]"
        },
        "hardwareProfile": {
          "vmSize": "[parameters('vmSpec').vmSize]"
        },
        "osProfile": {
          "computerName": "[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",
          "adminUsername": "[parameters('vmSpec').adminUsername]",
          "adminPassword": "[parameters('vmSpec').adminPassword]",
          "linuxConfiguration": {
            "disablePasswordAuthentication": false
          }
        },
        "storageProfile": {
          "imageReference": "[parameters('vmSpec').imageInfo.imageReference]",
          "osDisk": {
            "name":"[concat(variables('vmName'), copyIndex(),'_OSDisk')]",
            "caching": "ReadWrite",
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk1')]",
              "diskSizeGB": 512,
              "lun": 0,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk2')]",
              "diskSizeGB": 512,
              "lun": 1,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk3')]",
              "diskSizeGB": 512,
              "lun": 2,
              "createOption": "Empty"
            },
            {
              "name":  "[concat(variables('vmName'), copyIndex(),'_DataDisk4')]",
              "diskSizeGB": 512,
              "lun": 3,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',concat(variables('nicName'), copyIndex()))]"
            }
          ]
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines/extensions",
      "name": "[concat(variables('vmName'), copyIndex(), '/prepareDisks')]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "nameNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'), copyIndex())]"
      ],
      "properties": {
        "publisher": "Microsoft.Azure.Extensions",
        "type": "CustomScript",
        "typeHandlerVersion": "2.0",
        "autoUpgradeMinorVersion": true,
        "settings": {
          "fileUris": [
            "[concat(parameters('scriptsUri'), '/scripts/prepare-masternode-disks.sh')]",
            "[concat(parameters('scriptsUri'), '/scripts/initialize-node.sh')]"
          ],
          "commandToExecute": "[concat('sh initialize-node.sh ', variables('singleQuote'), variables('masterIP'), variables('singleQuote'), ' ', variables('singleQuote'), variables('workerIP'), variables('singleQuote'), ' ', variables('singleQuote'), parameters('dnsNamePrefix'), variables('singleQuote'), ' ', variables('singleQuote'), reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').masterNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').dataNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('vmSpec').adminUsername, variables('singleQuote'), ' ', 'masternode', ' >> /var/log/cloudera-azure-initialize.log 2>&1')]"
        }
      }
    }
  ],
  "outputs": {
    "fqdn": {
      "type": "string",
      "value": "[reference(concat(variables('publicIPAddressName'), '0')).dnsSettings.fqdn]"
    }
  }
                },
                "parameters": {
                    "vnetID": {
                        "value": "[variables('VNetId')]"
                    },
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "storageAccount": {
                        "value": "[variables('masterStorageAccount')]"
                    },
                    "vmCount": {
                        "value": "[variables('clusterSpec').masterNodeCount]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    }
                }
            }
        },
        {
            "name": "data-node",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/shared-resources"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[variables('data-nodeUri')]",
                    "contentVersion": "1.0.0.1",
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json",
                    "parameters": {
    "vnetID": {
      "type": "string"
    },
    "dnsNamePrefix": {
      "type": "string",
      "defaultValue": "azuredevcluster"
    },
    "scriptsUri": {
      "type": "string"
    },
    "vmCount": {
      "type": "int"
    },
    "vmSpec": {
      "type": "secureObject"
    },
    "networkSpec": {
      "type": "object"
    },
    "clusterSpec": {
      "type": "object"
    },
    "location": {
      "type": "string",
      "defaultValue": "[resourceGroup().location]",
      "metadata": {
        "description": "Location for all resources."
      }
    }
  },
                     "variables": {
    "singleQuote": "'",
    "masterIP": "[parameters('networkSpec').masterIP]",
    "workerIP": "[parameters('networkSpec').workerIP]",
    "vmName": "[concat(parameters('dnsNamePrefix'), '-dn')]",
    "nicName": "[concat(variables('vmName'), '-nic')]",
    "publicIPAddressName": "[concat(variables('vmName'), '-publicIP')]",
    "securityGroupName": "[concat(parameters('dnsNamePrefix'), '-dn-sg')]"
  },
  "resources": [
    {
      "type": "Microsoft.Network/networkSecurityGroups",
      "name": "[variables('securityGroupName')]",
      "location": "[parameters('location')]",
      "properties": {
        "securityRules": [
          {
            "name": "SSH",
            "properties": {
              "description": "Allows SSH traffic",
              "protocol": "Tcp",
              "sourcePortRange": "*",
              "destinationPortRange": "22",
              "sourceAddressPrefix": "*",
              "destinationAddressPrefix": "*",
              "access": "Allow",
              "priority": 100,
              "direction": "Inbound"
            }
          }
        ]
      }
    },
    {
      "type": "Microsoft.Network/publicIPAddresses",
      "name": "[concat(variables('publicIPAddressName'), copyIndex())]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "publicIPLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "publicIPAllocationMethod": "Dynamic",
        "dnsSettings": {
          "domainNameLabel": "[concat(parameters('dnsNamePrefix'), '-dn', copyIndex())]"
        }
      }
    },
    {
      "type": "Microsoft.Network/networkInterfaces",
      "name": "[concat(variables('nicName'), copyIndex())]",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'), copyIndex())]",
        "[concat('Microsoft.Network/networkSecurityGroups/', variables('securityGroupName'))]"
      ],
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "properties": {
        "ipConfigurations": [
          {
            "name": "ipconfig1",
            "properties": {
              "privateIPAllocationMethod": "Static",
              "privateIPAddress": "[concat(parameters('networkSpec').ipOctet01, add(parameters('networkSpec').ipOctet2, div(copyIndex(parameters('networkSpec').datanodeIpOctet3), 256)), '.', mod(copyIndex(parameters('networkSpec').datanodeIpOctet3), 256))]",
              "subnet": {
                "id": "[concat(parameters('vnetID'), '/subnets/', parameters('networkSpec').virtualNetworkSubnetName)]"
              },
              "publicIPAddress": {
                "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(variables('publicIPAddressName'), copyIndex()))]"
              }
            }
          }
        ],
        "networkSecurityGroup": {
          "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('securityGroupName'))]"
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines",
      "apiVersion": "2017-03-30",
      "name": "[concat(variables('vmName'), copyIndex())]",
      "plan": {
        "name": "cloudera-centos-6",
        "publisher": "cloudera",
        "product": "cloudera-centos-6"
      },
      "location": "[parameters('location')]",
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'), copyIndex())]"
      ],
      "properties": {
        "availabilitySet": {
          "id": "[resourceId('Microsoft.Compute/availabilitySets', parameters('vmSpec').dataNodeASName)]"
        },
        "hardwareProfile": {
          "vmSize": "[parameters('vmSpec').vmSize]"
        },
        "osProfile": {
          "computerName": "[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",
          "adminUsername": "[parameters('vmSpec').adminUsername]",
          "adminPassword": "[parameters('vmSpec').adminPassword]",
          "linuxConfiguration": {
            "disablePasswordAuthentication": false
          }
        },
        "storageProfile": {
          "imageReference": {
            "publisher": "cloudera",
            "offer": "cloudera-centos-6",
            "sku": "cloudera-centos-6",
            "version": "latest"
          },
          "osDisk": {
            "name": "[concat(variables('vmName'), copyIndex(),'_OSDisk')]",
            "caching": "ReadWrite",
            "createOption": "FromImage"
          },
          "dataDisks": [
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk1')]",
              "diskSizeGB": 512,
              "lun": 0,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk2')]",
              "diskSizeGB": 1023,
              "lun": 1,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk3')]",
              "diskSizeGB": 1023,
              "lun": 2,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk4')]",
              "diskSizeGB": 1023,
              "lun": 3,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk5')]",
              "diskSizeGB": 1023,
              "lun": 4,
              "createOption": "Empty"
            },
            {
              "name": "[concat(variables('vmName'), copyIndex(),'_DataDisk6')]",
              "diskSizeGB": 1023,
              "lun": 5,
              "createOption": "Empty"
            }
          ]
        },
        "networkProfile": {
          "networkInterfaces": [
            {
              "id": "[resourceId('Microsoft.Network/networkInterfaces',concat(variables('nicName'), copyIndex()))]"
            }
          ]
        }
      }
    },
    {
      "type": "Microsoft.Compute/virtualMachines/extensions",
      "name": "[concat(variables('vmName'), copyIndex(), '/prepareDisks')]",
      "location": "[parameters('location')]",
      "copy": {
        "name": "dataNodeNicLoop",
        "count": "[parameters('vmCount')]"
      },
      "dependsOn": [
        "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'), copyIndex())]"
      ],
      "properties": {
        "publisher": "Microsoft.Azure.Extensions",
        "type": "CustomScript",
        "typeHandlerVersion": "2.0",
        "autoUpgradeMinorVersion": true,
        "settings": {
          "fileUris": [
            "[concat(parameters('scriptsUri'), '/scripts/prepare-datanode-disks.sh')]",
            "[concat(parameters('scriptsUri'), '/scripts/initialize-node.sh')]"
          ],
          "commandToExecute": "[concat('sh initialize-node.sh ', variables('singleQuote'), variables('masterIP'), variables('singleQuote'), ' ', variables('singleQuote'), variables('workerIP'), variables('singleQuote'), ' ', variables('singleQuote'), parameters('dnsNamePrefix'), variables('singleQuote'), ' ', variables('singleQuote'), reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').masterNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('clusterSpec').dataNodeCount, variables('singleQuote'), ' ', variables('singleQuote'), parameters('vmSpec').adminUsername, variables('singleQuote'), ' ', 'datanode', ' >> /var/log/cloudera-azure-initialize.log 2>&1')]"
        }
      }
    }
  ]
                },
                "parameters": {
                    "vnetID": {
                        "value": "[variables('VNetId')]"
                    },
                    "templateAPIVersion": {
                        "value": "[variables('templateAPIVersion')]"
                    },
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "storageAccount": {
                        "value": "[variables('workerStorageAccount')]"
                    },
                    "vmCount": {
                        "value": "[variables('clusterSpec').dataNodeCount]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    }
                }
            }
        },
        {
            "name": "setup-cloudera",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2018-11-01",
            "dependsOn": [
                "Microsoft.Resources/deployments/data-node",
                "Microsoft.Resources/deployments/master-node"
            ],
            "properties": {
                "mode": "Incremental",
                "template": {
                    "uri": "[concat(variables('scriptsUri'), '/setup-cloudera.json')]",
                    "contentVersion": "1.0.0.1"
                },
                "parameters": {
                    "resourceAPIVersion": {
                        "value": "[variables('resourceAPIVersion')]"
                    },
                    "dnsNamePrefix": {
                        "value": "[parameters('dnsNamePrefix')]"
                    },
                    "scriptsUri": {
                        "value": "[variables('scriptsUri')]"
                    },
                    "vmSpec": {
                        "value": "[variables('vmSpec')]"
                    },
                    "networkSpec": {
                        "value": "[variables('networkSpec')]"
                    },
                    "clusterSpec": {
                        "value": "[variables('clusterSpec')]"
                    },
                    "fqdn": {
                        "value": "[reference('master-node').outputs.fqdn.value]"
                    },
                    "cmUsername": {
                        "value": "[parameters('cmUsername')]"
                    },
                    "cmPassword": {
                        "value": "[parameters('cmPassword')]"
                    },
                    "company": {
                        "value": "[parameters('Company')]"
                    },
                    "emailAddress": {
                        "value": "[parameters('emailAddress')]"
                    },
                    "businessPhone": {
                        "value": "[parameters('businessPhone')]"
                    },
                    "firstName": {
                        "value": "[parameters('firstName')]"
                    },
                    "lastName": {
                        "value": "[parameters('lastName')]"
                    },
                    "jobRole": {
                        "value": "[parameters('jobRole')]"
                    },
                    "jobFunction": {
                        "value": "[parameters('jobFunction')]"
                    },
                    "installCDH": {
                        "value": "[variables('installCDH')]"
                    }
                }
            }
        }
    ]
}

【问题讨论】:

  • 您使用哪个链接\模板来部署?它很奇怪,它的解决方案不起作用
  • 感谢您的及时回复。我很感激。这里没有足够的空间来包含整个代码,所以我只包含了错误开始的部分代码。我部署github.com/Azure/azure-quickstart-templates/blob/master/…;这调用主节点部署github.com/Azure/azure-quickstart-templates/blob/master/…; copyindex 错误来自哪里
  • 这里的东西太多了,我建议你联系业主,让他们知道他们的东西坏了
  • 你的意思是代码有很多问题吗?你能举个例子吗?我遇到了一些错误并解决了这些错误。我遇到的最后一个错误是这个。我可以在哪里向您发送我的 JSON 文件?
  • pastebin 或类似的东西?例如:他们没有附加到资源的 api 版本。那永远行不通

标签: json azure cloudera


【解决方案1】:

是的,我找到了问题(但遗憾的是,在那之后可能会出现更多问题)。问题来自您使用嵌套的 INLINE 模板以及它们如何与引用函数一起工作(或者更确切地说不是):

"[reference(concat(variables('publicIPAddressName'), copyIndex())).dnsSettings.fqdn]",

这是损坏的部分。所以我的建议是要么把它做成一个单一的模板,不嵌套它们,要么将它们转换成一个完整的嵌套模板(使用templateLink 属性,而不是template 属性:

{
    "name": "name",
    "type": "Microsoft.Resources/deployments",
    "apiVersion": "2017-05-10",
    "properties": {
        "mode": "Incremental",
        "templateLink": {
            "uri": "url_goes_here"
        },
        "parameters": {
            "prefix": {
                "value": "xxx"
            }
        }
    }
},

我建议您永远不要使用嵌套的 INLINE 模板。他们太前卫了。

【讨论】:

  • 非常感谢 :) 您接受修复此代码的工作吗?
  • 当然 ;) 你可以在我的个人资料中找到指向我博客的链接,并且有一个关于联系人的页面
  • 非常感谢,上面的答案对stackoverflow.com/questions/53928022/…下面的错误没有帮助
猜你喜欢
  • 2022-09-30
  • 1970-01-01
  • 1970-01-01
  • 2019-04-19
  • 2020-08-31
  • 2021-04-10
  • 1970-01-01
  • 2022-08-20
  • 2019-02-11
相关资源
最近更新 更多