Azure-CLI Blob 存储删除多个 Blob

Question

我需要使用 Azure-CLI 删除多个文件/blob。但它不受官方支持。所以我写了一个 for 循环来使用 az 命令删除多个文件。它看起来像这样：

for i in `az storage blob list --container-name $BUCKET --output table | awk '{print $1}'| sed '1,2d' | sed '/^$/d'` ; do
    if az storage blob delete --name $i --container-name $BUCKET --output table ; then
      echo "Files inside the bucket (container) ${BUCKET} have been erased"
    else
      echo "Unable to erase the files inside the bucket (container) ${BUCKET}." && exit 1
    fi
done

但这不能正常工作，我认为循环挂起。但我不知道为什么。

我希望任何人都可以帮助我。提前致谢。

编辑：

感谢您的帮助。但什么也没有发生。我正在使用 Xubuntu 17.04 并全新安装了 azure cli。

如果我按 CTRL+C，我会收到此消息：

Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open

    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 1253, in connect
    super().connect()
  File "/usr/lib/python3.5/http/client.py", line 849, in connect
(self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.5/socket.py", line 703, in create_connection
    sock.connect(sa)
KeyboardInterrupt

Original exception was:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 271, in <module>
  File "/usr/lib/python3/dist-packages/azure/cli/core/telemetry.py", line 263, in upload_telemetry
  File "/usr/lib/python3/dist-packages/applicationinsights/TelemetryClient.py", line 55, in flush
  File "/usr/lib/python3/dist-packages/applicationinsights/channel/TelemetryChannel.py", line 71, in flush
  File "/usr/lib/python3/dist-packages/applicationinsights/channel/SynchronousQueue.py", line 39, in flush
  File "/usr/lib/python3/dist-packages/applicationinsights/channel/SenderBase.py", line 118, in send
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 1297, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1107, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1152, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1103, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 1253, in connect
    super().connect()
  File "/usr/lib/python3.5/http/client.py", line 849, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.5/socket.py", line 703, in create_connection
    sock.connect(sa)
KeyboardInterrupt

在我运行这个脚本的地方完全没有必要。我总是遇到同样的错误:-(

--阿鲁卡

Answer 1

在我的测试中，你的脚本适合我，也许我们可以试试这个脚本：

export AZURE_STORAGE_ACCOUNT=jasontest321
export AZURE_STORAGE_ACCESS_KEY=+v6/ZQy74QhIwdvEMbnMV7HMaDaNUAkCDizBTOVdZkBUlw==
export container_name=vhds
for i in `az storage blob list --container-name $container_name --output table | awk '{print $1}'| sed '1,2d' | sed '/^$/d'` ; do
    if az storage blob delete --name $i --container-name $container_name >/dev/null ; then
      echo "Files $i inside the $container_name have been erased"
    else
      echo "Unable to erase the files $i inside the  $container_name." && exit 1
    fi
done

这是关于结果的屏幕截图：

更新：

#az --version
.
role (2.0.5)
sf (1.0.2)
sql (2.0.4)
storage (2.0.7)
vm (2.0.7)

Python (Linux) 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]

Python location '/root/lib/azure-cli/bin/python'
[root@jasoncli@jasonye ~]# python
Python 2.7.5 (default, Nov  6 2016, 00:28:07) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> 

在我的实验室中，我的根帐户版本是 2.7.5。我们应该为您的用户 aruka 检查您的 python 版本。

---

更新 2：
我们可以使用 Azure 门户 cloud shell 来测试这个脚本：

更新3：

root@jasonvm:~# history
    1  echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ wheezy main" |      sudo tee /etc/apt/sources.list.d/azure-cli.list
    2  sudo apt-key adv --keyserver packages.microsoft.com --recv-keys 417A0893
    3  sudo apt-get install apt-transport-https
    4  sudo apt-get update && sudo apt-get install azure-cli
    5  export AZURE_STORAGE_ACCOUNT=jasondisk321
    6  export AZURE_STORAGE_ACCESS_KEY=O1/xxxWRzOlWep78iRcxxx/htgxHImKxxytlsv7gnn0hg1aGvw==
    7  export container_name=vhds
    8  for i in `az storage blob list --container-name $container_name --output table | awk '{print $1}'| sed '1,2d' | sed '/^$/d'` ; do     if az storage blob delete --name $i --container-name $container_name >/dev/null ; then       echo "Files $i inside the $container_name have been erased";     else       echo "Unable to erase the files $i inside the  $container_name." && exit 1;     fi; done
    9  az login -u jason.ye@xxxx.onmicrosoft.com -p xxxx
   10  export AZURE_STORAGE_ACCOUNT=jasondisk321
   11  export AZURE_STORAGE_ACCESS_KEY=O1/2D1UxeGcOlWep78iRcR5PxxgxxxzRbWoytlsv7gnn0hg1aGvw==
   12  export container_name=vhds
   13  for i in `az storage blob list --container-name $container_name --output table | awk '{print $1}'| sed '1,2d' | sed '/^$/d'` ; do     if az storage blob delete --name $i --container-name $container_name >/dev/null ; then       echo "Files $i inside the $container_name have been erased";     else       echo "Unable to erase the files $i inside the  $container_name." && exit 1;     fi; done
   14  export AZURE_STORAGE_ACCOUNT=jasondisk321
   15  export AZURE_STORAGE_ACCESS_KEY=O1/2D1UC0xxx/ztABmMyQFCeGcWRzOlWep78iRcR5Pt/K9yKxMT/htgxHImKqgznzRbWoytlsv7gnn0hg1aGvw==
   16  export container_name=vhds
   17  for i in `az storage blob list --container-name $container_name --output table | awk '{print $1}'| sed '1,2d' | sed '/^$/d'` ; do     if az storage blob delete --name $i --container-name $container_name >/dev/null ; then       echo "Files $i inside the $container_name have been erased";     else       echo "Unable to erase the files $i inside the  $container_name." && exit 1;     fi; done
   18  history

Answer 2

一些提示（与问题的根源无关，这就是删除命令挂起的原因） - 您不必使用 sed/awk 来获取 blob 名称。有一种特定的输出格式 (tsv)，旨在为您提供对脚本更友好的输出（没有要排除的标题或其他分散注意力的东西）。结合--query，你可以选择你感兴趣的输出字段：

az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv

将为您提供给定容器中的 blob 列表。

就个人而言，如果我想并行处理许多项目并控制诸如我发出多少并发请求等事情，我倾向于使用 xargs。

    az storage blob list -c templates --account-name johanstetemplates --query "[].name" --output tsv | xargs -L1 -P10 -I{} az storage blob show -c templates --account-name johanstetemplates -n {}

（我将删除命令更改为显示，以防止用户盲目地复制/粘贴命令并在他们的所有 blob 都被删除时感到非常沮丧）

Answer 3

第一个命令（我自己的天蓝色）：

$ az storage blob list --container-name testbucket --output table
Name         Blob Type      Length  Content Type    Last Modified
-----------  -----------  --------  --------------  -------------------------
results.csv  BlockBlob         254  text/csv        2017-08-02T22:30:05+00:00

还有版本：

$ az --version
azure-cli (2.0.12)

acr (2.0.9)
acs (2.0.11)
appservice (0.1.11)
batch (3.0.3)
billing (0.1.3)
cdn (0.0.6)
cloud (2.0.7)
cognitiveservices (0.1.6)
command-modules-nspkg (2.0.1)
component (2.0.6)
configure (2.0.10)
consumption (0.1.3)
container (0.1.7)
core (2.0.12)
cosmosdb (0.1.11)
dla (0.0.10)
dls (0.0.11)
feedback (2.0.6)
find (0.2.6)
interactive (0.3.7)
iot (0.1.10)
keyvault (2.0.8)
lab (0.0.9)
monitor (0.0.8)
network (2.0.11)
nspkg (3.0.1)
profile (2.0.9)
rdbms (0.0.5)
redis (0.2.7)
resource (2.0.11)
role (2.0.9)
sf (1.0.5)
sql (2.0.8)
storage (2.0.11)
vm (2.0.11)

Python (Linux) 2.7.13 (default, Jan 19 2017, 14:48:08) 
[GCC 6.3.0 20170118]

Python location '/home/aruka/lib/azure-cli/bin/python'