如何通过 Python 脚本在 Google Cloud Storage 上上传字节图像

Question

我想通过 python 脚本将图像上传到 Google Cloud Storage。这是我的代码：

from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery

scopes = ['https://www.googleapis.com/auth/devstorage.full_control']
credentials = ServiceAccountCredentials.from_json_keyfile_name('serviceAccount.json', scop
es)
service = discovery.build('storage','v1',credentials = credentials)

body = {'name':'my_image.jpg'}

req = service.objects().insert(
   bucket='my_bucket', body=body,
   media_body=googleapiclient.http.MediaIoBaseUpload(
      gcs_image, 'application/octet-stream'))

resp = req.execute()

如果gcs_image = open('img.jpg', 'r') 代码可以正常工作并将我的图像正确保存在云存储中。如何直接上传字节图像？ （例如来自 OpenCV/Numpy 数组：gcs_image = cv2.imread('img.jpg')）

Answer 1

就我而言，我想将 PDF 文档从字节上传到云存储。

当我尝试以下操作时，它创建了一个包含我的字节字符串的文本文件。

blob.upload_from_string(bytedata)

为了使用字节字符串创建一个实际的 PDF 文件，我必须这样做：

blob.upload_from_string(bytedata, content_type='application/pdf')

我的字节数据是b64encoded，所以我也先b64decode。

Answer 2

如果你想从文件上传你的图片。

import os
from google.cloud import storage

def upload_file_to_gcs(bucket_name, local_path, local_file_name, target_key):
    try:
        client = storage.Client()
        bucket = client.bucket(bucket_name)
        full_file_path = os.path.join(local_path, local_file_name)
        bucket.blob(target_key).upload_from_filename(full_file_path)
        return bucket.blob(target_key).public_url

    except Exception as e:
        print(e)

    return None

但如果你想直接上传字节：

import os
from google.cloud import storage

def upload_data_to_gcs(bucket_name, data, target_key):
    try:
        client = storage.Client()
        bucket = client.bucket(bucket_name)
        bucket.blob(target_key).upload_from_string(data)
        return bucket.blob(target_key).public_url

    except Exception as e:
        print(e)

    return None

注意target_key是前缀和上传文件的名称。

Answer 3

MediaIoBaseUpload 需要一个类似io.Base 的对象并引发以下错误：

  'numpy.ndarray' object has no attribute 'seek'

收到 ndarray 对象后。为了解决它，我使用TemporaryFile 和numpy.ndarray().tofile()

from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient import discovery
import googleapiclient
import numpy as np
import cv2
from tempfile import TemporaryFile


scopes = ['https://www.googleapis.com/auth/devstorage.full_control']
credentials = ServiceAccountCredentials.from_json_keyfile_name('serviceAccount.json', scopes)
service = discovery.build('storage','v1',credentials = credentials)

body = {'name':'my_image.jpg'}
with TemporaryFile() as gcs_image:
    cv2.imread('img.jpg').tofile(gcs_image)
    req = service.objects().insert(
       bucket='my_bucket’, body=body,
       media_body=googleapiclient.http.MediaIoBaseUpload(
          gcs_image, 'application/octet-stream'))

    resp = req.execute()

请注意，googleapiclient 是非惯用的并且仅用于维护（不再开发）。我建议使用idiomatic one。

Answer 4

这里是如何直接从内存中上传 PIL 图像：

from google.cloud import storage
import io
from PIL import Image

# Define variables
bucket_name = XXXXX
destination_blob_filename = XXXXX

# Configure bucket and blob
client = storage.Client()
bucket = client.bucket(bucket_name)

im = Image.open("test.jpg")
bs = io.BytesIO()
im.save(bs, "jpeg")
blob.upload_from_string(bs.getvalue(), content_type="image/jpeg")

除此之外，这里是如何将 blobfiles 作为 PIL 图像直接下载到内存：

blob = bucket.blob(destination_blob_filename)
downloaded_im_data = blob.download_as_bytes()
downloaded_im = Image.open(io.BytesIO(downloaded_im_data))