如何在 Python 3 中下载带有 https URL 的图像？

Question

我尝试用 Python 编写一个简短的海量下载器脚本来在本地存储图像列表。

它对于http 图像网址非常有效，但是无法下载任何带有https 网址的图像。有问题的代码行是：

import urllib.request
urllib.request.urlretrieve(url, filename)

例如， https://cdn.discordapp.com/attachments/299398003486097412/303580387786096641/FB_IMG_1490534565948.jpg 生成 HTTP Error 403: Forbidden 以及任何其他 https 图像。

这给我留下了两个问题：

1. 如何使用 Python 下载这样的图像？
2. 如果图像基本上只是文件，为什么还要有https url？

编辑：

这是堆栈跟踪：

Traceback (most recent call last):
  File "img_down.py", line 52, in <module>
    main()
  File "img_down.py", line 38, in main
    save_img(d, l)
  File "img_down.py", line 49, in save_img
    stream = read_img(url)
  File "img_down.py", line 42, in read_img
    with urllib.request.urlopen(url) as response:
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "D:\Users\Jan\AppData\Local\Programs\Python\Python36-32\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Answer 1

希望这会有所帮助。

import requests
with open('FB_IMG_1490534565948.jpg', 'wb') as f:
    f.write(requests.get('https://url/to/image.jpg').content)

Answer 2

可以帮助你...

我做了这个script，但从未完成（最终目的是让它每天自动运行）

但不要成为那种推迟答案的人，这是您感兴趣的一段代码：

    def downloadimg(self):
        import datetime
        imgurl = self.getdailyimg();
        imgfilename = datetime.datetime.today().strftime('%Y%m%d') + '_' + imgurl.split('/')[-1]
        with open(IMGFOLDER + imgfilename, 'wb') as f:
            f.write(self.readimg(imgurl))

希望对你有所帮助！

已编辑

PS：使用python3

完整脚本

#!/usr/bin/python
# -*- coding: utf-8 -*-

import os
IMGFOLDER = os.getcwd() + '/images/'


class BingImage(object):
    """docstring for BingImage"""
    BINGURL = 'http://www.bing.com/'
    JSONURL = 'HPImageArchive.aspx?format=js&idx=0&n=1&mkt=pt-BR'
    LASTIMG = None

    def __init__(self):
        super(BingImage, self).__init__()
        try:
            self.downloadimg()
        except:
            pass

    def getdailyimg(self):
        import json
        import urllib.request
        with urllib.request.urlopen(self.BINGURL + self.JSONURL) as response:
            rawjson = response.read().decode('utf-8')
            parsedjson = json.loads(rawjson)
            return self.BINGURL + parsedjson['images'][0]['url'][1:]

    def downloadimg(self):
        import datetime
        imgurl = self.getdailyimg();
        imgfilename = datetime.datetime.today().strftime('%Y%m%d') + '_' + imgurl.split('/')[-1]
        with open(IMGFOLDER + imgfilename, 'wb') as f:
            f.write(self.readimg(imgurl))
        self.LASTIMG = IMGFOLDER + imgfilename

    def checkfolder(self):
        d = os.path.dirname(IMGFOLDER)
        if not os.path.exists(d):
            os.makedirs(d)

    def readimg(self, url):
        import urllib.request
        with urllib.request.urlopen(url) as response:
            return response.read()


def DefineBackground(src):
    import platform
    if platform.system() == 'Linux':
        MAINCMD = "gsettings set org.gnome.desktop.background picture-uri"
        os.system(MAINCMD + ' file://' + src)


def GetRandomImg():
    """Return a random image already downloaded from the images folder"""
    import random
    f = []
    for (dirpath, dirnames, filenames) in os.walk(IMGFOLDER):
        f.extend(filenames)
        break
    return IMGFOLDER + random.choice(f)


if __name__ == '__main__':
    # get a new today's image from Bing
    img = BingImage()
    # check whether a new image was get or not
    if(img.LASTIMG):
        DefineBackground(img.LASTIMG)
    else:
        DefineBackground(GetRandomImg())
    print('Background defined')

Answer 3

这是该问题的最新答案，我使用 openCV 来存储图像以及请求模块，它还将处理批处理操作，可以添加为通用代码

import numpy as np
from urllib.request import urlopen
import cv2
import os
current_path = os.getcwd()
try: os.mkdir(current_path + "\\Downloaded\\")
except:pass

def downloadImage(url):
    try:
        print("Downloading %s" % (url))
        image_name = str(url).split('/')[-1]
        resp = urlopen(url)
        image = np.asarray(bytearray(resp.read()), dtype="uint8")
        image = cv2.imdecode(image, cv2.IMREAD_COLOR)
        cv2.imwrite(current_path + "\\Downloaded\\" + image_name, image)
    except Exception as error:
        print(error)

if __name__ == '__main__':
    urls = ["https://www.google.com/logos/doodles/2019/st-georges-day-2019-6234830302871552.20-2x.png"]
    for url in urls:
        downloadImage(url)

Answer 4

参考以下链接here，即使使用@Thiago Cardoso 的解决方案，您也可能会遇到“HTTP Error 403: Forbidden”错误，因为服务器不知道请求来自何处。有些网站会验证 UserAgent 以防止异常访问。所以你应该提供你的虚假浏览器访问的信息。

所以我修改了代码readimg方法如下：-

def readimg(self, img_url):
    from urllib.request import urlopen, Request
    headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.3'}
    req = Request(url=img_url, headers=headers) 
    with urlopen(req) as response:
        return response.read()

Answer 5

您需要创建一个 userAgent。 可能这是一个阻止未知用户代理的服务器安全功能。

如果你设置一个已知的浏览器用户代理，将起作用。

def download_img(img_url, img_name):
    request = Request(img_url, headers={'User-Agent': 'Mozilla/5.0'})
    response = urlopen(request)
    with open(img_name, "wb") as f:
       f.write(response.read())