【发布时间】:2020-05-16 00:40:12
【问题描述】:
我正在使用刮刀为我的数据库获取一些数据我正在使用此代码从其他站点添加产品
def scrape():
path=''
counter=0
session= requests.Session()
session.headers={
"User-Agent":"my user agent"
}
url='some url'
content=session.get(url,verify=False).content
soup=bs4.BeautifulSoup(content,'html.parser')
result=soup.find_all('div',{'class':'column column-block block-list-large single-item'})
for i in result:
counter+=1
name=i.find_all('h1',{'class':'itemTitle'})[0]
price=i.find('h3',{'class':'itemPrice'})
image=i.find('img',{'class':'img-size-medium imageUrl'})['data-src']
path=f'pics/{counter}.jpg'
img=path
barcode=f'name{counter}'
description='this is my product'
urllib.request.urlretrieve(image,path)
cat=category.objects.get(id=140)
br=branch.objects.get(id=8)
products.objects.create(name=name.text,Barcode=barcode,branch=br,image=img,
description=description,price=price,category=cat)
scrape()
它正在下载产品的图像,但在此之后出现错误
value = value.resolve_expression(self.query, allow_joins=False, for_save=True)
TypeError: 'NoneType' 对象不可调用
【问题讨论】:
标签: django python-3.x beautifulsoup urllib3