【发布时间】：2022-11-09 13:21:22
【问题描述】：
I am trying to parse info from a multilingual site. I fail to grab information in English, the soup I make would always return info in Russian.
The link and my code are as follows.
'https://iherb.com/c/california-gold-nutrition'
`headers = {
"Accept-Language": "en",
"user-agent": "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36"
}
def make_soup(url):
r = requests.get(url=url, headers=headers)
r.encoding = 'utf-8'
return BeautifulSoup(r.text, 'lxml')
url = 'https://iherb.com/c/california-gold-nutrition'
with webdriver.Chrome() as browser:
browser.get(url)
menue_goer = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, \
'.language-select.hidden-xs.hidden-sm'))).click()
language = WebDriverWait(browser,5).until(EC.element_to_be_clickable((By.CSS_SELECTOR,
'.select-language.gh-dropdown'))).click()
English = WebDriverWait(browser,5).until(EC.element_to_be_clickable((By.CSS_SELECTOR,
".item.gh-dropdown-menu-item["data-val='en-US']"))).click()
save_button = WebDriverWait(browser,5).until(EC.element_to_be_clickable((By.XPATH,
"//button[@class='save-selection gh-btn gh-btn-primary']"))).click()
time.sleep(10)
soup = make_soup(url)
names = [x['title'].replace(u'\xa0', u' ') for x in soup.find('div', id='ProductsPage').find_all('a', class_='absolute-link product-link')]
print(names)`
So far I have tried to change lang settings using Selenium and play with headers, but alas none of them worked. Is there any way to change settings to a specific language?
-
Check with this locator - By.CSS_SELECTOR, ".item.gh-dropdown-menu-item["data-val='en-US']" , is this a correct one? You have to remove the double-quote before the text data-val, it should be like: ".item.gh-dropdown-menu-item[data-val='en-US']"
-
This is entirely up to the web site. If they provide a method for changing the language (and many sites to not), then you have to figure out how to select it.
-
@AbiSaran, Thank you, sir. I removed the double quote, but it wouldn't work anyway.
标签： python selenium parsing multilingual