Get all value and items from drop down list using selenium

Question

I am trying to extract values from dropdown using python selenium. I am getting the text but not getting the values with xpath. Code I used is

from selenium.common.exceptions import WebDriverException
from selenium import webdriver

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.3"
}


options = webdriver.ChromeOptions()
options.add_argument("--headless")
options.add_argument('--no-sandbox')
options.add_argument('--disable-dev-shm-usage')

URL = ['https://www.classicalmusicartists.com/cma/artists.aspx']
for url in URL:
    try:
        driver = webdriver.Chrome(executable_path = '/home/ubuntu/selenium_drivers/chromedriver', options = options)
        driver.get(url)
        driver.implicitly_wait(2)
        datas = driver.find_element("xpath",'//select[@id="ctl00_cphMainContent_lstCategory"]')
        d= Select(datas)
        for opt in d.options:
            print(opt.text)  
        driver.quit()
    except WebDriverException:
        driver.quit()

Answer 1

So what you should do is:

ds = [d.text for d in datas.find_elements('tag name','option')]

you were using the improper tag locator. Options are tags not names, the 'name=' attribute (similar to class names) inside a tag element. Secondly you were looking for a singular item and then iterating over that element (as it was a single item there isn't any support for iteration over it to retrieve the text element) so had to make it .find_elements() to find all such matches.

And to get the values you could do:

dv = [d.get_attribute('value') for d in datas.find_elements('tag name','option')]