How to scrape a website with many sibling elements, of which the number of them is uncertain? BeautifulSoup Python

Question

I want to scrape video game's release information. The structure of all relevant tags is shown in this picture (the specific case is from this link: https://www.mobygames.com/game/ps2/007-nightfire/release-info).

A screen shot from the above website link

What I want to do is to scrape all release information and store it in a dataframe.

The code I have so far is as following. However, I don't think how to finish the code as many things are undetermined. Is there a way to write such web scraper in a for-loop?

Here is my current code

def get_releases(url):
    response = requests.get(url + '/release-info', headers={"User-Agent": "Mozilla/5.0"})
    assert response.status_code == 200, "Problem with url request! %s throws %s" % (
        url,
        response.status_code,
    ) 
    page = response.text
    release_soup = BeautifulSoup(page, "lxml")
    return release_soup


def get_releases_info(release_soup):
    game_releases_info = defaultdict()
    
    title = release_soup.find('h1').findNext('a').text
    game_releases_info['title'] = title
    
    console = release_soup.find('h2').text
    game_releases_info['console'] = console
    
    release_list = release_soup.find('h2').findNextSiblings('div')
    num_cells = len(release_list)
    for tag in release_list:
        if tag.attrs == {'class': ['floatholder']}:
            field = tag.div.text.lower()
            value = tag.a.text
            game_releases_info[field] = value
        else: # not finishing...

Answer 1

Don't scrape MobyGames, as it makes the site slower for other users and you risk getting blocked. Request an API key to retrieve all this data you require.