解码数据框中的一列并删除 "b'\xc2\xa" n "\xc2\xa0"

Question

我有两个问题。

1. 我所有的专栏都以字母“b”开头。我想摆脱这个字符并将所有值转换为浮点数。 （我附上了整个数据框的图像）。

1. 对于价格列，有这个额外的编码“\xc2\xa”。我想删除它并保留十进制值。 （我附上了这个专栏的图片）。

通过将列转换为字符串然后使用以下代码，我能够删除该列的“b”字符：

price.replace('b','')

但是当我用“\xc2\xa”尝试这段代码时，它不起作用。我还认为将所有列转换为字符串效率有点低，那么有什么更好的选择？

如果有帮助，这是我的全部代码：

import requests
import pandas as pd
from bs4 import BeautifulSoup

Base_url = ("https://www.nseindia.com/live_market/dynaContent/live_watch/fxTracker/optChainDataByExpDates.jsp")

page = requests.get(Base_url)

soup = BeautifulSoup(page.content, 'html.parser')
table_it = soup.find_all(class_="opttbldata")

spot = soup.select_one("div:contains('REFERENCE RATE') > strong").text
ATM = (round(float(spot)*4))/4
OTMCE = ATM + 0.50
OTMPE = ATM - 0.50

table_cls_1 = soup.find_all(id = "octable")
col_list = []

for mytable in table_cls_1:
    table_head = mytable.find('thead')

    try:
        rows = table_head.find_all('tr')
        for tr in rows:
            cols = tr.find_all('th')
            for th in cols:
                er = th.text
                ee = er.encode('utf-8')
                col_list.append(ee)
    except:
        print('no thread')

col_list_fnl = [e for e in col_list if e not in ('CALLS', 'PUTS', 'Chart', '\xc2\xa0')]

table_cls_2 = soup.find(id = "octable")
all_trs = table_cls_2.find_all('tr')
req_row = table_cls_2.find_all('tr')

df = pd.DataFrame(index=range(0,len(req_row)-3),columns = col_list_fnl)

row_marker = 0

for row_number, tr_nos in enumerate(req_row):
    if row_number <= 1 or row_number == len(req_row)-1:
        continue # To insure we only choose non empty rows

    td_columns = tr_nos.find_all('td')

    # Removing the graph column
    select_cols = td_columns[1:22]
    cols_horizontal = range(0,len(select_cols))

    for nu, column in enumerate(select_cols):

        utf_string = column.get_text()
        utf_string = utf_string.strip('\n\r\t": ')
        tr = utf_string.encode('utf-8')

        df.iloc[row_marker,[nu]] = tr

    row_marker += 1

print(df)

Answer 1

我根据@cs95 和@eyllanesc 的cmets 更改了您的代码。我可以毫无错误地执行代码，它会产生一个没有字节编码的数据帧。

import requests
import pandas as pd
from bs4 import BeautifulSoup

Base_url = ("https://www.nseindia.com/live_market/dynaContent/live_watch/fxTracker/optChainDataByExpDates.jsp")

page = requests.get(Base_url)

soup = BeautifulSoup(page.text, 'html.parser')
table_it = soup.find_all(class_="opttbldata")

spot = soup.select_one("div:contains('REFERENCE RATE') > strong").text
ATM = (round(float(spot)*4))/4
OTMCE = ATM + 0.50
OTMPE = ATM - 0.50

table_cls_1 = soup.find_all(id = "octable")
col_list = []

for mytable in table_cls_1:
    table_head = mytable.find('thead')

    try:
        rows = table_head.find_all('tr')
        for tr in rows:
            cols = tr.find_all('th')
            for th in cols:
                er = th.text
                col_list.append(er)
    except:
        print('no thread')

col_list_fnl = [e for e in col_list if e not in ('CALLS', 'PUTS', 'Chart', '\xc2\xa0')]

table_cls_2 = soup.find(id = "octable")
all_trs = table_cls_2.find_all('tr')
req_row = table_cls_2.find_all('tr')

df = pd.DataFrame(index=range(0,len(req_row)-3),columns = col_list_fnl)

row_marker = 0

for row_number, tr_nos in enumerate(req_row):
    if row_number <= 1 or row_number == len(req_row)-1:
        continue # To insure we only choose non empty rows

    td_columns = tr_nos.find_all('td')

    # Removing the graph column
    select_cols = td_columns[1:22]
    cols_horizontal = range(0,len(select_cols))

    for nu, column in enumerate(select_cols):

        utf_string = column.get_text()
        utf_string = utf_string.strip('\n\r\t": ')
        tr = utf_string

        df.iloc[row_marker,[nu]] = tr

    row_marker += 1

display(df)

打印出来：

加法

要将列正确转换为唯一名称并将值正确转换为浮点值，请执行以下操作：

cols = ['_first_col', 'Chart ', 'OI', 'Change in OI', 'Volume', 'IV', 'LTP', 'BidQty',
       'BidPrice', 'AskPrice_01', 'AskQty', 'Strike Price', 'BidQty', 'BidPrice',
       'AskPrice_02', 'AskQty', 'LTP', 'IV', 'Volume', 'Change in OI', 'OI',
       'Chart']
df.columns = cols

df.AskPrice_01 = df.AskPrice_01.apply(lambda x: float(x) if x != "-" else None)

df.AskPrice_02 = df.AskPrice_02.apply(lambda x: float(x) if x != "-" else None)

要过滤特定的列，您可以使用：

df[df.AskPrice_01 > 65.25].AskPrice_01

我希望这会有所帮助。祝你的项目好运！