读取 pandas / python 中的 xls 文件：不支持的格式，或损坏的文件：预期的 BOF 记录；找到 b'\xef\xbb\xbf<?xml'

Question

我正在尝试将xls 文件（只有一个选项卡）打开到 pandas 数据框中。

这是一个我通常可以在 excel 或 excel for web 中读取的文件，实际上这是原始文件本身：https://www.dropbox.com/scl/fi/zbxg8ymjp8zxo6k4an4dj/product-screener.xls?dl=0&rlkey=3aw7whab78jeexbdkthkjzkmu。

我注意到前两行合并了单元格，一些列也是如此。

我尝试了几种方法（从堆栈中），但都失败了。

# method 1 - read excel
file = "C:\\Users\\admin\\Downloads\\product-screener.xls"
df = pd.read_excel(file)
print(df)

错误：Excel file format cannot be determined, you must specify an engine manually.

# method 2 - pip install xlrd and use engine
file = "C:\\Users\\admin\\Downloads\\product-screener.xls"
df = pd.read_excel(file, engine='xlrd')
print(df)

错误：Unsupported format, or corrupt file: Expected BOF record; found b'\xef\xbb\xbf<?xml'

# method 3 - rename to xlsx and open with openpyxl
file = "C:\\Users\\admin\\Downloads\\product-screener.xlsx"
df = pd.read_excel(file, engine='openpyxl')
print(df)

错误：File is not a zip file （可能转换，而不是重命名，是一种选择）。

# method 4 - use read_xml
file = "C:\\Users\\admin\\Downloads\\product-screener.xls"
df = pd.read_xml(file)
print(df)

此方法实际上会产生一个结果，但会产生一个与工作表无关的 DataFrame。大概需要解释 xml（似乎很复杂）？

   Style       Name  Table
0    NaN       None    NaN
1    NaN  All funds    NaN


# method 5 - use read_table
file = "C:\\Users\\admin\\Downloads\\product-screener.xls"
df = pd.read_table(file)
print(df)

此方法将文件读入一列（系列）DataFrame。那么如何使用这些信息来创建与 xls 文件形状相同的标准 2d DataFrame 呢？

0       <Workbook xmlns="urn:schemas-microsoft-com:off...
1                                                <Styles>
2                                 <Style ss:ID="Default">
3                          <Alignment Horizontal="Left"/>
4                                                </Style>
...                                                   ...
226532                                            </Cell>
226533                                             </Row>
226534                                           </Table>
226535                                       </Worksheet>
226536                                        </Workbook>



# method 5 - use read_html
file = "C:\\Users\\admin\\Downloads\\product-screener.xls"
df = pd.read_html(file)
print(df)

这会返回一个空白列表[]，而人们可能期望至少有一个 DataFrame 列表。

所以问题是将这个文件读入数据框（或类似的可用格式）的最简单方法是什么？

Answer 1

不是一个完整的解决方案，但它应该可以帮助您入门。 "xls" 文件实际上是 SpreadsheetML 格式的普通 xml 文件。将文件扩展名更改为.xml 并在您的互联网浏览器中查看它，其结构（至少是给定文件的结构）相当简单。

以下将数据内容读入pandas DataFrame：

import pandas as pd
import xml.etree.ElementTree as ET

tree = ET.parse('product-screener.xls')
root = tree.getroot()

data = [[c[0].text for c in r] for r in root[1][0][2:]]
types = [c[0].get('{urn:schemas-microsoft-com:office:spreadsheet}Type') for c in root[1][0][2]]

df = pd.DataFrame(data)
df = df.replace('-', None)
for c in df.columns:
    if types[c] == 'Number':
        df[c] = pd.to_numeric(df[c])
    elif types[c] == 'DateTime':
        df[c] = pd.to_datetime(df[c])

由于合并的单元格，从第 0 行和第 1 行获取列名有点复杂 - 我把它留给读者作为练习?。

Answer 2

我将在此处发布完整的解决方案，其中包含上述批准的解决方案（由@Stef 提供）以及将标头最终添加到 DataFrame 中。

'''
get xls file
convert to xml
parse into dataframe
add headers

'''


import pandas as pd
import xml.etree.ElementTree as ET

import shutil

file_xls = "C:\\Users\\admin\\Downloads\\product-screener.xls"
file_xml = 'C:\\Users\\admin\\Downloads\\product-screener.xml'

shutil.copyfile(file_xls, file_xml)

tree = ET.parse(file_xml)
root = tree.getroot()

data = [[c[0].text for c in r] for r in root[1][0][2:]]
types = [c[0].get('{urn:schemas-microsoft-com:office:spreadsheet}Type') for c in root[1][0][2]]

df = pd.DataFrame(data)
df = df.replace('-', None)
for c in df.columns:
    if types[c] == 'Number':
        df[c] = pd.to_numeric(df[c])
    elif types[c] == 'DateTime':
        df[c] = pd.to_datetime(df[c])

print(df)


headers = [[c[0].text for c in r] for r in root[1][0][:2]]
# print(headers[0])
# print(len(headers[0]))
# print()
# print(headers[1])
# print(len(headers[1]))
# print()

# upto column (AF) comes from headers[0]
df_headers = headers[0][0:32]
# the next 9 are discrete
x_list = ['discrete: ' + s for s in headers[1][0:9]  ]
df_headers = df_headers + x_list

# the next 10 are annualised
x_list = ['annualised: ' + s for s in headers[1][9:19]  ]
df_headers = df_headers + x_list

# the next 10 are cumulative
x_list = ['cumulative: ' + s for s in headers[1][19:29]  ]
df_headers = df_headers + x_list

# the next 9 are calendar
x_list = ['calendar: ' + s for s in headers[1][29:38]  ]
df_headers = df_headers + x_list

# the next 5 are portfolio characteristics (metrics)
x_list = ['metrics: ' + s for s in headers[1][38:43]  ]
df_headers = df_headers + x_list

# the next 6 are portfolio characteristics
x_list = ['characteristics: ' + s for s in headers[1][43:49]  ]
df_headers = df_headers + x_list

# the final 5 are sustainability characteristics
x_list = ['sustain: ' + s for s in headers[1][49:54]  ]
df_headers = df_headers + x_list
print(df_headers)

# add headers to dataframe
df.columns = df_headers
print(df)