Python：Pandas - 来自多个标题 excel 表的整洁数据框答案

【问题标题】：Python: Pandas - tidy data frame from multiple header excel sheetPython：Pandas - 来自多个标题 excel 表的整洁数据框
【发布时间】：2020-09-18 10:00:51
【问题描述】：

我有一个软件工具的 excel 表输出，该软件工具以以下多标题方式构造。 excel结构：

+---+-------+--------------+--------------+
|   |       |              |              |
+---+-------+--------------+--------------+
|   |       | not relevant | not relevant |
+---+-------+--------------+--------------+
|   |       | X1           | Y1           |
+---+-------+--------------+--------------+
|fr | Time  | not relevant | not relevant |
+---+-------+--------------+--------------+
| 1 | 0.000 | 12           | 32           |
+---+-------+--------------+--------------+
| 2 | 0.010 | 23           | 3            |
+---+-------+--------------+--------------+
| 3 | 0.020 | 45           | 4            |
+---+-------+--------------+--------------+
| 4 | 0.030 | 4            | 1            |
+---+-------+--------------+--------------+
|   |       |              |              |
+---+-------+--------------+--------------+
|   |       | not relevant |              |
+---+-------+--------------+--------------+
|   |       | Y2           |              |
+---+-------+--------------+--------------+
|fr | Time  | not relevant |              |
+---+-------+--------------+--------------+
| 1 | 0.000 | 5            |              |
+---+-------+--------------+--------------+
| 2 | 0.010 | 89           |              |
+---+-------+--------------+--------------+
| 3 | 0.020 | 5            |              |
+---+-------+--------------+--------------+
| 4 | 0.030 | 3            |              |
+---+-------+--------------+--------------+
|   |       |              |              |
+---+-------+--------------+--------------+
|   |       | not relevant |              |
+---+-------+--------------+--------------+
|   |       | X3           |              |
+---+-------+--------------+--------------+
|fr | Time  | not relevant |              |
+---+-------+--------------+--------------+
| 1 | 0.000 | 17           |              |
+---+-------+--------------+--------------+
| 2 | 0.010 | 2            |              |
+---+-------+--------------+--------------+
| 3 | 0.020 | 4            |              |
+---+-------+--------------+--------------+
| 4 | 0.030 | 23           |              |
+---+-------+--------------+--------------+

csv 结构：

,,,
,,not relevant,not relevant
,,X1,Y1
fr,Time,not relevant,not relevant
1,0.000,12,32
2,0.010,23,3
3,0.020,45,4
4,0.030,4,1
,,,
,,not relevant,
,,Y2,
fr,Time,not relevant,
1,0.000,5,
2,0.010,89,
3,0.020,5,
4,0.030,3,
,,,
,,not relevant,
,,X3,
fr,Time,not relevant,
1,0.000,17,
2,0.010,2,
3,0.020,4,
4,0.030,23,

我正在寻找一种快速的方法将这些杂乱的数据转换成整洁的 pandas 数据框。

每个子系列的时间戳值和数量相同。
子系列的数量是可变的。

最终结果应如下所示。

  Time    X1     Y1     Y2     X3  
  0.000   12     32     5      17    
  0.010   23     3      89     2     
  0.020   45     4      5      4     
  0.030   4      1      3      23

【问题讨论】：

在pd.read_excel 方法中查找skiprows 参数。您将能够轻松获得所需的输出。
@MayankPorwal 我知道skiprows，很容易用于跳过顶部的行，但这里的挑战是多个子系列在 excel 数据中连接在一起。我可能只是使用.split

标签： python excel pandas dataframe import

【解决方案1】：

我做了以下...不是很高兴，但它有效。

import numpy as np
import pandas as pd

filename = 'test_data'

df = pd.read_excel(filename + '.xlsx', header=None)

df_list = np.split(df, df[df.isnull().all(1)].index)

del df_list[0]

for i, df in enumerate(df_list):

    df.iloc[3, 2:] = df.iloc[2, 2:]

    new_header = df.iloc[3]
    df.columns = new_header

    df = df.iloc[4:]
    df_tmp = df.drop(['Frame'], axis=1)    
    df = df_tmp.set_index("Time")
    df.dropna(axis=1, how='all', inplace=True)    
    df.columns.name = None

    df_list[i] = df

df = pd.concat(df_list, axis=1)
df = df.reindex(sorted(df.columns), axis=1)

df.to_csv(filename + '.csv')

【讨论】：