加快循环时间迭代 Python 中相对较大的 Excel 工作表

Question

当我为每一行测试的数字与我创建的预定义范围相匹配时，我正在尝试从 Excel 工作表的每一行中提取数据。该代码有效，但移动缓慢。我正在尝试遍历包含 200,000 行数据的工作表，并注意到当我将工作表分解为更小的部分时，程序执行得更快，但仍然没有我想要的那么快。我也不太明白为什么会这样。

关于如何加快速度的任何建议？

import xlsxwriter
import openpyxl
import os
from tqdm import tqdm

os.chdir(r'C:\Users\JTNeumay\Desktop\test folder')
xlfile = 'test.xlsx'
wb = openpyxl.load_workbook(xlfile, read_only=True)
sheet = wb['Sheet1']

newbook = xlsxwriter.Workbook('Commercial.xlsx')
newbook2 = xlsxwriter.Workbook('Industrial.xlsx')
newsheet = newbook.add_worksheet()
newsheet2 = newbook2.add_worksheet()
i = 1
j = 1

for row in tqdm(range(1, sheet.max_row + 1)):
    check = sheet.cell(row=row, column=11).value
    if check is not None:
        if (220000 <= check <= 221310 or 221320 <= check <= 221330 or 237200 <= check <= 237210 or 334611 <= check
                <= 334612 or 420000 <= check <= 493190 or 511210 <= check <= 512210 or 512240 <= check <= 541310 or
                541330 <= check <= 541340 or 541370 <= check <= 541700 or 541720 <= check <= 562900 or 562920 <= check
                <= 811430 or 812000 <= check <= 983000):
            for column in range(1, sheet.max_column + 1):
                newsheet.write(i, column, sheet.cell(row=row, column=column).value)
            i += 1
        elif (210000 <= check <= 213115 or 230000 <= check <= 237130 or 237300 <= check <= 334610 or 334613 <=
                check <= 339999 or 510000 <= check <= 511200 or 512220 <= check <= 512230 or check == 541320 or
                check == 541360 or check == 541710 or check == 562910 or check == 811490):
            for column in range(1, sheet.max_column + 1):
                newsheet2.write(i, column, sheet.cell(row=row, column=column).value)
            j += 1
        else:
            pass
    else:
        pass
newbook.close()
newbook2.close()

请注意，我还使用了一种方法，在该方法中，我创建了两个列表，其中包含 if 语句中的范围，并且发现该方法花费的时间要长得多。

Answer 1

问题：加快循环时间迭代

试试这个openpyxl only解决方案：

回来报告你的速度体验。

# Create new Workbooks
wb1 = Workbook()
ws1 = wb1.active
wb2 = Workbook()
ws2 = wb2.active

# Predefine Ranges and Target Worksheet
# You can gain additional speed, by sorting the range tuples 
# of the most expected Ranges, to the beginning.
ranges = [(2, 3, ws1), (221320, 221330, ws2), (237200, 237210, ws1), (812000, 983000, ws2)]

# The Row to start
min_row = 2

# Iterate all Cells in Column 'min/max_col'
for row, cell in enumerate(next(ws.iter_cols(min_row=min_row, min_col=2, max_col=2)), min_row):

    if cell.value:
        # Condition: If cell.value in Range(...)
        for r in ranges:
            if (r[0] <= cell.value <= r[1]):
                # Match: Append this 'row' to the given Worksheet
                r[2].append([cell.value for cell in next(ws.iter_rows(min_row=row, max_row=row))])
                break

# Save Worksheets
wb1.save('../test/test1.xlsx')
wb2.save('../test/test2.xlsx')

使用 Python 测试：3.4.2 - openpyxl：2.4.1 - LibreOffice：4.3.3.2