从二进制文件导入数据很慢

Question

我有一个文件夹，里面装满了包含测试数据的 .txt 文件。我编写了一个宏来筛选 .txt 文件，以根据一些搜索条件提取我想要的信息，然后将这些搜索结果写入二进制文件。

所以现在我有一个包含精简数据集的二进制文件，我编写了另一个宏来搜索二进制文件中我真正想要的内容。

由于某种原因，我的宏从二进制文件中读取数据非常慢。

作为比较，我编写了一个宏，它查看所有 txt 文件以进行特定搜索，将其写入二进制文件，然后将其读回 Excel。这只花了60秒。

这是一个sn-p。我想知道 if-else 语句是否符合我的搜索条件 [LC(a) 和 EID(e)] 是否会减慢速度或二进制文件的大小（只有 200 MB）。

Type MyBinaryRecordInfo
   MyBinaryRecordInfo1(1 To 12) As String ' variable length
End Type    

i = 1
Open currentpath & "\" & bin_fname & ".DAT" For Binary As #f
' read records from the binary file
For a = 1 To totalLC
    For e = 1 To totalElm
        Do While Loc(f) < LOF(f)
            Call ReadBinRecord(MyRecord, f, ElmType)
            Sheets(ElmType).Select
            If MyRecord.MyBinaryRecordInfo1(1) = LC(a) Then
                If MyRecord.MyBinaryRecordInfo1(2) = EID(e) Then
                    For j = 1 To totalbinrec
                        With MyRecord
                            Cells(i + 3, j) = .MyBinaryRecordInfo1(j)
                        End With
                    Next j
                    i = i + 1
                    Exit Do
                End If
            End If
        Loop
    Next e
Next a
Close #f ' close the file

Sub ReadBinRecord(MyRecord As MyBinaryRecordInfo, f As Integer, ElmType As String)
' reads the next record from an open binary file
Dim intSize As Integer

For j = 1 To totalbinrec
   With MyRecord
       Get f, , intSize ' read the size of the ID field
       .MyBinaryRecordInfo1(j) = String(intSize, " ") ' set the variable length
       Get f, , .MyBinaryRecordInfo1(j) ' read the variable string field
   End With
Next j

Answer 1

可能速度较慢的部分是写入 Excel。特别是，您每次阅读该行时都会更改工作表

假设你事先知道二进制文件的行数，你可以读入内存，最后写出一次。代码如下所示：

Option Explicit

Type MyBinaryRecordInfo
   MyBinaryRecordInfo1(1 To 12) As String ' variable length
End Type

Sub x()
num_rows = 5000
Open currentpath & "\" & bin_fname & ".DAT" For Binary As #f
Dim V(1 To totalElm) As Variant
Dim V2(1 To num_rows, 1 To 12) As Variant
For e = 1 To totalElm
  V(e) = V2
Next e
' read records from the binary file
For a = 1 To totalLC
    For e = 1 To totalElm
        Do While Loc(f) < LOF(f)
            Call ReadBinRecord(MyRecord, f, ElmType)
            If MyRecord.MyBinaryRecordInfo1(1) = LC(a) Then
                If MyRecord.MyBinaryRecordInfo1(2) = EID(e) Then
                    For j = 1 To totalbinrec
                        With MyRecord
                            V(e)(i, j) = .MyBinaryRecordInfo1(j)
                        End With
                    Next j
                    i = i + 1
                    Exit Do
                End If
            End If
        Loop
    Next e
Next a
Close #f ' close the file

' write out
For e = 1 To totalElm
  Sheets(ElmType).Select
  Cells(3, 1).Resize(num_rows, 12).Value = V(e)
Next e
End Sub

Sub ReadBinRecord(MyRecord As MyBinaryRecordInfo, f As Integer, ElmType As String)
' reads the next record from an open binary file
Dim intSize As Integer

For j = 1 To totalbinrec
   With MyRecord
       Get f, , intSize ' read the size of the ID field
       .MyBinaryRecordInfo1(j) = String(intSize, " ") ' set the variable length
       Get f, , .MyBinaryRecordInfo1(j) ' read the variable string field
   End With
Next j
End Sub

如果您不知道行数，那么您只需 Redim 每 500 行左右保留一次内部变体。

由于这会在最后写出所有内容，您可能会发现使用 Application.Statusbar = "my string" 编写进度消息很有帮助