合并 csv 文件但删除最后一行和多列

Question

我最初的问题是将多个 csv 合并为一个，分别删除前几行（在我的情况下为 5）和所有空白行。我能够找到以下解决方案：

@echo off
>Output.csv(
    for %%f in (*.csv) do (
        for /f "delims=" %%l in ('more +5 %%f') do (
            echo %%f,%%l
        )
    )
)

我还不能重写代码来删除每个文件的最后一行，并从 output.csv 文件中删除几列（或单个列）。

这是一个 csv 文件的示例：

Timecard Report
06/12/2017 - 06/12/2017
Departments : All_Departments-TOTAL HOURS

EMPLOYEE NAME,EMPLOYEE PAYROLL ID,FIRST NAME,LAST NAME,DEPARTMENT NAME,REG,REG Pay,OT1 Hours,OT1 Pay,OT2 Hours,OT2 Pay,VAC Hours,VAC Pay,HOL Hours,HOL Pay,SIC Hours,SIC Pay,OTH Hours,OTH Pay,TOTAL Hours,Total Pay 
Oc Br,999,Oc,Br,Fulfillment,8.00,114.8,.53,11.41,,,,,,,,,,,8.53,126.21 
Brat Hat,3423,Brat,Hat,Logistics Admin,5.42,75.88,,,,,,,,,,,,,5.42,75.88 
Tod Vindo,,Tod,Vindo,Logistics Admin,8.00,128,1.18,28.32,,,,,,,,,,,9.18,156.32 

TOTAL,,,,,73.53,1143.25,3.30,73.23,,,,,,,,,,,76.83,1216.48 

谁有想法？

Answer 1

新的 powershell 答案。这个脚本：

$KeepCols = @(
"EMPLOYEE NAME",
"EMPLOYEE PAYROLL ID",
"FIRST NAME",
"LAST NAME",
"DEPARTMENT NAME",
"REG",
"REG Pay",
"OT1 Hours",
"OT1 Pay",
"TOTAL Hours",
"Total Pay ")

Get-ChildItem '*.csv' -Exclude '*_dq.csv'|
  ForEach-Object {
    $fn=$_.Fullname
    "Processing $fn"
    (Get-Content $fn) | Select-Object -Skip 4 | ConvertFrom-Csv|
    Where-Object "EMPLOYEE NAME" -ne "TOTAL"|
    Select-Object -Property $KeepCols|
      Export-Csv -path ($fn.replace('.csv','_dq.csv')) -NoType
  }

将从您的上述示例中生成此输出：

"EMPLOYEE NAME","EMPLOYEE PAYROLL ID","FIRST NAME","LAST NAME","DEPARTMENT NAME","REG","REG Pay","OT1 Hours","OT1 Pay","TOTAL Hours","Total Pay "
"Oc Br","999","Oc","Br","Fulfillment","8.00","114.8",".53","11.41","8.53","126.21 "
"Brat Hat","3423","Brat","Hat","Logistics Admin","5.42","75.88","","","5.42","75.88 "
"Tod Vindo","","Tod","Vindo","Logistics Admin","8.00","128","1.18","28.32","9.18","156.32 "

那么步骤

• 处理所有 .csv 文件，排除带有尾随 _dq 的文件，并附加 _dq 保存
• 剥离 4 行
• 删除不需要的列
• 正确引用字段
• 删除最后的TOTAL 行

完成了。还需要做什么：

• 合并文件（没有，现在只有一个，标题行）
• 检查最后一列是否真的有尾随空格。

Answer 2

IMO 合并通常意味着添加列，而不是追加/连接到末尾。而不是更多你可以使用"skip=5 delims="
要删除最后一行，您可以将实际行存储在 var 中并打印前一行。由于这是在（代码块）内，因此您需要 DelayedExpansion（可能会删除 csv 中的感叹号）。

要删除列，您需要指定分隔符和匹配的标记参数以省略不需要的列。
提供“delims=”，并且您想从 6 中删除第 2 列和第 5 列

@echo off&SetLocal EnableDelayedExpansion
(   for %%f in (*.csv) do (
        Set "last="
        for /f "tokens=1,3-4,6 delims=," %%A in ('more +5 %%f') do (
            if defined last echo %%f,!last!
            Set "last=%%A,%%B,%%C,%%D"
        )
    )
) >Output.csv

我的示例输出：

Output.csv


FatTwin1.csv,1,3,4,6
FatTwin2.csv,13,15,16,18

要克服空字段的问题，您可以 normalize csv 文件，方法是使用 powershell 将它们双引号括起来，从而将它们导入/导出。

此 cmd 行将调用 powershell 来 import-csv 和 export-csv 当前文件夹中的所有 csv 文件，并在名称后附加 _dq 进行存储。这要求文件的标题具有唯一的列名。

Powershell -NonI -NoP -Com "gci *.csv -excl '*_dq.csv'|%{$fn=$_.Fullname;(import-csv $fn )|export-csv -path ($fn.replace('.csv','_dq.csv')) -notype}"