PowerShell中的数据操作\重复数据删除

Question

您好，我希望对一些数据进行重复数据删除并合并 CSV 中的列。我不知道该怎么做。以下是我正在处理的数据示例：

cmmc,stig,descr
AC.1.001,SV-205663r569188_rule,The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
AC.1.001,SV-205667r569188_rule,Inappropriate granting of user rights can provide system administrative and other high-level capabilities.
AC.1.002,SV-205663r569188_rule,The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
AC.1.002,SV-205665r569188_rule,Enterprise Domain Controllers groups on domain controllers.

我非常接近我正在寻找的数据，但很难在第二列中的项目之后添加|<value of 'descr'>：

这是我的脚本：

Import-CSV '.\input.csv' | Group-Object 'cmmc' |
    ForEach-Object {
        [PsCustomObject]@{
            cmmc = $_.name
            stig = $_.group.stig -Join '
'
                    }
    } | Export-Csv '.\output.csv' -NoTypeInformation

输出如下所示（为便于阅读而格式化，省略列名）：

AC1.001    SV-205663r569188_rule
           SV-205665r569188_rule
AC1.002    SV-205663r569188_rule
           SV-205665r569188_rule

但我正在寻找这个：

AC.1.001 SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
         SV-205667r569188_rule|Inappropriate granting of user rights can provide system administrative and other high-level capabilities.
AC.1.002 SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
         SV-205665r569188_rule|Enterprise Domain Controllers groups on domain controllers.

Answer 1

使用以下内容，将 calculated properties 与应用于 Group-Object 调用结果的 Select-Object cmdlet 结合使用：

Import-Csv .\input.csv | 
  Group-Object cmmc |
    Select-Object @{ Name = 'cmmc'; e = 'Name' },
      @{ Name = 'stig_descr'; e = { 
          [array] $stigs, [array] $descrs, $i = $_.Group.stig, $_.Group.descr, 0
          $sigs.ForEach( { $stigs[$i], $descrs[$i++] -join '|' }) -join "`n" 
        } 
      } | Export-Csv -NoTypeInformation -Encoding utf8 .\output.csv

^{注意：
• 需要$stigs 和$descrs 的[array] 类型约束来处理组仅包含一个 记录的情况，在这种情况下$_.Group.sig 和$_.Group.descr，由于member enumeration 的行为，只返回一个 单个字符串而不是 单个元素数组；如果没有[array] 演员表，则将对[string] 实例执行索引（例如[$i]），这将从字符串中返回该位置的单个字符。
• 在Export-Csv 呼叫中，根据需要调整-Encoding。 PowerShell (Core) 7+ 现在默认使用无 BOM 的 UTF-8，并且不再需要 -NoTypeInformation。}

生成的文件具有以下内容，显示了列内部换行符的使用（受"..." 中包含的整个值的保护）：

"cmmc","stig_descr"
"AC.1.001","SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
SV-205667r569188_rule|Inappropriate granting of user rights can provide system administrative and other high-level capabilities."
"AC.1.002","SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
SV-205665r569188_rule|Enterprise Domain Controllers groups on domain controllers."

---

要直观地显示这会产生所需的数据，您可以重新导入生成的文件并使用-Wrap 开关将其通过管道传输到-Wrap：

PS> Import-Csv .\output.csv | Format-Table -Wrap

cmmc     stig_descr
----     ---------
AC.1.001 SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
         SV-205667r569188_rule|Inappropriate granting of user rights can provide system administrative and other high-level capabilities.
AC.1.002 SV-205663r569188_rule|The ability to set access permissions and auditing is critical to maintaining the security and proper access controls of a system. To support this volumes must be formatted using a file system that supports NTFS attributes.
         SV-205665r569188_rule|Enterprise Domain Controllers groups on domain controllers.

请注意，-Wrap 尊重属性内部的换行符，但如果单独的行对于控制台窗口来说太宽，也会将它们分成多行。