将 XML latin1 转换为 UTF-8 或其他方式答案

【问题标题】：Convert XML latin1 to UTF-8 and other way around将 XML latin1 转换为 UTF-8 或其他方式
【发布时间】：2017-02-13 15:01:48
【问题描述】：

我正在尝试将 XML 文件从 Latin1 转换为 UTF-8，反之亦然。我一直在做一些测试，但我没有成功。我正在使用

Get-Content C:\inputfile.xml | Set-Content -Encoding utf8 C:\outputfile.xml

但这并没有转换任何东西。所以我尝试在Get-Content 中给出编码，但在 PowerShell 中无法识别 Latin1（或者这就是错误消息告诉我的内容）。获得这个的最佳方法是什么？

【问题讨论】：

标签： xml powershell utf-8 converter iso-8859-1

【解决方案1】：

最快的方法是使用 .NET System.IO.File 类，尤其是对于大型 XML 文件。

将ReadAllText 与明确提供的Latin-1 编码一起使用：

[IO.File]::ReadAllText('r:\1.txt', [Text.Encoding]::GetEncoding('iso-8859-1')) | 
    Set-Content r:\2.txt -Encoding UTF8

如果您的 xml 文件有 <?xml version="1.0" encoding="iso-8859-1" ?> 它也需要更改：

[IO.File]::ReadAllText('r:\1.txt', [Text.Encoding]::GetEncoding('iso-8859-1')).
    Replace('<?xml version="1.0" encoding="iso-8859-1"',
            '<?xml version="1.0" encoding="UTF-8"') | 
    Set-Content r:\2.txt -Encoding UTF8

要编写 Latin-1 编码，请使用 WriteAllText 和明确提供的 Latin-1 编码：

[IO.File]::WriteAllText(
    'r:\2.txt',
    [IO.File]::ReadAllText('r:\1.txt', [Text.Encoding]::UTF8).
        Replace('<?xml version="1.0" encoding="UTF-8"',
                '<?xml version="1.0" encoding="iso-8859-1"'),
    [Text.Encoding]::GetEncoding('iso-8859-1')
)

可以处理任何大小的文件（1TB？没问题！）的内存高效转码：

function transcodeXML(
    [ValidateScript({Test-Path -Literal $_})]
    [string]$source,
    [ValidateSet('IBM037', 'IBM437', 'IBM500', 'ASMO-708', 'DOS-720', 'ibm737', 'ibm775', 'ibm850', 'ibm852', 'IBM855', 'ibm857', 'IBM00858', 'IBM860', 'ibm861', 'DOS-862', 'IBM863', 'IBM864', 'IBM865', 'cp866', 'ibm869', 'IBM870', 'windows-874', 'cp875', 'shift_jis', 'gb2312', 'ks_c_5601-1987', 'big5', 'IBM1026', 'IBM01047', 'IBM01140', 'IBM01141', 'IBM01142', 'IBM01143', 'IBM01144', 'IBM01145', 'IBM01146', 'IBM01147', 'IBM01148', 'IBM01149', 'utf-16', 'utf-16BE', 'windows-1250', 'windows-1251', 'Windows-1252', 'windows-1253', 'windows-1254', 'windows-1255', 'windows-1256', 'windows-1257', 'windows-1258', 'Johab', 'macintosh', 'x-mac-japanese', 'x-mac-chinesetrad', 'x-mac-korean', 'x-mac-arabic', 'x-mac-hebrew', 'x-mac-greek', 'x-mac-cyrillic', 'x-mac-chinesesimp', 'x-mac-romanian', 'x-mac-ukrainian', 'x-mac-thai', 'x-mac-ce', 'x-mac-icelandic', 'x-mac-turkish', 'x-mac-croatian', 'utf-32', 'utf-32BE', 'x-Chinese-CNS', 'x-cp20001', 'x-Chinese-Eten', 'x-cp20003', 'x-cp20004', 'x-cp20005', 'x-IA5', 'x-IA5-German', 'x-IA5-Swedish', 'x-IA5-Norwegian', 'us-ascii', 'x-cp20261', 'x-cp20269', 'IBM273', 'IBM277', 'IBM278', 'IBM280', 'IBM284', 'IBM285', 'IBM290', 'IBM297', 'IBM420', 'IBM423', 'IBM424', 'x-EBCDIC-KoreanExtended', 'IBM-Thai', 'koi8-r', 'IBM871', 'IBM880', 'IBM905', 'IBM00924', 'EUC-JP', 'x-cp20936', 'x-cp20949', 'cp1025', 'koi8-u', 'iso-8859-1', 'iso-8859-2', 'iso-8859-3', 'iso-8859-4', 'iso-8859-5', 'iso-8859-6', 'iso-8859-7', 'iso-8859-8', 'iso-8859-9', 'iso-8859-13', 'iso-8859-15', 'x-Europa', 'iso-8859-8-i', 'iso-2022-jp', 'csISO2022JP', 'iso-2022-jp', 'iso-2022-kr', 'x-cp50227', 'euc-jp', 'EUC-CN', 'euc-kr', 'hz-gb-2312', 'GB18030', 'x-iscii-de', 'x-iscii-be', 'x-iscii-ta', 'x-iscii-te', 'x-iscii-as', 'x-iscii-or', 'x-iscii-ka', 'x-iscii-ma', 'x-iscii-gu', 'x-iscii-pa', 'utf-7', 'utf-8')]
    [string]$sourceEncoding,

    [ValidateScript({Test-Path -Literal $_ -IsValid})]
    [string]$target,
    [ValidateSet('IBM037', 'IBM437', 'IBM500', 'ASMO-708', 'DOS-720', 'ibm737', 'ibm775', 'ibm850', 'ibm852', 'IBM855', 'ibm857', 'IBM00858', 'IBM860', 'ibm861', 'DOS-862', 'IBM863', 'IBM864', 'IBM865', 'cp866', 'ibm869', 'IBM870', 'windows-874', 'cp875', 'shift_jis', 'gb2312', 'ks_c_5601-1987', 'big5', 'IBM1026', 'IBM01047', 'IBM01140', 'IBM01141', 'IBM01142', 'IBM01143', 'IBM01144', 'IBM01145', 'IBM01146', 'IBM01147', 'IBM01148', 'IBM01149', 'utf-16', 'utf-16BE', 'windows-1250', 'windows-1251', 'Windows-1252', 'windows-1253', 'windows-1254', 'windows-1255', 'windows-1256', 'windows-1257', 'windows-1258', 'Johab', 'macintosh', 'x-mac-japanese', 'x-mac-chinesetrad', 'x-mac-korean', 'x-mac-arabic', 'x-mac-hebrew', 'x-mac-greek', 'x-mac-cyrillic', 'x-mac-chinesesimp', 'x-mac-romanian', 'x-mac-ukrainian', 'x-mac-thai', 'x-mac-ce', 'x-mac-icelandic', 'x-mac-turkish', 'x-mac-croatian', 'utf-32', 'utf-32BE', 'x-Chinese-CNS', 'x-cp20001', 'x-Chinese-Eten', 'x-cp20003', 'x-cp20004', 'x-cp20005', 'x-IA5', 'x-IA5-German', 'x-IA5-Swedish', 'x-IA5-Norwegian', 'us-ascii', 'x-cp20261', 'x-cp20269', 'IBM273', 'IBM277', 'IBM278', 'IBM280', 'IBM284', 'IBM285', 'IBM290', 'IBM297', 'IBM420', 'IBM423', 'IBM424', 'x-EBCDIC-KoreanExtended', 'IBM-Thai', 'koi8-r', 'IBM871', 'IBM880', 'IBM905', 'IBM00924', 'EUC-JP', 'x-cp20936', 'x-cp20949', 'cp1025', 'koi8-u', 'iso-8859-1', 'iso-8859-2', 'iso-8859-3', 'iso-8859-4', 'iso-8859-5', 'iso-8859-6', 'iso-8859-7', 'iso-8859-8', 'iso-8859-9', 'iso-8859-13', 'iso-8859-15', 'x-Europa', 'iso-8859-8-i', 'iso-2022-jp', 'csISO2022JP', 'iso-2022-jp', 'iso-2022-kr', 'x-cp50227', 'euc-jp', 'EUC-CN', 'euc-kr', 'hz-gb-2312', 'GB18030', 'x-iscii-de', 'x-iscii-be', 'x-iscii-ta', 'x-iscii-te', 'x-iscii-as', 'x-iscii-or', 'x-iscii-ka', 'x-iscii-ma', 'x-iscii-gu', 'x-iscii-pa', 'utf-7', 'utf-8')]
    [string]$targetEncoding
) {
    $reader = [IO.StreamReader]::new(
        $source,
        [Text.Encoding]::GetEncoding($sourceEncoding)
    )
    $writer = [IO.StreamWriter]::new(
        $target,
        $false, # don't append = overwrite
        [Text.Encoding]::GetEncoding($targetEncoding)
    )
    $buf = [char[]]::new(16MB)

    $nRead = $reader.Read($buf, 0, $buf.Length)
    $writer.Write(
        ([regex]"(<\?xml [^>]*?encoding="")(?i)$sourceEncoding(?="")").Replace(
            [string]::new($buf, 0, $nRead),
            '$1' + $targetEncoding,
            1 # speedup: one replacement only
        )
    )
    while (!$reader.EndOfStream) {
        $nRead = $reader.Read($buf, 0, $buf.Length)
        $writer.Write($buf, 0, $nRead)
    }
    $reader.Close()
    $writer.Close()
}

用法：

transcodeXML 'r:\1.xml' iso-8859-1 'r:\2.xml' utf-8

【讨论】：

是的，请不要让 XML 声明中的编码成为谎言。第二个要点至关重要。
这些技术还保留了 XML 语义认为无关紧要的人类价值空白。

【解决方案2】：

我建议使用Load 方法将XML 拉入System.Xml.Linq.XDocument，然后根据需要更改XDocument 的Encoding 属性(https://msdn.microsoft.com/en-us/library/system.xml.linq.xdocument.declaration(v=vs.110).aspx) 或添加一个如果Declaration 为空，最后你可以使用Save 方法保存更改的文档。

【讨论】：