T-SQL XML to Table 性能问题

Question

我是 xml 新手，我尝试将大约 200 个 XML 文件（总计约 60GB）转换为一个表。这有效，但运行了近 7 天。

也许你们中的一个可以给我一些关于如何调整这个查询的提示。

无论如何，这是数据：

DECLARE @xml xml = 
    '<DB>
        <BN>61</BN>
        
        <DA>
            <PN>1</PN>
            <SN>94</SN>
            <A>2015</A>
            <B>2020</B>
            <T M="1">91</T>
            <T M="3">92</T>
            <T M="7">93</T>
            <TBS V="+">22</TBS>
            <TBS V="+">24</TBS>
            <AK>
                <P>11</P>
                <PL>V</PL>
                <BI>103</BI>
            </AK>
            <GB>
                <GT>S</GT>
                <VZ>+</VZ>
                <OG>
                    <OI>S</OI>
                    <VA>U</VA>
                    <E>
                        <EI>32</EI>
                        <PN>1</PN>
                        <VZ>+</VZ>
                    </E>
                </OG>
            </GB>
        </DA>
        <DA>
            <PN>1</PN>
            <SN>5A</SN>
            <A>2015</A>
            <B>2022</B>
            <T M="4">51</T>
            <T M="2">52</T>
            <T M="9">12</T>
        </DA>
        <DA>
            <PN>1</PN>
            <SN>58</SN>
            <A>2016</A>
            <T M="14">91</T>
            <T M="12">92</T>
            <T M="11">11</T>
        </DA>
        <DA>
            <PN>1</PN>
            <SN>74</SN>
            <A>2010</A>
            <B>2018</B>
            <T M="51">51</T>
            <T M="11">52</T>
            <T M="11">33</T>
            <NI>
                <NS>747</NS>
                <T M="32">11</T>
                <T M="33">12</T>
                <T M="35">13</T>
            </NI>
        </DA>
    </DB>';

这是我正在使用的查询

SELECT 
        XX.YY.value('(BN)[1]', 'nvarchar(100)') AS BN,
        H.HH.value('(PN)[1]', 'nvarchar(100)') AS DA_PN,
        H.HH.value('(SN)[1]', 'nvarchar(100)') AS DA_NR,
        H.HH.value('(A)[1]', 'nvarchar(100)') AS DA_A,
        H.HH.value('(B)[1]', 'nvarchar(100)') AS DA_B,
        I.II.value('@M', 'nvarchar(10)') AS DA_M,
        I.II.value('(.)[1]', 'nvarchar(100)') AS DA_T,
        J.JJ.value('@V', 'nvarchar(10)') AS TBS_V,
        J.JJ.value('(.)[1]', 'nvarchar(100)') AS TBS,
        K.KK.value('(NS)[1]', 'nvarchar(100)') AS NI_NS,
        L.LL.value('@M', 'nvarchar(10)') AS NI_M,
        L.LL.value('(.)[1]', 'nvarchar(100)') AS NI_T,
        M.MM.value('(P)[1]', 'nvarchar(100)') AS AK_P,
        M.MM.value('(PL)[1]', 'nvarchar(100)') AS AK_PL,
        M.MM.value('(BI)[1]', 'nvarchar(100)') AS AK_BI,
        N.NN.value('(GT)[1]', 'nvarchar(100)') AS GB_GT,
        N.NN.value('(VZ)[1]', 'nvarchar(100)') AS GB_VZ,
        O.OO.value('(OI)[1]', 'nvarchar(100)') AS OG_OI,
        O.OO.value('(VA)[1]', 'nvarchar(100)') AS OG_VA,
        P.PP.value('(EI)[1]', 'nvarchar(100)') AS OG_EI,
        P.PP.value('(PN)[1]', 'nvarchar(100)') AS OG_PN,
        P.PP.value('(VZ)[1]', 'nvarchar(10)') AS E_VZ
FROM @xml.nodes('/DB') AS XX(YY)
OUTER APPLY XX.YY.nodes('DA') AS H(HH)
OUTER APPLY H.HH.nodes('T') AS I(II)
OUTER APPLY H.HH.nodes('TBS') AS J(JJ)
OUTER APPLY H.HH.nodes('NI') AS K(KK)
OUTER APPLY K.KK.nodes('T') AS L(LL)
OUTER APPLY H.HH.nodes('AK') AS M(MM)
OUTER APPLY H.HH.nodes('GB') AS N(NN)
OUTER APPLY N.NN.nodes('OG') AS O(OO)
OUTER APPLY O.OO.nodes('E') AS P(PP);

我认为，主要问题是那些 9 outer apply。但我找不到更好的方法来完成这项工作。

谢谢你:)

Answer 1

这里的问题是，在一个级别中的重复节点意味着您将获得一个非常大的非规范化表，但也许您可以更好地对其进行规范化。比如DA_T和NI_T可以合并吗？

您不需要.nodes 来提取单个值，仅当您要将子节点分解为行时才需要它。

因此，如果有任何节点您确定只能获得其中一个，或者您知道有多少节点并且无论如何都想向上旋转，您可以删除该.nodes，而只使用.value on它的父级。

---

你应该选择正确的数据类型，这里看起来大部分应该是int。另外，text() 比. 快，所以你的代码会变成这样

SELECT 
        XX.YY.value('(BN/text())[1]', 'int') AS BN,
        H.HH.value('(PN/text())[1]', 'int') AS DA_PN,
        H.HH.value('(SN/text())[1]', 'int') AS DA_NR,
        H.HH.value('(A/text())[1]', 'int') AS DA_A,
        H.HH.value('(B/text())[1]', 'int') AS DA_B,
        I.II.value('@M', 'nvarchar(10)') AS DA_M,
        I.II.value('text()[1]', 'int') AS DA_T,
        J.JJ.value('@V', 'nvarchar(10)') AS TBS_V,
        J.JJ.value('text()[1]', 'int') AS TBS,
        K.KK.value('(NS/text())[1]', 'int') AS NI_NS,
        L.LL.value('@M', 'nvarchar(10)') AS NI_M,
        L.LL.value('text()[1]', 'int') AS NI_T,
        M.MM.value('(P/text())[1]', 'int') AS AK_P,
        M.MM.value('(PL/text())[1]', 'int') AS AK_PL,
        M.MM.value('(BI/text())[1]', 'int') AS AK_BI,
        N.NN.value('(GT/text())[1]', 'int') AS GB_GT,
        N.NN.value('(VZ/text())[1]', 'int') AS GB_VZ,
        O.OO.value('(OI/text())[1]', 'int') AS OG_OI,
        O.OO.value('(VA/text())[1]', 'int') AS OG_VA,
        P.PP.value('(EI/text())[1]', 'int') AS OG_EI,
        P.PP.value('(PN/text())[1]', 'int') AS OG_PN,
        P.PP.value('(VZ/text())[1]', 'nvarchar(10)') AS E_VZ
FROM @xml.nodes('/DB') AS XX(YY)
OUTER APPLY XX.YY.nodes('DA') AS H(HH)
OUTER APPLY H.HH.nodes('T') AS I(II)
OUTER APPLY H.HH.nodes('TBS') AS J(JJ)
OUTER APPLY H.HH.nodes('NI') AS K(KK)
OUTER APPLY K.KK.nodes('T') AS L(LL)
OUTER APPLY H.HH.nodes('AK') AS M(MM)
OUTER APPLY H.HH.nodes('GB') AS N(NN)
OUTER APPLY N.NN.nodes('OG') AS O(OO)
OUTER APPLY O.OO.nodes('E') AS P(PP);