插入百万条记录时的Oracle查询优化答案

【问题标题】：Oracle query optimization while inserting million records插入百万条记录时的Oracle查询优化
【发布时间】：2020-12-31 12:38:14
【问题描述】：

我们正在尝试在我们的生产环境中执行查询，该查询意外地花费了 5 多个小时，但仍未完成。此查询预计将插入 180 万条记录。我对 FSG_WRK.FSG_PRCB_CSP_EXT_HR 表具有完全控制权，但 FAC.CMC_PRSQ_SITE_QA 是我们的系统表，我无法在其上添加任何索引。

请提出建议。

查询：

INSERT INTO FAC.CMC_PRSQ_SITE_QA 
WITH CTE (MPIN, TAX_ID_NBR, POS_ADR_ID, BSAR_ID, CANC_DT, EFF_DT, EXTND_HRS)
AS(
    SELECT EXT.MPIN , 
    EXT.TAX_ID_NBR  , 
    EXT.POS_ADR_ID  , 
    EXT.BSAR_ID     , 
    temp.CANC_DT    , 
    EXT.EFF_DT      ,   
    EXT.EXTND_HRS 
    FROM FSG_WRK.FSG_PRCB_CSP_EXT_HR ext
    INNER JOIN ( SELECT  EXT.MPIN, 
                    EXT.TAX_ID_NBR, 
                    EXT.POS_ADR_ID, 
                    EXT.BSAR_ID, 
                    MAX(EXT.CANC_DT) CANC_DT
                FROM FSG_WRK.FSG_PRCB_CSP_EXT_HR EXT
                WHERE FLAG ='Y'
                GROUP BY EXT.MPIN, EXT.TAX_ID_NBR, EXT.POS_ADR_ID, EXT.BSAR_ID) temp
    ON( ext.MPIN = temp.MPIN
    AND ext.TAX_ID_NBR = temp.TAX_ID_NBR
    AND ext.POS_ADR_ID = temp.POS_ADR_ID
    AND ext.BSAR_ID    = temp.BSAR_ID
    AND ext.CANC_DT    = temp.CANC_DT)
)
SELECT DISTINCT 
        PRSQ.PRPR_ID,
        PRSQ.PRAD_ID ,
        PRSQ.PRAD_TYPE,
        PRSQ.PRAD_EFF_DT,
        EXT.EFF_DT  PRSQ_LAST_VER_DT,
        CASE    WHEN TRIM(EXT.EXTND_HRS)='W' THEN 'EHRW'
                WHEN TRIM(EXT.EXTND_HRS)='B' THEN 'EHRB'
                WHEN TRIM(EXT.EXTND_HRS)='E' THEN 'EHRE'
                WHEN TRIM(EXT.EXTND_HRS)='N' THEN 'EHRN'
        END AS PRSQ_MCTR_ITEM,
        PRSQ.PRSQ_INIT_VER_DT,
        EXT.CANC_DT PRSQ_NEXT_VER_DT,
        PRSQ.PRSQ_RCVD_VER_DT,
        PRSQ.PRSQ_MCTR_VSRC,
        PRSQ.PRSQ_PRIM_VER_IND,
        PRSQ.PRSQ_MCTR_VRSL,
        ' ' PRSQ_LAST_VER_NAME,
        PRSQ.PRSQ_MCTR_VMTH,
        ' ' PRSQ_COMMENT,
        PRSQ.PRSQ_LOCK_TOKEN,
        PRSQ.ATXR_SOURCE_ID         
        FROM FAC.CMC_PRSQ_SITE_QA PRSQ 
        JOIN CTE EXT
        ON EXT.MPIN = SUBSTR(PRSQ.PRPR_ID,1,9)
        AND TRIM(EXT.BSAR_ID) = TRIM (PRSQ.PRSQ_LAST_VER_NAME)
        AND TRIM (EXT.POS_ADR_ID) = TRIM (PRSQ.PRSQ_COMMENT)
        AND TRIM(EXT.CANC_DT) = TRIM(PRSQ.PRSQ_NEXT_VER_DT)
        AND PRSQ.PRSQ_MCTR_ITEM IN ('SAID')
        AND NOT EXISTS(
                        SELECT 1 FROM FAC.CMC_NWPR_RELATION NWPR
                        WHERE NWPR.PRPR_ID = PRSQ.PRPR_ID    
                        AND NWPR.NWNW_ID = 'MINETWORK'
            );

Explain plan :

【问题讨论】：

您可以启用 sql 跟踪并在此处发布吗？在此插入期间，我们需要确切的等待事件。 “负载表常规”行中的最后经过时间太大。可能有表或行锁
我不想猜测，所以最好得到真正的等待事件。如果您拥有带有许可诊断和调整包的 EE，则提供 ashrpt 输出或原始 ash 转储会更容易

标签： oracle oracle-sqldeveloper query-optimization oracle12c

【解决方案1】：

顺便说一句，您似乎想通过 EXT.CANC_DT 在每个组中获取最新行。如果是这样的话，使用FIRST/LAST aggregate functions会更容易：

INSERT INTO FAC.CMC_PRSQ_SITE_QA 
WITH CTE (MPIN, TAX_ID_NBR, POS_ADR_ID, BSAR_ID, CANC_DT, EFF_DT, EXTND_HRS)
AS(
    SELECT
            EXT.MPIN, EXT.TAX_ID_NBR, EXT.POS_ADR_ID, EXT.BSAR_ID
           ,MAX(EXT.CANC_DT) CANC_DT
           ,MAX(EXT.EFF_DT   ) keep(dense_rank first order by EXT.CANC_DT desc) EFF_DT
           ,MAX(EXT.EXTND_HRS) keep(dense_rank first order by EXT.CANC_DT desc) EXTND_HRS
    FROM FSG_WRK.FSG_PRCB_CSP_EXT_HR EXT
    WHERE FLAG ='Y'
    GROUP BY EXT.MPIN, EXT.TAX_ID_NBR, EXT.POS_ADR_ID, EXT.BSAR_ID
)
SELECT DISTINCT 
        PRSQ.PRPR_ID,
        PRSQ.PRAD_ID ,
        PRSQ.PRAD_TYPE,
        PRSQ.PRAD_EFF_DT,
        EXT.EFF_DT  PRSQ_LAST_VER_DT,
        CASE    WHEN TRIM(EXT.EXTND_HRS)='W' THEN 'EHRW'
                WHEN TRIM(EXT.EXTND_HRS)='B' THEN 'EHRB'
                WHEN TRIM(EXT.EXTND_HRS)='E' THEN 'EHRE'
                WHEN TRIM(EXT.EXTND_HRS)='N' THEN 'EHRN'
        END AS PRSQ_MCTR_ITEM,
        PRSQ.PRSQ_INIT_VER_DT,
        EXT.CANC_DT PRSQ_NEXT_VER_DT,
        PRSQ.PRSQ_RCVD_VER_DT,
        PRSQ.PRSQ_MCTR_VSRC,
        PRSQ.PRSQ_PRIM_VER_IND,
        PRSQ.PRSQ_MCTR_VRSL,
        ' ' PRSQ_LAST_VER_NAME,
        PRSQ.PRSQ_MCTR_VMTH,
        ' ' PRSQ_COMMENT,
        PRSQ.PRSQ_LOCK_TOKEN,
        PRSQ.ATXR_SOURCE_ID         
        FROM FAC.CMC_PRSQ_SITE_QA PRSQ 
        JOIN CTE EXT
        ON EXT.MPIN = SUBSTR(PRSQ.PRPR_ID,1,9)
        AND TRIM(EXT.BSAR_ID) = TRIM (PRSQ.PRSQ_LAST_VER_NAME)
        AND TRIM (EXT.POS_ADR_ID) = TRIM (PRSQ.PRSQ_COMMENT)
        AND TRIM(EXT.CANC_DT) = TRIM(PRSQ.PRSQ_NEXT_VER_DT)
        AND PRSQ.PRSQ_MCTR_ITEM IN ('SAID')
        AND NOT EXISTS(
                        SELECT 1 FROM FAC.CMC_NWPR_RELATION NWPR
                        WHERE NWPR.PRPR_ID = PRSQ.PRPR_ID    
                        AND NWPR.NWNW_ID = 'MINETWORK'
            );

【讨论】：

【解决方案2】：

如果您“只是”将数据从 A 迁移到 B，您可以看看语句末尾的 merge into 和 WHEN NOT MATCHED THEN INSERT 是否不是更好的选择。

180 万条记录不应该对您的 oracle 实例造成太大挑战。

【讨论】：