连接表的 SQL UPDATE：内存不足错误

Question

我正在尝试更新 pSQL 中的表并得到各种内存/执行错误。

奇怪的是，支持更新的 SELECT 查询非常快。我确定我只是不明白幕后发生了什么。

一些上下文。

---

相关表格

address_book:
loan_id,
county,
zip
---
loan:
id
---
loan_property:
loan_id,
property_id
---
property:
id,
zip,
county

目标

目标是使用 address_book 中的值更新属性表的 zip 和 county。 address_book 有一个 loan_id，它是属性的连接。

---

SQL

我们来看一个简单的SELECT

WITH ab AS (
SELECT DISTINCT
    left(ab.loan_id, 6) AS loan_id,
    ab.zip AS zip,
    ab.county AS county
FROM 
    address_book ab
WHERE
    ab.address IS NOT NULL
)

SELECT ab.county, p.name

FROM property p
INNER JOIN loan_property lp ON lp.property_id = p.id
INNER JOIN loan           l ON lp.loan_id     = l.id
INNER JOIN               ab ON ab.loan_id     = l.id
WHERE 
    l.id = ab.loan_id

这非常有效并且非常快（大约 10k 条记录上 0.4 秒）

让我们把上面的内容变成一个 UPDATE 调用：

WITH ab AS (
SELECT DISTINCT
    left(ab.loan_id, 6) AS loan_id,
    ab.zip AS zip,
    ab.county AS county
FROM 
    address_book ab
WHERE
    ab.address IS NOT NULL
)

UPDATE property
SET zip=ab.zip, county=ab.county

FROM property p
INNER JOIN loan_property lp ON lp.property_id = p.id
INNER JOIN loan           l ON lp.loan_id     = l.id
INNER JOIN               ab ON ab.loan_id     = l.id
WHERE 
    l.id = ab.loan_id

此更新运行 2 分钟，然后通常会失败

SQL 错误 [53200]：错误：内存不足

是否有更优化的方式来运行此更新？即使我必须通过 LIMIT/OFFSET 进行批处理或将 SELECT 结果保存到表中，然后直接从该表执行更新 - 有什么方法可以运行此更新而不会遇到内存错误？

非常感谢大家！

Answer 1

As documented in the manual do not 在 UPDATE 语句中重复目标表：

...
UPDATE property
   SET zip = ab.zip, 
       county = ab.county
FROM loan_property lp
   JOIN loan l ON lp.loan_id = l.id
   JOIN ab ON ab.loan_id = l.id
WHERE lp.property_id = p.id

Answer 2

我猜你的房产有很多贷款。运行：

select property_id, count(*)
from loan_property
group by property_id
order by count(*) desc;

问题是你想从哪些信息中填写信息。

您也有可能拥有多个地址的贷款。 select distinct 在那里很可疑。

Answer 3

最可取的是对记录进行分组、排序和限制，然后在更新语句中使用相同的记录，我想可能有很多这是你内存不足的主要原因。因为更新是逐行进行的，所以冗余数据会让更新工作更多，并恶化时间限制。因此，将更新中的记录分组为select 语句的最佳选择已经花费了更少的时间，因此没有必要对其进行优化。试试下面的示例

      WITH xyz AS (
      Select zip,property from property p
    INNER JOIN loan_property lp ON lp.property_id = p.id
     INNER JOIN loan           l ON lp.loan_id     = l.id
        INNER JOIN               ab ON ab.loan_id     = l.id
   WHERE 
    l.id = ab.loan_id group by some_value/order by zip)

    UPDATE xyz
     SET zip=ab.zip, county=ab.county

Answer 4

我认为我的答案与@a_horse_with_no_name 的答案相似——在重新引用目标表时有些奇怪。

我实际上将更新中的 FROM 子句组合到另一个别名 SELECT 调用中，如下所示：

WITH ab as (
SELECT distinct
    p.id as p_id, 
    ab.county as county, 
    ab.zip as zip
FROM 
    address_book ab
inner join loan l on ab.loan_id = l.id
inner join loan_property lp on loan_id = l.id
inner join property p on lp.property_id = p.id
WHERE
    ab.address IS NOT null
    and l.id = ab.loan_id
)

UPDATE property
SET county__c=ab.county, zip_code__c=ab.zip
FROM ab
WHERE ab.p_id = id

从更新中的 FROM 调用中分离连接（尤其是与目标表）解决了该问题。