【问题标题】:Insert 10M rows Postgresql temp table and select all rows is too slow插入 10M 行 Postgresql 临时表并选择所有行太慢
【发布时间】:2019-07-22 14:35:27
【问题描述】:

我在 PostgreSQL 9.4 中有 11 个表,我创建了一个从表中选择所有表的函数。我使用插入所有十一个表的临时表创建它,插入完成然后从临时表中选择所有。但是这个函数执行43秒,太慢了。我将临时缓冲区从 8 MB 增加到 100 MB 和 1000 MB。但执行并没有减少。如何优化查询并提高性能?

我在临时表上创建索引,但它并没有提高性能。

CREATE OR REPLACE FUNCTION public.fn_useractivitylog_ndis()
  RETURNS TABLE(loguser_name text, logaction_name character varying, logaction_type character varying, logdetail_1 text, logdetail_2 text, logtime1 timestamp without time zone, logtime2 timestamp without time zone, logaction_time timestamp without time zone)
  LANGUAGE plpgsql
  AS $function$
      begin
          create temp table
          temp_table
          (
            tempuser_name text,
            tempaction_name varchar(100),
            tempaction_type varchar(500),
            tempdetail1 text,
            tempdetail2 text,
            temptime1 timestamp without time zone,
            temptime2 timestamp without time zone,
            tempaction_time timestamp without time zone
          ) on
          commit drop ;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblone;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tbltwo;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblthree;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblfour;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblfive;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblsix;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblseven;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tbleigth;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblnine;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tblten;

          insert into temp_table (tempuser_name, tempaction_name, tempaction_type, tempdetail1, tempdetail2, temptime1, temptime2, tempaction_time)
              select * from public.tbleleven;

        return query
            select 
                tempuser_name ,
                tempaction_name ,
                tempaction_type,
                tempdetail1 ,
                tempdetail2 ,
                temptime1 ,
                temptime2 ,
                tempaction_time 
            from temp_table
            order by tempaction_time desc, tempuser_name, tempaction_name;
      end $function$ ;
Function Scan on fn_useractivitylog_ndis  (cost=0.25..10.25 rows=1000 width=184) (actual time=39827.375..40817.591 rows=10200021 loops=1)
Planning time: 0.019 ms 
Execution time: 41076.709 ms

【问题讨论】:

  • 为什么一开始就有 11 个结构相同的表?这听起来像是一个非常糟糕的数据库设计。为什么那不是一张桌子?

标签: postgresql function insert temp


【解决方案1】:

与其使用临时表,不如在返回查询中使用联合:

RETURN QUERY
  SELECT tempuser_name, tempaction_name,tempaction_type, 
         tempdetail1,tempdetail2,temptime1,temptime2, tempaction_time
  FROM public.tblone
  UNION ALL
  SELECT tempuser_name, tempaction_name,tempaction_type, 
         tempdetail1,tempdetail2,temptime1,temptime2, tempaction_time
  FROM public.tbltwo
  ...
  ORDER BY tempaction_time desc, tempuser_name, tempaction_name;

【讨论】:

  • 然后把函数改成language sql或者更好:从UNION语句中创建一个视图
  • 我没有将函数更改为语言 SQL,因为某些 sql 命令仅适用于 pgsql。 @a_horse_with_no_name。不幸的是,创建视图也会得到重复的行。
  • 我使用了联合,但执行时间为 39 秒。 @肖恩约翰斯顿
  • @MustafaBYKSY:你的函数没有显示 PL/pgSQL 语言结构。
  • @MustafaBYKSY:如果那个 UNION ALL 显示重复,你的临时表也会有重复
猜你喜欢
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 1970-01-01
  • 2020-08-01
  • 2015-03-12
  • 1970-01-01
  • 1970-01-01
  • 2015-07-09
相关资源
最近更新 更多