Tableau + Redshift 游标缓慢

Question

我知道 Tableau Server 使用游标来刷新数据提取，但是，对于运行一些包含大量列的简单查询，其中一些游标的执行持续时间存在很大的不一致。例如，当我运行时：

select * from svl_qlog
where userid = (select usesysid from pg_user where usename='tableau')
order by starttime desc
limit 20;

我明白了：

+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+
|userid|query   |xid     |pid |starttime                 |endtime                   |elapsed  |aborted|label                                                                                                                                                                                                                                                                                                                           |substring                                                   |source_query|concurrency_scaling_status_txt|from_sp_call|
+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+
|108   |14993377|36192048|3270|2021-08-24 03:34:48.862153|2021-08-24 03:38:09.404563|200542410|0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR7";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993370|36192035|3270|2021-08-24 03:34:41.174557|2021-08-24 03:34:41.185152|10595    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR6";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993368|36192034|3270|2021-08-24 03:34:40.991779|2021-08-24 03:34:41.021350|29571    |0      |default                                                                                                                                                                                                                                                                                                                         |Undoing 1 transactions on table 1728726 with current xid 361|NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993367|36192032|3270|2021-08-24 03:34:40.861741|2021-08-24 03:34:40.907681|45940    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR3";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993365|36192025|3262|2021-08-24 03:34:38.135543|2021-08-24 03:34:38.229458|93915    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR7";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993363|36192022|3262|2021-08-24 03:34:38.006010|2021-08-24 03:34:38.008911|2901     |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR6";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993360|36192020|3262|2021-08-24 03:34:37.250081|2021-08-24 03:34:37.885200|635119   |0      |default                                                                                                                                                                                                                                                                                                                         |Undoing 1 transactions on table 1728724 with current xid 361|NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993359|36192018|3262|2021-08-24 03:34:35.811267|2021-08-24 03:34:35.865765|54498    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR3";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993287|36191920|2934|2021-08-24 03:33:16.921494|2021-08-24 03:33:38.143570|21222076 |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR7";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993285|36191917|2934|2021-08-24 03:33:16.618563|2021-08-24 03:33:16.623745|5182     |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR6";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993281|36191916|2934|2021-08-24 03:33:15.619813|2021-08-24 03:33:16.493711|873898   |0      |default                                                                                                                                                                                                                                                                                                                         |Undoing 1 transactions on table 1728722 with current xid 361|NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993280|36191914|2934|2021-08-24 03:33:14.720016|2021-08-24 03:33:14.787236|67220    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR3";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993071|36191663|2258|2021-08-24 03:30:25.760462|2021-08-24 03:31:05.340131|39579669 |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR7";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993069|36191660|2258|2021-08-24 03:30:25.359800|2021-08-24 03:30:25.366651|6851     |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR6";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993064|36191657|2258|2021-08-24 03:30:25.170646|2021-08-24 03:30:25.245196|74550    |0      |default                                                                                                                                                                                                                                                                                                                         |Undoing 1 transactions on table 1728720 with current xid 361|NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993063|36191655|2258|2021-08-24 03:30:25.045651|2021-08-24 03:30:25.079935|34284    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR3";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993053|36191642|2182|2021-08-24 03:30:18.163032|2021-08-24 03:30:18.381360|218328   |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR7";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993050|36191638|2182|2021-08-24 03:30:18.029206|2021-08-24 03:30:18.032746|3540     |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR6";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993043|36191620|2182|2021-08-24 03:30:15.207471|2021-08-24 03:30:15.853592|646121   |0      |default                                                                                                                                                                                                                                                                                                                         |Undoing 1 transactions on table 1728718 with current xid 361|NULL        |0 - Ran on the main cluster   |NULL        |
|108   |14993042|36191618|2182|2021-08-24 03:30:14.086680|2021-08-24 03:30:14.131522|44842    |0      |default                                                                                                                                                                                                                                                                                                                         |fetch 100000 in "SQL_CUR3";                                 |NULL        |0 - Ran on the main cluster   |NULL        |
+------+--------+--------+----+--------------------------+--------------------------+---------+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------+------------+------------------------------+------------+

只需查看经过的列，您就可以分辨出问题所在。所以，我的问题是，这些游标中的一些需要更长的时间是否有已知的原因？另外，Tableau 或 Redshift 是罪魁祸首吗？这些是什么：

Undoing 1 transactions on table 1728722 with current xid 361

谢谢！

Answer 1

第一次获取游标运行查询。随后的提取只是检索已经计算并在领导节点上等待的下一组数据。在您的示例中，您返回的数据看起来少于 10K 行，因为我看不到同一事务的后续提取。这是正确的还是你错过了一些获取？由于您说您正在刷新数据提取，因此似乎需要更多数据，但我不知道您的情况。

鉴于此，不同的获取时间很可能是由于不同的查询复杂性和运行时间。如果您查看这些 XID 的活动 (svl_statementtext)，您可以看到正在发送什么查询来填充游标。然后，您可以自己运行这些查询，看看它们的复杂程度和一般运行时间。

其他可能性包括排队延迟和/或数据库的高查询负载和/或领导节点上的高负载。

您知道返回的数据集的大小吗？如果这非常大，那么可能有更好的方法将刷新数据获取到 Tableau。使用选择查询通过领导节点提取数据，但这些方法需要努力设置。

要回答的问题 - 获取哪些查询？这些需要多长时间才能独立运行？每个查询返回多少数据？了解这些答案将有助于为改进铺平道路。