将用户日志表映射到用户第一个、最后一个和上一个活动的 Sql 查询

Question

我有一个用户日志表，按 action_date（表名 - user_action_log）分区，包含数十亿行和列

user_id、action_name、action_date

样本数据 -

+---------+-------------+-------------+
| user_id | action_name | action_date |
+---------+-------------+-------------+
| 123     | login       | 2018-01-30  |
| 123     | logout      | 2018-01-31  |
| 123     | click       | 2018-02-28  |
| 123     | comment     | 2018-02-15  |
| 123     | post        | 2018-03-15  |
+---------+-------------+-------------+

我想编写一个 ETL/sql 来将这些数据转换成这样的东西（表名 - user_action_record）。

user_id（主键）、first_action_date、last_action_date、previous_action_date

样本输出数据-

+---------+-------------------+------------------+---------------------------+
| user_id | first_action_date | last_action_date | previous_last_action_date |
+---------+-------------------+------------------+---------------------------+
| 123     | 2018-01-30        | 2018-03-15       | 2018-02-28                |
+---------+-------------------+------------------+---------------------------+

我尝试将问题分为两个步骤 -

1. 插入 user_action_record 中不存在的新用户。
2. 通过从“last_action_date”中的值更新“previous_last_action_date”来更新现有用户，并根据 user_action_log 表更新 last_action_date。

问题在于，由于 user_action_log 在 action_date 上进行分区，我可以每天查询该表 (action_date = CURRENT_DATE)

在这种情况下，谁能帮我用 sqls 填充我的目标表？

-- 编辑下面的附加信息

1. “2018-01-30”日的源和预期目标表

+---------+-------------+-------------+
| user_id | action_name | action_date |
+---------+-------------+-------------+
| 123     | login       | 2018-01-30  |
| 123     | logout      | 2018-01-30  |
| 123     | click       | 2018-01-30  |
+---------+-------------+-------------+

+---------+-------------------+------------------+---------------------------+
| user_id | first_action_date | last_action_date | previous_last_action_date |
+---------+-------------------+------------------+---------------------------+
| 123     | 2018-01-30        | 2018-01-30       | 2018-01-30                |
+---------+-------------------+------------------+---------------------------+

1. “2018-01-31”日的源和预期目标表

+---------+-------------+-------------+
| user_id | action_name | action_date |
+---------+-------------+-------------+
| 123     | login       | 2018-01-30  |
| 123     | logout      | 2018-01-30  |
| 123     | click       | 2018-01-30  |
| 123     | login       | 2018-01-31  |
| 123     | logout      | 2018-01-31  |
+---------+-------------+-------------+

+---------+-------------------+------------------+---------------------------+
| user_id | first_action_date | last_action_date | previous_last_action_date |
+---------+-------------------+------------------+---------------------------+
| 123     | 2018-01-30        | 2018-01-31       | 2018-01-30                |
+---------+-------------------+------------------+---------------------------+

1. “2018-02-15”日的源和预期目标表

+---------+-------------+-------------+
| user_id | action_name | action_date |
+---------+-------------+-------------+
| 123     | login       | 2018-01-30  |
| 123     | logout      | 2018-01-30  |
| 123     | click       | 2018-01-30  |
| 123     | login       | 2018-01-31  |
| 123     | logout      | 2018-01-31  |
| 123     | logout      | 2018-02-15  |
| 123     | logout      | 2018-02-15  |
+---------+-------------+-------------+

+---------+-------------------+------------------+---------------------------+
| user_id | first_action_date | last_action_date | previous_last_action_date |
+---------+-------------------+------------------+---------------------------+
| 123     | 2018-01-30        | 2018-02-15       | 2018-01-31                |
+---------+-------------------+------------------+---------------------------+

Answer 1

您可以在user_id 上使用自联接模拟 MySQL action_date：

SELECT u1.*, COUNT(u2.user_id) AS rn
FROM user_action_log u1
LEFT JOIN user_action_log u2 ON u2.user_id = u1.user_id AND u2.action_date > u1.action_date
GROUP BY u1.user_id, u1.action_name, u1.action_date
ORDER BY rn;

输出：

user_id action_name action_date rn
123     post        2018-03-15  0
123     click       2018-02-28  1
123     comment     2018-02-15  2
123     logout      2018-01-31  3
123     login       2018-01-30  4

然后这个表可以用作派生表，我们可以从rn = 1的行中找到previous_last_action_date：

SELECT user_id,
       MIN(action_date) AS first_action_date,
       MAX(action_date) AS last_action_date,
       MAX(CASE WHEN rn = 1 THEN action_date END) AS previous_last_action_date
FROM (SELECT u1.*, COUNT(u2.user_id) AS rn
      FROM user_action_log u1
      LEFT JOIN user_action_log u2 ON u2.user_id = u1.user_id AND u2.action_date > u1.action_date
      GROUP BY u1.user_id, u1.action_name, u1.action_date) ual
GROUP BY user_id

输出：

user_id first_action_date   last_action_date    previous_last_action_date
123     2018-01-30          2018-03-15          2018-02-28

Demo on dbfiddle

Answer 2

在早期版本的 MySQL 中，您可以使用 group_concat()/substring_index() 技巧：

select user_id,
       min(action_date) as first_action_date,
       max(action_date) as last_action_date,
       substring_index(substring_index(group_concat(action_date order by action_date desc), ',', 2), ',', -1) as second_to_last_date
from user_action_log ual
group by user_id;

另一种选择是相关子查询：

select user_id,
       min(action_date) as first_action_date,
       max(action_date) as last_action_date,
       (select max(ual2.action_date)
        from user_action_log ual2
        where ual2.user_id = ual.user_id and
              ual2.action_date < max(ual.action_date)
      ) as second_to_last_date
from user_action_log ual
group by user_id;