如何删除 postgresql / Rails 连接结果集中的重复项

Question

我有以下查询：

SELECT "users".* 
  FROM "users" 
       INNER JOIN "users_roles" 
                  ON "users_roles"."user_id" = "users"."id" 
       INNER JOIN "roles" 
                  ON "roles"."id" = "users_roles"."role_id" 
       LEFT JOIN events_users 
                  ON events_users.user_id = users.id 
       LEFT JOIN events 
                  ON events.id = events_users.event_id 
       LEFT JOIN booths 
                  ON booths.user_id = users.id 
GROUP BY 
       users.id, 
       roles.id, 
       events.id, 
       booths.id
ORDER BY 
       id ASC

我希望能够删除重复项，但如果多次出现，它似乎会生成多个重复的角色、摊位和活动。

下面是在 SQL 中生成上述查询的 Rails 活动记录命令：

users = User.
      joins(:roles).
      joins("LEFT JOIN events_users ON events_users.user_id = users.id LEFT JOIN events ON events.id = events_users.event_id").
      joins("LEFT JOIN booths ON booths.user_id = users.id").
      group("users.id, roles.id, events.id").
      order("#{sort_column} #{sort_direction}")

我也试过下面没有运气：

users = User.
      joins(:roles).
      joins("LEFT JOIN events_users ON events_users.user_id = users.id LEFT JOIN events ON events.id = events_users.event_id").
      joins("LEFT JOIN booths ON booths.user_id = users.id").
      group("users.id, roles.id, events.id").
      order("#{sort_column} #{sort_direction}")
      select("distinct on(users.id, roles.id, events.id, booths.id) users.*")

有没有办法删除结果集中的所有重复项？

Answer 1

尝试在您的选择语句中使用DISTINCT 子句。如果将这类工作留给 SQL，几乎总是会更好。

SELECT DISTINCT "users".* 
FROM "users" 
INNER JOIN "users_roles" 
  ON "users_roles"."user_id" = "users"."id" 
INNER JOIN "roles" 
  ON "roles"."id" = "users_roles"."role_id" 
LEFT JOIN events_users 
  ON events_users.user_id = users.id 
LEFT JOIN events 
  ON events.id = events_users.event_id 
LEFT JOIN booths 
  ON booths.user_id = users.id 
GROUP BY users.id, 
         roles.id, 
         events.id, 
         booths.id
ORDER BY id ASC

Answer 2

在postgresql 中尝试此查询以删除所有重复行：

delete from table1 where ctid not in
(select max(t1.id) from
(select ctid id,* from table1)t1
group by t1.name,t1.family);

要删除重复行，您需要在行中使用唯一值 postgresql 给我们 ctid 作为表中每一行的唯一值，因为我们可以使用 ctid 删除所有重复行。

SELECT DISTINCT * FROM
(SELECT "users".* 
FROM "users" 
INNER JOIN "users_roles" ON "users_roles"."user_id" = "users"."id" 
INNER JOIN "roles" ON "roles"."id" = "users_roles"."role_id" 
LEFT JOIN events_users ON events_users.user_id = users.id 
LEFT JOIN events ON events.id = events_users.event_id 
LEFT JOIN booths ON booths.user_id = users.id 
GROUP BY users.id, roles.id, events.id, booths.id
ORDER BY id asc)t1;

SQL Fiddle

Answer 3

我不太了解 SQL 解决方案，但我认为纯 Ruby 解决方案是使用 Array 的 uniq 方法

这是文档：http://www.ruby-doc.org/core-2.1.1/Array.html#method-i-uniq

此方法允许您删除数组的所有重复项。 例如：

[User.first, User.first].uniq # => [User.first]

希望对你有帮助！

Answer 4

我是 ruby​​ 新手，对操作数据库并不完全满意，我更喜欢纯 ruby​​ 解决方案。我有一个带有:listing_id 和:school_id 的Assignment 连接表，我的代码导致了数十万个重复条目，因此school.listings 导致了许多重复列表。首先我通过使用Assignment.find_or_create_by 而不是Assignment.create 修复了代码问题，然后我使用下面的 rake 任务来删除重复的条目。删除重复项需要 30 分钟，所以当然有更好的方法来做到这一点，但我对这个结果很满意，因为它有效。

desc "remove duplicate relationships in Assignment"
task :clean_assignment => :environment do

  listing_ids = Assignment.pluck(:listing_id)
  listing_ids = listing_ids.uniq

  listing_ids.each do |listing_id|
    count = 0
    assignments = Assignment.where(:listing_id => listing_id)
    school_ids = []
    assignments.each do |assign|
      if school_ids.include?(assign.school_id)
        assign.destroy
        count += 1
      else
        school_ids << assign.school_id
      end
    end
    if count > 0
     p "#{count} duplicates deleted from #{listing_id}"
    end
  end
end

rake任务完成后，我检查了没有重复：

a = Assignment.pluck(:listing_id, :school_id)
b = a.uniq

irb(main):023:0> a.count
=> 191350
irb(main):024:0> b.count
=> 191350

重复项被删除。