如何优化和组合这些低效的数据库查询？

Question

这是我的数据库查询：

User has_many UserFollow（UserFollow 是 User 模型之间的关系）。 用户 has_many 照片。 Photo has_many PhotoFollow 关系（PhotoFollow 是 User 和 Photo 模型之间的关系）。

@user_list = Array.new
user_followers = UserFollow.where("user_1_id = ?", current_user.id).includes(:follower)
user_followers.each do |f|
  @user_list << f.user
end

photos = Photo.where("user_id = ?", current_user.id).includes(:follow_relationships => [:photo])
photos.each do |p|
  p.follow_relationships.each do |f|
    @user_list << f.user if !@user_list.include? f.user
  end
end

if @user_list.size < 150
  users = User.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).includes(:solutions).limit(150 - @user_list.size)
  users.each do |u|
    @user_list << u if !@user_list.include? u
  end
end

显然，所有这些都需要大量的时间。使用包含有帮助，但我想知道是否有一些方法可以更有效地执行这组操作。

谢谢， 林哥

Answer 1

先附加您的关联。

class User < ActiveRecord::Base
  has_many :follows_to_user, :class_name => 'UserFollow', :foreign_key => 'follower_id'
  has_many :follows_to_photo, :class_name => 'PhotoFollow', :foreign_key => 'user_id' # if you don't have it now
end

现在，前两个查询可以在仅一个 SQL 查询中变得更加优雅，返回 AR::Relation 范围。

@user_list = User.includes(:follows_to_user => {}, :follows_to_photo => {:photo => {}}).where(["user_follows.user_1_id = :user_id OR photos.user_id = :user_id", :user_id => current_user.id])

大约 150...[更新]

当然，您最好实现该逻辑，将前面的 SQL 语句附加条件和 UNION 语句（仅使用 SQL 语法），应该返回 AR::关系和会快一点。但是你可以保持懒惰并将其留在 ruby​​ 中，尽管它会返回 Array：

if (count = @user_list.count) && count < 150 # run COUNT just once and store value into local variable
  @user_list |= User.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).includes(:solutions).limit(150 - count)
end

Answer 2

查看您的代码，您计划将用户列表发送到您的@user_list。您可以先建立用户 ID 列表，这样我们就不会创建不必要的 AR 对象

第一个代码

@user_list = Array.new
user_followers = UserFollow.where("user_1_id = ?", current_user.id).includes(:follower)
user_followers.each do |f|
  @user_list << f.user
end

可以改成

# assuming you have a user_id column on user_follows table
user_ids = User.joins(:user_follows).where(user_follows: { user_1_id: current_user.id })
  .uniq.pluck('user_follows.user_id')

第二个代码

photos = Photo.where("user_id = ?", current_user.id).includes(:follow_relationships =>[:photo])
photos.each do |p|
  p.follow_relationships.each do |f|
    @user_list << f.user if !@user_list.include? f.user
  end
end

可以改成

user_ids += Photo.where(user_id: current_user.id).joins(follow_relationships: :photo)
  .uniq.pluck('follow_relationships.user_id')

第三个代码

if @user_list.size < 150
  users = User.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).includes(:solutions).limit(150 - @user_list.size)
  users.each do |u|
    @user_list << u if !@user_list.include? u
  end
end

可以改成

user_ids += users = User.where(verified: true)
  .where('first_name IS NOT NULL AND last_name IS NOT NULL')
  .where('id NOT IN (?)', user_ids)
  .limit(150 - user_ids.size).pluck(:id)

然后你可以使用user_ids获取所有用户

@user_list = User.where(id: user_ids)

Answer 3

必须有比我的答案更好的方法，但是为什么不包括 :user，因为您在迭代查询时正在加载它们？

@user_list = Array.new
user_followers = UserFollow.includes(:user).where("user_1_id = ?", current_user.id)
# why did you include followers?
user_followers.each do |f|
  @user_list << f.user
end

photos = Photo.includes(follow_relationships: { photo: :user }).where("user_id = ?", current_user.id)
photos.each do |p|
  p.follow_relationships.each do |f|
    @user_list << f.user unless @user_list.include? f.user
  end
end

if @user_list.size < 150
  users = User.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).limit(150 - @user_list.size)
  # why did you include solutions?
  users.each do |u|
    @user_list << u unless @user_list.include? u
  end
end

也许这更快，我不确定：

@follower_ids = UserFollow.where("user_1_id = ?", current_user.id).pluck(:user_1_id).uniq

@photo_ids = Photo.joins(follow_relationships: :photo)
@photo_ids = @photo_ids.where("user_id = ? and user_id not in (?)", current_user.id, @follower_ids)
@photo_ids = @photo_ids.pluck(:user_id).uniq

@followers = User.where("id in (?)", @follower_ids)
@photo_users = User.where("id in (?) and not in (?)", @photo_ids, @follower_ids) 

@array_size = (@follower_ids + @photo_ids).size
if @array_size < 150
  @users = User.where("verified = ? and first_name is not null and last_name is not null", true)
  @users = @users.where("id not in (?)", @photo_ids + @follower_ids).limit(150 - @array_size)
else
  @users = []
end

@final_array = @followers + @photo_users + @users

我还没有测试这是否有效，或者它是否更快。它有更多的数据库查询，但迭代次数更少。

更新

如果您向用户模型添加另一列，该列将更新为 1 到 3 的值，这取决于他们是否有关注者、照片或什么都没有。

那么你需要做的就是：

# in User model
def self.valid_users
  where("verified = ? and first_name is not null and last_name is not null", true)
end


@users = User.valid_users.order("sortable ASC").limit(150)

Answer 4

这只是猜测。

@user_list = current_user.followers.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).includes(:solutions, :photos => {:follow_relationships}).limit(150)

总的来说，我认为这一切都是错误的，因为它本身就很复杂。如果你不能清楚地阅读它在做什么，那么你应该重新开始。

Answer 5

您是否为您的表设置了表索引？

如果您还没有，请为您的所有外键和您需要包含在条件中的列设置它们。在您的数据库迁移脚本中（当然，使用正确的表名和列名）。它们将加快您的查询速度，尤其是在您拥有大型数据集的情况下：

add_index :user_follows, :follower_id
add_index :user_follows, :followed_id

add_index :photos, :user_id

add_index :photo_follow_relationships, :photo_id
add_index :photo_follow_relationships, :follower_id

add_index :users, :verified
add_index :users, :first_name
add_index :users, :last_name

另外，一些cmets和建议：

# ... [SNIP] Query and add to user list.
user_followers = []  # You are not actually using the UserFollow records. Unload
                     # them from memory. Otherwise, they will be stored until we
                     # leave the block.

# There is no need to fetch other Photo data here, and there is no need to load
# :photo for FollowRelationship. But there is a need to load :user.
photos = Photo.where(:user_id => current_user.id).select('photos.id').
    includes(:follow_relationships => [:user])
# ... [SNIP] Add to user list.
photos = []  # Unload photos.

# ... [SNIP] Your condition.
# There is no need to load :solutions for the users here.
users = User.where("verified = ? and first_name IS NOT NULL and last_name IS NOT NULL", true).limit(150 - @user_list.size)
# ... [SNIP] Add to user list.

当然，如果你也重构代码会更好，就像mind blank 的一些建议一样。您也可以使用 has_many :through 关联来清理您的控制器。

Answer 6

将以下关系添加到您的模型中

class User  
  has_many :user_follows
  has_many :inverse_user_follows, :class_name=>'UserFollow', :foreign_key=>:follower_id

  # followers for user
  has_many :followers, :through => :user_follows
  # users followed by user
  has_many :followed_users, :through => :inverse_user_follows, :source => :user


  # photos created by user
  has_many :photos

  has_many :photo_user_follows, :through => :photos, :source => :user_follows

  # followers for user's photo

  has_many :photo_followers, :through => :photo_user_follows, :source => :follower


  has_many :photo_follows
  # photos followed by user
  has_many :followed_photos, :source => :photo, :through => :photo_follows

end  

class UserFollow
  # index user_id and follower_id columns
  belongs_to :user
  belongs_to :follower, :class_name => "User"
end

照片相关模特

class Photo
  # index user_id column
  belongs_to :user
  has_many   :photo_follows
  has_many   :followers, :through => :photo_follows
end

class PhotoFollow
  # index photo_id and follower_id columns
  belongs_to :photo
  belongs_to :follower, :class_name => "User"
end  

现在您可以获取当前用户关注者或当前用户的照片关注者或活跃用户的用户..

user_ids = current_user.follower_ids | current_user.photo_follower_ids

User.where("ids IN (?) OR 
   ( verified = ? and first_name IS NOT NULL and last_name IS NOT NULL )", 
  user_ids, true).limit(150)