如何在一段时间内聚合python任务并且每个任务只执行一次

Question

考虑一个通用应用程序，每次用户做好事时都会给他们积分

我有模型

class GoodDeed(models.Model):
     user = models.ForgeinKey(CustomUser)
     points = models.IntegerField(default=0, blank=True)

class CustomUser(models.Model):
     points = models.IntegerField(default=0, blank=True)
     rank = models.IntegerField(blank=True, null=True)

现在，每次添加新契约时，都会重新计算该用户的总积分，并且每次重新计算用户的积分时，也会重新计算所有用户（相对于其他用户）的排名。

这太可怕了，我想汇总所有排名更新请求并每隔几分钟执行一次。有没有可以做这种事情的python库？

cron 作业是一个选项，但它会每隔几分钟更新一次所有用户的排名，无论是否需要更新。

编辑：根据@The Django Ninja 的建议，我编写了这个装饰器并分享它

用法：

    class CustomUser(models.Model):

        @cached_model_property
        def points(self):
            del self.rank  # removed cached rank value
            return sum(self.good_deeds.values_list("points", flat=True))

        @cached_model_property(readonly=False)
        def rank(self):
            user_list = []
            for user in User.onjects.all().only("points")
                user_list.push((user.points, user))
            # sort the list
            user_list.sort(reverse=True)
            # update all ranking
            rank = 0
            my_rank = None
            for points, user in user_list:
                rank += 1
                if user.pk == self.pk:
                    # Don't update this instance's rank yet, as it will be
                    # updated when the function return
                    my_rank = rank
                    continue
                # Save the new rank in the cache
                user.rank = rank
            # If you forget to return something (not None) caching will always assumed to be invalid
            return my_rank

效果

>>> user.points
# call points() method, cache the result, return it
>>> user.points
# return cached result without calling points() method
>>> del user.points
>>> user.points
# call points() method, cache the result, return it

>>> user.rank
# 1. Calculate and save (in cache) rank of all users except current user
# 2. Cache current user's rank and return it
>>> user.rank
# return cached user's rank
>>> del user.points
>>> user.rank
# Recalculate user's points and all ranking values 
>>> user.points = 9
# Readonly property exception

id 为 20 的用户的缓存排名值将存储在名为 'User.20.rank'

装饰器代码

def cached_model_property(f=None, **kwargs):
    """
    cached_model_property is a decorator for model functions that takes no arguments
    The function is converted into a property that support caching out of the box
    Sample usage:

    class Team(models.Model):

        @cached_model_property
        def points(self):
            # Do complex DB queries
            return result

        @cached_model_property(readonly=False)
        def editable_points(self):
            # get result
            return result

    Now try
    team = Team.objects.first()
    team.points  <-- complex DB queries will happen, result will be returned
    team.points  <-- this time result is returned from cache (points function is not called at all!
    del team.points <-- points value has been removed from cache
    team.points  <-- complex DB queries will happen, result will be returned

    set readlonly parameter False to make the property writeable
    team.editable_points = 88
    in this case the assigned value will replace the value stored in the cache
    team.editable_points
    returns 88
    """

    readonly = kwargs.get("readonly", True)

    def func(f):
        def _get_cache_key(obj):
            model_name = obj.__class__.__name__
            method_name = f.__name__
            return "%s.%s.%s" % (model_name, obj.pk, method_name)

        def getX(obj):
            """
            This decorator can convert any function that **doesn't** take any arguments into a cached property
            :param obj: model object instance (python provide it by default to class members)
            :return: the cached value if present otherwise call the actual method

            Note:
            This decorator doesn't work if the model function return None.
            The function will be called as long as it return None, no caching!
            """
            # Try to get the cache key for that method
            cache_key = _get_cache_key(obj)
            result = cache.get(cache_key)
            # If not cached, call the actual method and cache the result
            if result is None:
                result = f(obj)
                cache.set(cache_key, result)
            return result

        def delX(obj):
            """
            Remove that property from the cache
            :param obj:
            :return: None
            """
            cache_key = _get_cache_key(obj)
            # Remove that key from the cache
            cache.delete(cache_key)

        def setX(obj, value):
            """
            Set the cache value of that property
            :param obj:
            :return: None
            """
            cache_key = _get_cache_key(obj)
            # Remove that key from the cache
            cache.set(cache_key, value)

        if readonly:
            return property(fget=getX, fdel=delX)
        else:
            return property(fget=getX, fset=setX, fdel=delX)

    # f (= class method) is passed when using @cached_model_property
    if f:
        return func(f)
    # f is not passed when using @cached_model_property(readonly=True) or even @cached_model_property()
    return f

Answer 1

如上所述，排名任务似乎在数据库上做了一些繁重的工作。

我会选择一个基本的缓存失效解决方案，如果用户排名更新，缓存应该失效：

from django.core.cache import cache

def the_function_that_update_user_rank()
    # updating the rank of the user
    # when the rank is updated we set our cache key (global-rank-update-needed) to True
    cache.set('global-rank-update-needed', 'True')

然后，一个 cron 作业应该每 x 秒调用一次 manage.py 命令，该命令将从缓存中检查 'global-rank-update-needed' 是否为 True，然后更新全局排名：

from django.core.cache import cache

class UpdateGlobalRank(BaseCommand):
    help = 'Closes the specified poll for voting'


    def handle(self, *args, **options):
        if cache.get('global-rank-update-needed') == 'True':
            # here do the update rank stuff
            print 'Updating the Global Rank'
        cache.set('global-rank-update-needed', 'False')                         

按照这种方法，全局排名只有在某些用户的排名发生变化时才会更新

希望对您有所帮助。

Answer 2

对于这些类型的任务，我更喜欢 celery。

t是一个任务队列，专注于实时处理，同时也支持任务调度