【问题标题】:What does Redis do when it runs out of memory?Redis 内存不足时会做什么?
【发布时间】:2011-07-01 09:50:50
【问题描述】:

这可能是一个简单的问题,但我很难找到答案。 Redis 2.0 如何处理最大分配内存不足的问题?它如何决定删除哪些数据或将哪些数据保留在内存中?

【问题讨论】:

标签: nosql redis


【解决方案1】:

Redis不像memcached那样是缓存,默认情况下(maxmemory-policy参数设置为noeviction)你放入redis的所有数据都不会被删除,唯一的例外是使用EXPIRE。

【讨论】:

  • 那么当内存耗尽时它会怎么做呢?它只会将新数据存储在磁盘而不是内存中?
  • 这(现在)不正确,Redis 有一个密钥驱逐机制,有几个可用的策略:redis.io/topics/lru-cache
  • @LoicAG:对我来说听起来完全正确......除非有过期政策,否则 Redis 不会驱逐任何密钥。这很好:例如,我不能让 Redis 自己摆脱密钥。
  • @Cory:如果设置了驱逐策略,它将删除现有密钥。但是,如果您没有设置任何驱逐政策,您应该会收到内存不足错误。
  • @Michael 我想这是一个术语问题:总是有一个 maxmemory-policy ,默认值确实是“noeviction”;但是“allkeys-lru”和“allkeys-random”策略会从整个集合中逐出键,而其他(“volatile-*”)策略会从定义了 TTL 的键子集中逐出键。
【解决方案2】:

我最近才开始阅读有关 Redis 的信息,所以我并不积极。但是,我确实遇到了一些可能有用的花絮。

这是来自http://antirez.com/post/redis-as-LRU-cache.html的sn-p:

使用 Redis 作为缓存的另一种方法是 maxmemory 指令,一个特性 允许指定最大值 要使用的内存量。当新数据 被添加到服务器,并且内存 已经达到限制,服务器 将删除一些旧数据删除一个 易失性密钥,即具有 EXPIRE(超时)设置,即使 密钥还远未到期 自动。

此外,Redis 2.0 有一个 VM 模式,其中所有键都必须适合内存,但很少使用的键的值可以在磁盘上:

【讨论】:

    【解决方案3】:

    如果您启用了虚拟内存功能(编辑:现已弃用),那么当内存耗尽时,Redis 就会开始将“不那么常​​用”的数据存储到磁盘中。 p>

    如果 Redis 中的虚拟内存被禁用(默认)并且设置了maxmemory 参数(默认),Redis 将不会使用超过maxmemory 允许的内存。如果关闭maxmemory,Redis 将开始使用虚拟内存(即交换),性能将大幅下降。

    当达到maxmemory 时,较新版本的 Redis 有各种策略:

    • volatile-lru - 删除其中的一个键 那些有过期设置的,试图 删除最近未使用的密钥。
    • volatile-ttl - 删除其中的一个键 那些有过期设置的,试图 删除剩余时间短的密钥 生活。
    • volatile-random - 删除一个 随机密钥 过期设置。
    • allkeys-lru - 喜欢 volatile-lru,但会删除每个 一种钥匙,普通钥匙或钥匙 设置了过期时间。
    • allkeys-random - 喜欢volatile-random,但会删除 各种键,都是普通键 以及设置了过期时间的密钥。

    如果你选择一个只删除带有 EXPIRE 集的键的策略,那么当 Redis 内存不足时,看起来程序只是中止了 malloc() 操作。也就是说,如果您尝试存储更多数据,写入操作就会失败。

    更多信息的链接:

    【讨论】:

    • Redis 虚拟内存现已弃用。见redis.io/topics/virtual-memory
    • 当您将 Redis 用作可回收缓存时,您会怎么做?在销毁之前,您如何管理将重要数据从 Redis 中取出并放入另一种形式的存储中?当内存填满时,你能把它和驱逐配对吗?
    • @jocull - 在数据填满之前将数据移出 Redis 是留给应用程序的一个细节。如果什么都不做,Redis 就会耗尽内存。此时,后续写入将失败。您可以监控 Redis 内存使用情况并编写自定义应用程序逻辑来移动数据。
    【解决方案4】:

    我最近遇到了没有可用内存的情况,我的应用程序停止了(无法写入,可以读取),正在运行的 PHP 脚本中途停止运行,必须kill -9'd手动(即使在内存可用之后)。

    我假设发生了数据丢失(或数据不一致),因此我执行了flushdb 并从备份中恢复。学过的知识?备份是您的朋友。

    【讨论】:

      【解决方案5】:

      来自redis.conf,2.8 版

      # Don't use more memory than the specified amount of bytes.
      # When the memory limit is reached Redis will try to remove keys
      # according to the eviction policy selected (see maxmemory-policy).
      #
      # If Redis can't remove keys according to the policy, or if the policy is
      # set to 'noeviction', Redis will start to reply with errors to commands
      # that would use more memory, like SET, LPUSH, and so on, and will continue
      # to reply to read-only commands like GET.
      #
      # This option is usually useful when using Redis as an LRU cache, or to set
      # a hard memory limit for an instance (using the 'noeviction' policy).
      #
      # WARNING: If you have slaves attached to an instance with maxmemory on,
      # the size of the output buffers needed to feed the slaves are subtracted
      # from the used memory count, so that network problems / resyncs will
      # not trigger a loop where keys are evicted, and in turn the output
      # buffer of slaves is full with DELs of keys evicted triggering the deletion
      # of more keys, and so forth until the database is completely emptied.
      #
      # In short... if you have slaves attached it is suggested that you set a lower
      # limit for maxmemory so that there is some free RAM on the system for slave
      # output buffers (but this is not needed if the policy is 'noeviction').
      #
      # maxmemory <bytes>
      
      # MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
      # is reached. You can select among five behaviors:
      #
      # volatile-lru -> remove the key with an expire set using an LRU algorithm
      # allkeys-lru -> remove any key according to the LRU algorithm
      # volatile-random -> remove a random key with an expire set
      # allkeys-random -> remove a random key, any key
      # volatile-ttl -> remove the key with the nearest expire time (minor TTL)
      # noeviction -> don't expire at all, just return an error on write operations
      #
      # Note: with any of the above policies, Redis will return an error on write
      #       operations, when there are no suitable keys for eviction.
      #
      #       At the date of writing these commands are: set setnx setex append
      #       incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
      #       sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
      #       zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
      #       getset mset msetnx exec sort
      #
      # The default is:
      #
      # maxmemory-policy volatile-lru
      

      【讨论】:

      【解决方案6】:

      如果您想知道 Redis (2.8) 在达到其配置定义的最大值时实际响应什么,它看起来像这样:

      $ redis-cli
      127.0.0.1:6379> GET 5
      "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
      127.0.0.1:6379> SET 5 a
      (error) OOM command not allowed when used memory > 'maxmemory'.
      

      【讨论】:

        【解决方案7】:

        更新 redis 4.0

        127.0.0.1:6379> MEMORY HELP
        1) "MEMORY DOCTOR                        - Outputs memory problems report"
        2) "MEMORY USAGE <key> [SAMPLES <count>] - Estimate memory usage of key"
        3) "MEMORY STATS                         - Show memory usage details"
        4) "MEMORY PURGE                         - Ask the allocator to release memory"
        5) "MEMORY MALLOC-STATS                  - Show allocator internal stats"
        

        /usr/local/etc/redis.conf

        ############################## MEMORY MANAGEMENT ################################
        
        # Set a memory usage limit to the specified amount of bytes.
        # When the memory limit is reached Redis will try to remove keys
        # according to the eviction policy selected (see maxmemory-policy).
        #
        # If Redis can't remove keys according to the policy, or if the policy is
        # set to 'noeviction', Redis will start to reply with errors to commands
        # that would use more memory, like SET, LPUSH, and so on, and will continue
        # to reply to read-only commands like GET.
        #
        # This option is usually useful when using Redis as an LRU or LFU cache, or to
        # set a hard memory limit for an instance (using the 'noeviction' policy).
        #
        # WARNING: If you have slaves attached to an instance with maxmemory on,
        # the size of the output buffers needed to feed the slaves are subtracted
        # from the used memory count, so that network problems / resyncs will
        # not trigger a loop where keys are evicted, and in turn the output
        # buffer of slaves is full with DELs of keys evicted triggering the deletion
        # of more keys, and so forth until the database is completely emptied.
        #
        # In short... if you have slaves attached it is suggested that you set a lower
        # limit for maxmemory so that there is some free RAM on the system for slave
        # output buffers (but this is not needed if the policy is 'noeviction').
        #
        # maxmemory <bytes>
        
        # MAXMEMORY POLICY: how Redis will select what to remove when maxmemory
        # is reached. You can select among five behaviors:
        #
        # volatile-lru -> Evict using approximated LRU among the keys with an expire set.
        # allkeys-lru -> Evict any key using approximated LRU.
        # volatile-lfu -> Evict using approximated LFU among the keys with an expire set.
        # allkeys-lfu -> Evict any key using approximated LFU.
        # volatile-random -> Remove a random key among the ones with an expire set.
        # allkeys-random -> Remove a random key, any key.
        # volatile-ttl -> Remove the key with the nearest expire time (minor TTL)
        # noeviction -> Don't evict anything, just return an error on write operations.
        #
        # LRU means Least Recently Used
        # LFU means Least Frequently Used
        #
        # Both LRU, LFU and volatile-ttl are implemented using approximated
        # randomized algorithms.
        #
        # Note: with any of the above policies, Redis will return an error on write
        #       operations, when there are no suitable keys for eviction.
        #
        #       At the date of writing these commands are: set setnx setex append
        #       incr decr rpush lpush rpushx lpushx linsert lset rpoplpush sadd
        #       sinter sinterstore sunion sunionstore sdiff sdiffstore zadd zincrby
        #       zunionstore zinterstore hset hsetnx hmset hincrby incrby decrby
        #       getset mset msetnx exec sort
        #
        # The default is:
        #
        # maxmemory-policy noeviction
        
        # LRU, LFU and minimal TTL algorithms are not precise algorithms but approximated
        # algorithms (in order to save memory), so you can tune it for speed or
        # accuracy. For default Redis will check five keys and pick the one that was
        # used less recently, you can change the sample size using the following
        # configuration directive.
        #
        # The default of 5 produces good enough results. 10 Approximates very closely
        # true LRU but costs more CPU. 3 is faster but not very accurate.
        #
        # maxmemory-samples 5
        

        【讨论】:

          猜你喜欢
          • 2011-11-29
          • 1970-01-01
          • 2013-12-16
          • 1970-01-01
          • 1970-01-01
          • 2012-10-27
          • 2021-12-13
          • 1970-01-01
          • 1970-01-01
          相关资源
          最近更新 更多