【问题标题】:Getting a lost Sentinel error message for Redis获取 Redis 丢失的 Sentinel 错误消息
【发布时间】:2015-06-15 21:51:32
【问题描述】:

我正在使用 spring data redis 运行一个 spring boot 服务,这里是以下配置。

该服务似乎可以正常工作,但我在日志中看到了一连串 Lost Sentinel 消息。可以从我运行服务的 VM 访问哨兵节点。我能够直接从该虚拟机远程登录到他们。知道为什么会这样吗?

spring:
  profiles:
    active: core-perf,swagger
    default: core-perf,swagger
  redis:
    Pool:  #Pool properties
      # Max number of "idle" connections in the pool. Use a negative value to indicate
      # an unlimited number of idle connections.
      maxIdle: 8
      # Target for the minimum number of idle connections to maintain in the pool.
      # This setting only has an effect if it is positive.
      minIdle: 0
      # Max number of connections that can be allocated by the pool at a given time. Use a negative value for no limit.
      maxActive: 8
      # Maximum amount of time (in milliseconds) a connection allocation should block
      # before throwing an exception when the pool is exhausted. Use a negative value
      # to block indefinitely.
      maxWait: -1
    sentinel: #Redis sentinel properties.
      master: mymaster
      nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379

2015-06-15 17:30:54.896 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.894 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:30:59.897 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.975 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:04.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:09.976 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.054 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:15.055 ERROR 6677 --- [Thread-9] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.58.80:26379. Sleeping 5000ms and retrying.
2015-06-15 17:31:20.055 ERROR 6677 --- [Thread-8] redis.clients.jedis.JedisSentinelPool    : Lost connection to Sentinel at  10.202.56.213:26379. Sleeping 5000ms and retrying.

【问题讨论】:

    标签: redis spring-boot spring-data-redis


    【解决方案1】:

    我们发现了这个问题。 application.yml 中的节点对之间有一个空白,一旦我们删除这个“”,Lost Sentinel 日志消息就消失了。

    所以从

    nodes: 10.202.56.209:26379, 10.202.56.213:26379, 10.202.58.80:26379
    

    nodes: 10.202.56.209:26379,10.202.56.213:26379,10.202.58.80:26379
    

    如果提交者看到这可能是一件好事,因为它对用户来说似乎有些神秘。

    【讨论】:

      【解决方案2】:

      我在这个问题上迷失了两天,直到我决定重新安装它,然后通过我的旧配置文件配置它,然后用下面的文本替换 sentinel.conf 文件。 终于成功了。

      # *** IMPORTANT ***
      #
      # By default Sentinel will not be reachable from interfaces different than
      # localhost, either use the 'bind' directive to bind to a list of network
      # interfaces, or disable protected mode with "protected-mode no" by
      # adding it to this configuration file.
      #
      # Before doing that MAKE SURE the instance is protected from the outside
      # world via firewalling or other means.
      #
      # For example you may use one of the following:
      #
      # bind 127.0.0.1 192.168.1.1
      #
      # protected-mode no
      
      # port <sentinel-port>
      # The port that this sentinel instance will run on
      port 26379
      
      # By default Redis Sentinel does not run as a daemon. Use 'yes' if you need it.
      # Note that Redis will write a pid file in /var/run/redis-sentinel.pid when
      # daemonized.
      daemonize no
      
      # When running daemonized, Redis Sentinel writes a pid file in
      # /var/run/redis-sentinel.pid by default. You can specify a custom pid file
      # location here.
      pidfile /var/run/redis-sentinel.pid
      
      # Specify the log file name. Also the empty string can be used to force
      # Sentinel to log on the standard output. Note that if you use standard
      # output for logging but daemonize, logs will be sent to /dev/null
      logfile ""
      
      # sentinel announce-ip <ip>
      # sentinel announce-port <port>
      #
      # The above two configuration directives are useful in environments where,
      # because of NAT, Sentinel is reachable from outside via a non-local address.
      #
      # When announce-ip is provided, the Sentinel will claim the specified IP address
      # in HELLO messages used to gossip its presence, instead of auto-detecting the
      # local address as it usually does.
      #
      # Similarly when announce-port is provided and is valid and non-zero, Sentinel
      # will announce the specified TCP port.
      #
      # The two options don't need to be used together, if only announce-ip is
      # provided, the Sentinel will announce the specified IP and the server port
      # as specified by the "port" option. If only announce-port is provided, the
      # Sentinel will announce the auto-detected local IP and the specified port.
      #
      # Example:
      #
      # sentinel announce-ip 1.2.3.4
      
      # dir <working-directory>
      # Every long running process should have a well-defined working directory.
      # For Redis Sentinel to chdir to /tmp at startup is the simplest thing
      # for the process to don't interfere with administrative tasks such as
      # unmounting filesystems.
      dir /tmp
      
      # sentinel monitor <master-name> <ip> <redis-port> <quorum>
      #
      # Tells Sentinel to monitor this master, and to consider it in O_DOWN
      # (Objectively Down) state only if at least <quorum> sentinels agree.
      #
      # Note that whatever is the ODOWN quorum, a Sentinel will require to
      # be elected by the majority of the known Sentinels in order to
      # start a failover, so no failover can be performed in minority.
      #
      # Replicas are auto-discovered, so you don't need to specify replicas in
      # any way. Sentinel itself will rewrite this configuration file adding
      # the replicas using additional configuration options.
      # Also note that the configuration file is rewritten when a
      # replica is promoted to master.
      #
      # Note: master name should not include special characters or spaces.
      # The valid charset is A-z 0-9 and the three characters ".-_".
      sentinel monitor mymaster 127.0.0.1 6379 2
      
      # sentinel auth-pass <master-name> <password>
      #
      # Set the password to use to authenticate with the master and replicas.
      # Useful if there is a password set in the Redis instances to monitor.
      #
      # Note that the master password is also used for replicas, so it is not
      # possible to set a different password in masters and replicas instances
      # if you want to be able to monitor these instances with Sentinel.
      #
      # However you can have Redis instances without the authentication enabled
      # mixed with Redis instances requiring the authentication (as long as the
      # password set is the same for all the instances requiring the password) as
      # the AUTH command will have no effect in Redis instances with authentication
      # switched off.
      #
      # Example:
      #
      # sentinel auth-pass mymaster MySUPER--secret-0123passw0rd
      
      # sentinel auth-user <master-name> <username>
      #
      # This is useful in order to authenticate to instances having ACL capabilities,
      # that is, running Redis 6.0 or greater. When just auth-pass is provided the
      # Sentinel instance will authenticate to Redis using the old "AUTH <pass>"
      # method. When also an username is provided, it will use "AUTH <user> <pass>".
      # In the Redis servers side, the ACL to provide just minimal access to
      # Sentinel instances, should be configured along the following lines:
      #
      #     user sentinel-user >somepassword +client +subscribe +publish \
      #                        +ping +info +multi +slaveof +config +client +exec on
      
      # sentinel down-after-milliseconds <master-name> <milliseconds>
      #
      # Number of milliseconds the master (or any attached replica or sentinel) should
      # be unreachable (as in, not acceptable reply to PING, continuously, for the
      # specified period) in order to consider it in S_DOWN state (Subjectively
      # Down).
      #
      # Default is 30 seconds.
      sentinel down-after-milliseconds mymaster 30000
      
      # IMPORTANT NOTE: starting with Redis 6.2 ACL capability is supported for
      # Sentinel mode, please refer to the Redis website https://redis.io/topics/acl
      # for more details.
      
      # Sentinel's ACL users are defined in the following format:
      #
      #   user <username> ... acl rules ...
      #
      # For example:
      #
      #   user worker +@admin +@connection ~* on >ffa9203c493aa99
      #
      # For more information about ACL configuration please refer to the Redis
      # website at https://redis.io/topics/acl and redis server configuration 
      # template redis.conf.
      
      # ACL LOG
      #
      # The ACL Log tracks failed commands and authentication events associated
      # with ACLs. The ACL Log is useful to troubleshoot failed commands blocked 
      # by ACLs. The ACL Log is stored in memory. You can reclaim memory with 
      # ACL LOG RESET. Define the maximum entry length of the ACL Log below.
      acllog-max-len 128
      
      # Using an external ACL file
      #
      # Instead of configuring users here in this file, it is possible to use
      # a stand-alone file just listing users. The two methods cannot be mixed:
      # if you configure users here and at the same time you activate the external
      # ACL file, the server will refuse to start.
      #
      # The format of the external ACL user file is exactly the same as the
      # format that is used inside redis.conf to describe users.
      #
      # aclfile /etc/redis/sentinel-users.acl
      
      # requirepass <password>
      #
      # You can configure Sentinel itself to require a password, however when doing
      # so Sentinel will try to authenticate with the same password to all the
      # other Sentinels. So you need to configure all your Sentinels in a given
      # group with the same "requirepass" password. Check the following documentation
      # for more info: https://redis.io/topics/sentinel
      #
      # IMPORTANT NOTE: starting with Redis 6.2 "requirepass" is a compatibility
      # layer on top of the ACL system. The option effect will be just setting
      # the password for the default user. Clients will still authenticate using
      # AUTH <password> as usually, or more explicitly with AUTH default <password>
      # if they follow the new protocol: both will work.
      #
      # New config files are advised to use separate authentication control for
      # incoming connections (via ACL), and for outgoing connections (via
      # sentinel-user and sentinel-pass) 
      #
      # The requirepass is not compatable with aclfile option and the ACL LOAD
      # command, these will cause requirepass to be ignored.
      
      # sentinel sentinel-user <username>
      #
      # You can configure Sentinel to authenticate with other Sentinels with specific
      # user name. 
      
      # sentinel sentinel-pass <password>
      #
      # The password for Sentinel to authenticate with other Sentinels. If sentinel-user
      # is not configured, Sentinel will use 'default' user with sentinel-pass to authenticate.
      
      # sentinel parallel-syncs <master-name> <numreplicas>
      #
      # How many replicas we can reconfigure to point to the new replica simultaneously
      # during the failover. Use a low number if you use the replicas to serve query
      # to avoid that all the replicas will be unreachable at about the same
      # time while performing the synchronization with the master.
      sentinel parallel-syncs mymaster 1
      
      # sentinel failover-timeout <master-name> <milliseconds>
      #
      # Specifies the failover timeout in milliseconds. It is used in many ways:
      #
      # - The time needed to re-start a failover after a previous failover was
      #   already tried against the same master by a given Sentinel, is two
      #   times the failover timeout.
      #
      # - The time needed for a replica replicating to a wrong master according
      #   to a Sentinel current configuration, to be forced to replicate
      #   with the right master, is exactly the failover timeout (counting since
      #   the moment a Sentinel detected the misconfiguration).
      #
      # - The time needed to cancel a failover that is already in progress but
      #   did not produced any configuration change (SLAVEOF NO ONE yet not
      #   acknowledged by the promoted replica).
      #
      # - The maximum time a failover in progress waits for all the replicas to be
      #   reconfigured as replicas of the new master. However even after this time
      #   the replicas will be reconfigured by the Sentinels anyway, but not with
      #   the exact parallel-syncs progression as specified.
      #
      # Default is 3 minutes.
      sentinel failover-timeout mymaster 180000
      
      # SCRIPTS EXECUTION
      #
      # sentinel notification-script and sentinel reconfig-script are used in order
      # to configure scripts that are called to notify the system administrator
      # or to reconfigure clients after a failover. The scripts are executed
      # with the following rules for error handling:
      #
      # If script exits with "1" the execution is retried later (up to a maximum
      # number of times currently set to 10).
      #
      # If script exits with "2" (or an higher value) the script execution is
      # not retried.
      #
      # If script terminates because it receives a signal the behavior is the same
      # as exit code 1.
      #
      # A script has a maximum running time of 60 seconds. After this limit is
      # reached the script is terminated with a SIGKILL and the execution retried.
      
      # NOTIFICATION SCRIPT
      #
      # sentinel notification-script <master-name> <script-path>
      # 
      # Call the specified notification script for any sentinel event that is
      # generated in the WARNING level (for instance -sdown, -odown, and so forth).
      # This script should notify the system administrator via email, SMS, or any
      # other messaging system, that there is something wrong with the monitored
      # Redis systems.
      #
      # The script is called with just two arguments: the first is the event type
      # and the second the event description.
      #
      # The script must exist and be executable in order for sentinel to start if
      # this option is provided.
      #
      # Example:
      #
      # sentinel notification-script mymaster /var/redis/notify.sh
      
      # CLIENTS RECONFIGURATION SCRIPT
      #
      # sentinel client-reconfig-script <master-name> <script-path>
      #
      # When the master changed because of a failover a script can be called in
      # order to perform application-specific tasks to notify the clients that the
      # configuration has changed and the master is at a different address.
      # 
      # The following arguments are passed to the script:
      #
      # <master-name> <role> <state> <from-ip> <from-port> <to-ip> <to-port>
      #
      # <state> is currently always "failover"
      # <role> is either "leader" or "observer"
      # 
      # The arguments from-ip, from-port, to-ip, to-port are used to communicate
      # the old address of the master and the new address of the elected replica
      # (now a master).
      #
      # This script should be resistant to multiple invocations.
      #
      # Example:
      #
      # sentinel client-reconfig-script mymaster /var/redis/reconfig.sh
      
      # SECURITY
      #
      # By default SENTINEL SET will not be able to change the notification-script
      # and client-reconfig-script at runtime. This avoids a trivial security issue
      # where clients can set the script to anything and trigger a failover in order
      # to get the program executed.
      
      sentinel deny-scripts-reconfig yes
      
      # REDIS COMMANDS RENAMING
      #
      # Sometimes the Redis server has certain commands, that are needed for Sentinel
      # to work correctly, renamed to unguessable strings. This is often the case
      # of CONFIG and SLAVEOF in the context of providers that provide Redis as
      # a service, and don't want the customers to reconfigure the instances outside
      # of the administration console.
      #
      # In such case it is possible to tell Sentinel to use different command names
      # instead of the normal ones. For example if the master "mymaster", and the
      # associated replicas, have "CONFIG" all renamed to "GUESSME", I could use:
      #
      # SENTINEL rename-command mymaster CONFIG GUESSME
      #
      # After such configuration is set, every time Sentinel would use CONFIG it will
      # use GUESSME instead. Note that there is no actual need to respect the command
      # case, so writing "config guessme" is the same in the example above.
      #
      # SENTINEL SET can also be used in order to perform this configuration at runtime.
      #
      # In order to set a command back to its original name (undo the renaming), it
      # is possible to just rename a command to itself:
      #
      # SENTINEL rename-command mymaster CONFIG CONFIG
      
      # HOSTNAMES SUPPORT
      #
      # Normally Sentinel uses only IP addresses and requires SENTINEL MONITOR
      # to specify an IP address. Also, it requires the Redis replica-announce-ip
      # keyword to specify only IP addresses.
      #
      # You may enable hostnames support by enabling resolve-hostnames. Note
      # that you must make sure your DNS is configured properly and that DNS
      # resolution does not introduce very long delays.
      #
      SENTINEL resolve-hostnames no
      
      # When resolve-hostnames is enabled, Sentinel still uses IP addresses
      # when exposing instances to users, configuration files, etc. If you want
      # to retain the hostnames when announced, enable announce-hostnames below.
      #
      SENTINEL announce-hostnames no
      

      【讨论】:

        猜你喜欢
        • 1970-01-01
        • 1970-01-01
        • 2018-08-22
        • 1970-01-01
        • 2015-05-26
        • 1970-01-01
        • 1970-01-01
        • 2016-04-03
        • 1970-01-01
        相关资源
        最近更新 更多