【问题标题】:AWS beanstalk docker exception: `shim reaped`AWS beanstalk docker 异常:`shim reaped`
【发布时间】:2019-05-09 08:00:13
【问题描述】:

我目前正在使用 docker 环境在 AWS beanstalk 上部署一个应用程序(nodejs websocket 服务器)。

周期性地,容器'crash'(实际上是容器中的主进程重启),我不知道为什么。 /var/log/docker 包含这些日志(在事件发生的确切时刻):

time="2018-12-07T00:48:46Z" level=info msg="shim reaped" id=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f 
time="2018-12-07T00:48:46.052832134Z" level=info msg="ignoring event" module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2018-12-07T00:48:46Z" level=info msg="shim docker-containerd-shim started" address="/containerd-shim/moby/0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f/shim.sock" debug=false pid=9192

此时 CPU 和 RAM 似乎还可以。 有人有线索吗?

编辑 还有其他日志,但我怀疑是后果:

/var/log/nginx/error.log:

2018/12/07 00:48:45 [error] 4268#0: *10397 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 172.31.43.209, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."
2018/12/07 00:48:45 [error] 4268#0: *1009 recv() failed (104: Connection reset by peer) while proxying upgraded connection, client: 172.31.43.209, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."
2018/12/07 00:48:46 [error] 4267#0: *11092 connect() failed (111: Connection refused) while connecting to upstream, client: 172.31.12.149, server: , request: "GET /stream?s=000 HTTP/1.1", upstream: "http://172.17.0.2:80/stream?s=000", host: "..."

/var/log/docker-events.log

2018-12-07T00:48:46.052880449Z container die 0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f (exitCode=1, image=2fc4abcada2b, name=inspiring_euler)
2018-12-07T00:48:46.176330610Z network disconnect 94c449d445a5a434af70517a1c8734c540c5c1f9ddbbc1a53a002f25dbc7f581 (container=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f, name=bridge, type=bridge)
2018-12-07T00:48:46.626514590Z network connect 94c449d445a5a434af70517a1c8734c540c5c1f9ddbbc1a53a002f25dbc7f581 (container=0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f, name=bridge, type=bridge)
2018-12-07T00:48:46.869988171Z container start 0af18fa159c07b167a29012b34c6c925c877f98d9a09dcd67078aa6c12f4ef2f (image=2fc4abcada2b, name=inspiring_euler)

【问题讨论】:

  • 您能否检查一下您的内存是否足够? docker run -it kunalkushwaha/high-mem-allocator 4 可以模拟内存使用情况。

标签: amazon-web-services docker amazon-elastic-beanstalk shim


【解决方案1】:

此故障可能是由于容器运行在启用了 THP(透明大页)的系统上。内存管理方案与导致失败的容器的内存分配模式不一致。 https://github.com/containerd/containerd/issues/2202

报告了类似的问题

很遗憾,您无法调整 Elastic Beanstalk 主机的内核设置来解决此问题。该解决方案记录在 mongodb 中,因为它与 THP 存在类似问题。

https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/

【讨论】:

    猜你喜欢
    • 2019-12-02
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2014-12-22
    • 2017-07-26
    • 2015-04-16
    • 2015-09-15
    • 2012-03-24
    相关资源
    最近更新 更多