Docker容器应用程序日志到ELK堆栈没有filebeat

Question

我正在使用 Elasti Cloud，因为它似乎最适合快速设置应用程序日志记录。我有 24 个 docker 容器在不同的节点上运行，有些容器也没有副本。我想将 docker 容器内的日志导出到 elk 堆栈。我不想在我的每个容器上安装 Filebeat，因为这似乎直接违背了 Docker 的职责分离原则。

....如何从我的应用程序容器中获取日志以记录存储服务器

Answer 1

您可以通过像这样配置rsyslogd 将您的系统日志发送到 Logstash

# /etc/rsyslog.d/99-ship-syslog.conf
*.*;syslog;auth,authpriv.none action(
  type="omfwd"
  Target="myremote.elk-server.net"
  Port="5001"
  Protocol="udp"
)

如果您还没有运行rsyslog，您可以像这样添加它（alpine linux 示例）：

# Dockerfile
FROM alpine:3.7

RUN  apk update \
  && apk add rsyslog

COPY rsyslog.conf /etc/rsyslog.conf

EXPOSE 514 514/udp

VOLUME [ "/var/log", "/etc/rsyslog.d" ]
ENTRYPOINT [ "rsyslogd", "-n" ]

--

# rsyslogd.conf
#
# if you experience problems, check:
# http://www.rsyslog.com/troubleshoot

#### MODULES ####

module(load="imuxsock")    # local system logging support (e.g. via logger command)
#module(load="imklog")     # kernel logging support (previously done by rklogd)
module(load="immark")      # --MARK-- message support
module(load="imudp")       # UDP listener support


input(type="imudp" port="514")

# Log all kernel messages to the console.
# Logging much else clutters up the screen.
#kern.*                                                 action(type="omfile" file="/dev/console")

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none                action(type="omfile" file="/var/log/messages")

# The authpriv file has restricted access.
authpriv.*                                              action(type="omfile" file="/var/log/secure")

# Log all the mail messages in one place.
mail.*                                                  action(type="omfile" file="/var/log/maillog")

# Log cron stuff
cron.*                                                  action(type="omfile" file="/var/log/cron")

# Everybody gets emergency messages
*.emerg                                                 action(type="omusrmsg" users="*")

# Save news errors of level crit and higher in a special file.
uucp,news.crit                                          action(type="omfile" file="/var/log/spooler")

# Save boot messages also to boot.log
local7.*                                                action(type="omfile" file="/var/log/boot.log")

# log every host in its own directory
if $fromhost-ip then /var/log/$fromhost-ip/messages

# Include all .conf files in /etc/rsyslog.d
$IncludeConfig /etc/rsyslog.d/*.conf
$template GRAYLOGRFC5424,"<%PRI%>%PROTOCOL-VERSION% %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n"
*.info;mail.none;authpriv.none;cron.none;*.* @@graylog:514;GRAYLOGRFC5424 # forward everything to remote server

当您在 java 应用程序中运行时，您甚至可以将日志直接发送到 syslog。这是一个使用 log4j 的小配置示例

log4j.rootLogger=INFO, SYSLOG

log4j.appender.SYSLOG=org.apache.log4j.net.SyslogAppender
log4j.appender.SYSLOG.syslogHost=myremote.elk-server.net
log4j.appender.SYSLOG.layout=org.apache.log4j.PatternLayout
log4j.appender.SYSLOG.layout.conversionPattern=%d{ISO8601} %-5p [%t] %c{2} %x - %m%n
log4j.appender.SYSLOG.Facility=LOCAL1