【发布时间】:2022-08-18 15:42:21
【问题描述】:
我首先创建了一个 docker 文件来安装 kafka 和 kafka s3 连接器:
FROM bitnami/kafka:2.4.1
LABEL description=\"This images install bitnami kafka2.4.1 and kafka s3 sink connector 10.1.0\"
USER root
# general container environment
ENV JAVA_HOME=/opt/bitnami/java/
ENV PWD=/opt/bitnami/kafka/bin
ENV PATH= /opt/bitnami/java/bin:/opt/bitnami/common/bin:/opt/bitnami/kafka/bin:/opt/bitnami/common/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
# kafka environment properties
ENV KAFKA_BROKER_ID=1
ENV KAFKA_CFG_LISTENERS=PLAINTEXT://:9092
ENV KAFKA_CFG_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
**ENV KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181**
ENV ALLOW_PLAINTEXT_LISTENER=yes
ENV KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS=100
# kafka s3 sink connector environment properties
ENV CONNECT_GROUP_ID=kafka-connect
ENV CONNECT_CONFIG_STORAGE_TOPIC=_kafka-connect-configs
ENV CONNECT_OFFSET_STORAGE_TOPIC=_kafka-connect-offsets
ENV CONNECT_STATUS_STORAGE_TOPIC=_kafka-connect-status
ENV CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR=\"1\"
ENV CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR=\"1\"
ENV CONNECT_STATUS_STORAGE_REPLICATION_FACTOR=\"1\"
ENV CONNECT_PLUGIN_PATH=\'/plugins/,/usr/share/java,/usr/share/confluent-hub-components/,/connectors/\'
# preparation for install
# Installing bash tools: top ping wget vi and ps
RUN apt-get update -y && apt-get install -y procps && apt-get install -y iputils-ping && apt-get install -y wget && \\
apt-get install -y vim && apt-get install -y procps && apt-get install -y netcat
# download and install connector
RUN mkdir -p /plugins/lib
# download and install kafka s3 connector zip
RUN cd /plugins && \\
wget https://api.hub.confluent.io/api/plugins/confluentinc/kafka-connect-s3/versions/10.1.0/archive && \\
unzip ./archive && \\
cp -f confluentinc-kafka-connect-s3-10.1.0/lib/* /plugins/lib/
# install s3 connector properties files
COPY ./connector.properties /plugins/.
COPY ./s3-sink.properties /plugins/.
WORKDIR /opt/bitnami/kafka/bin
请注意 ENV KAFKA_CFG_ZOOKEEPER_CONNECT zookeeper:2181 这一行 zookeeper 是我稍后将在 docker-composer.yml 文件中定义的服务名称
我创建了一个 docker compose yml 文件,如下所示:
version: \"2\"
services:
zookeeper:
image: \'bitnami/zookeeper:latest\'
ports:
- 2181:2181
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
kafka-s3-connector:
image: \'bitnami.kafka.and.s3connector:2.4.1\'
user: root
ports:
- 9092:9092
environment:
...
最后当我运行 docker-compose up -d 启动了两个容器:zookeeper和kafka+connector
但是,如果我登录到 kafka+connector 容器, 我发现kafka没有启动,我怀疑是因为无法连接zookeeper。 如果我将 docker-compose.yml 文件更改为直接使用 bitnami kafka2.4.1 图像并指定相同的 KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181 kafka服务可以成功启动。
将 zookeeper 服务暴露给 docker compose 服务中包含的嵌套图像有什么建议吗?
标签: image docker-compose service environment-variables