【问题标题】:How to connect Presto to Mongodb container如何将 Presto 连接到 Mongodb 容器
【发布时间】:2019-12-16 16:02:23
【问题描述】:

问题:无法将 presto 容器连接到 mongodb 容器。

我有三个正在运行的容器:

    [root@localhost tmp]# docker container ls
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                      NAMES
8f6686547c55        mongo               "docker-entrypoint.s…"   6 minutes ago       Up 6 minutes        0.0.0.0:27017->27017/tcp   tmp_mongo_1
0b639ef12c12        mongo-express       "tini -- /docker-ent…"   6 minutes ago       Up 6 minutes        0.0.0.0:8081->8081/tcp     tmp_mongo-express_1
c51178e29a3e        prestosql/presto    "/usr/lib/presto/bin…"   4 days ago          Up About an hour    0.0.0.0:8080->8080/tcp     presto

presto 容器可以访问 mongo db(网络方式):

[root@c51178e29a3e /]# ping mongo
PING mongo (172.20.0.3) 56(84) bytes of data.
64 bytes from tmp_mongo_1.tmp_default (172.20.0.3): icmp_seq=1 ttl=64 time=0.130 ms
64 bytes from tmp_mongo_1.tmp_default (172.20.0.3): icmp_seq=2 ttl=64 time=0.108 ms

也配置了 Presto 属性(/usr/lib/presto/default/etc/catalog/mongodb.properties):

connector.name=mongodb
mongodb.seeds=mongo:27017
mongodb.credentials=root:example@liran

“liran”集合存在:

尝试访问 mongo 的“liran”集合时:

    presto> SELECT COUNT(*) FROM mongodb.local.liran;
Query 20191216_154753_00000_sz73z failed: com.mongodb.MongoTimeoutException: Timed out after 30000 ms while waiting to connect. Client view of cluster state is {type=UNKNOWN, servers=[{address=mongo:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSecurityException: Exception authenticating MongoCredential{mechanism=null, userName='root', source='liran', password=<hidden>, mechanismProperties={}}}, caused by {com.mongodb.MongoCommandException: Command failed with error 18: 'Authentication failed.' on server mongo:27017. The full response is { "ok" : 0.0, "errmsg" : "Authentication failed.", "code" : 18, "codeName" : "AuthenticationFailed" }}}]

在 mongo 上我得到:

    mongo_1          | 2019-12-16T15:51:00.795+0000 I  NETWORK  [conn322] end connection 172.20.0.4:34810 (4 connections now open)
mongo_1          | 2019-12-16T15:51:10.796+0000 I  NETWORK  [listener] connection accepted from 172.20.0.4:34812 #323 (5 connections now open)
mongo_1          | 2019-12-16T15:51:10.796+0000 I  NETWORK  [conn323] received client metadata from 172.20.0.4:34812 conn323: { driver: { name: "mongo-java-driver", version: "3.6.0" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.11.6.el7.x86_64" }, platform: "Java/Oracle Corporation/11.0.4+11-LTS" }
mongo_1          | 2019-12-16T15:51:10.798+0000 I  ACCESS   [conn323] SASL SCRAM-SHA-1 authentication failed for root on liran from client 172.20.0.4:34812 ; UserNotFound: Could not find user "root" for db "liran"
mongo_1          | 2019-12-16T15:51:10.799+0000 I  NETWORK  [conn323] end connection 172.20.0.4:34812 (4 connections now open)
mongo_1          | 2019-12-16T15:51:20.800+0000 I  NETWORK  [listener] connection accepted from 172.20.0.4:34818 #324 (5 connections now open)
mongo_1          | 2019-12-16T15:51:20.801+0000 I  NETWORK  [conn324] received client metadata from 172.20.0.4:34818 conn324: { driver: { name: "mongo-java-driver", version: "3.6.0" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.11.6.el7.x86_64" }, platform: "Java/Oracle Corporation/11.0.4+11-LTS" }
mongo_1          | 2019-12-16T15:51:20.803+0000 I  ACCESS   [conn324] SASL SCRAM-SHA-1 authentication failed for root on liran from client 172.20.0.4:34818 ; UserNotFound: Could not find user "root" for db "liran"
mongo_1          | 2019-12-16T15:51:20.804+0000 I  NETWORK  [conn324] end connection 172.20.0.4:34818 (4 connections now open)
mongo_1          | 2019-12-16T15:51:30.806+0000 I  NETWORK  [listener] connection accepted from 172.20.0.4:34820 #325 (5 connections now open)
mongo_1          | 2019-12-16T15:51:30.807+0000 I  NETWORK  [conn325] received client metadata from 172.20.0.4:34820 conn325: { driver: { name: "mongo-java-driver", version: "3.6.0" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.11.6.el7.x86_64" }, platform: "Java/Oracle Corporation/11.0.4+11-LTS" }
mongo_1          | 2019-12-16T15:51:30.809+0000 I  ACCESS   [conn325] SASL SCRAM-SHA-1 authentication failed for root on liran from client 172.20.0.4:34820 ; UserNotFound: Could not find user "root" for db "liran"
mongo_1          | 2019-12-16T15:51:30.810+0000 I  NETWORK  [conn325] end connection 172.20.0.4:34820 (4 connections now open)
mongo_1          | 2019-12-16T15:51:40.811+0000 I  NETWORK  [listener] connection accepted from 172.20.0.4:34822 #326 (5 connections now open)
mongo_1          | 2019-12-16T15:51:40.812+0000 I  NETWORK  [conn326] received client metadata from 172.20.0.4:34822 conn326: { driver: { name: "mongo-java-driver", version: "3.6.0" }, os: { type: "Linux", name: "Linux", architecture: "amd64", version: "3.10.0-862.11.6.el7.x86_64" }, platform: "Java/Oracle Corporation/11.0.4+11-LTS" }
mongo_1          | 2019-12-16T15:51:40.814+0000 I  ACCESS   [conn326] SASL SCRAM-SHA-1 authentication failed for root on liran from client 172.20.0.4:34822 ; UserNotFound: Could not find user "root" for db "liran"

更改 mongodb.credentials=root:example@local 只会在 local 而不是 liran 出现相同的错误。

最后一件事,当我运行 mongo 容器时,我使用了以下 stack.yml:

    [root@localhost tmp]# cat stack.yml 

# Use root/example as user/password credentials
version: '3.1'

services:

  mongo:
    image: mongo
    restart: always
    ports:
      - 27017:27017
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: example
      net.tls.mode: disabled

  mongo-express:
    image: mongo-express
    restart: always
    ports:
      - 8081:8081
    environment:
      ME_CONFIG_MONGODB_ADMINUSERNAME: root
      ME_CONFIG_MONGODB_ADMINPASSWORD: example

正在使用的码头工人: mongo, presto

【问题讨论】:

    标签: sql mongodb containers presto


    【解决方案1】:

    我们需要将“身份验证数据库”放在@ 之后,如下所示。 docker 镜像中的默认名称为“admin”。

    connector.name=mongodb
    mongodb.seeds=mongo:27017
    mongodb.credentials=root:example@admin
    

    【讨论】:

    • 谢谢。现在“liran”集合在 admin db 下,但我收到此错误:presto> SELECT COUNT(*) FROM mongodb.admin.liran;查询 20191218_155830_00007_us7wh 失败:com.mongodb.MongoTimeoutException:等待连接时在 30000 毫秒后超时。集群状态的客户端视图是 {type=UNKNOWN, servers=[{address=25a1ed6ef906:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: 异常发送消息},由 {javax.net. ssl.SSLHandshakeException: Remote host terminate the handshake},由{java.io.EOFException: SSL peer shut down wrongly}}]
    • 与 mongo 相同,而不是容器名称:presto> SELECT COUNT() FROM mongodb.admin.liran;查询 20191218_161403_00003_fh2vp 失败:com.mongodb.MongoTimeoutException:等待连接时 30000 毫秒后超时。集群状态的客户端视图是 {type=UNKNOWN, servers=[{address=**mongo**:27017, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketWriteException: 异常发送消息} ,由{javax.net.ssl.SSLHandshakeException:远程主机终止握手},由{java.io.EOFException:SSL对等关闭不正确}}]
    • 能否确认 Presto 和 MongoDB 之间的网络连接?另外,我会邀请您加入我们的社区 Slack (prestosql.io/slack.html),以便我们进行互动交流。
    • 当然。两种方法都有效。 1) [root@c51178e29a3e /]# ping -a mongo PING mongo (172.21.0.3) 56(84) 字节数据。来自 docker_mongo_1.docker_default (172.21.0.3) 的 64 个字节:icmp_seq=1 ttl=64 time=0.125 ms 2)[root@c51178e29a3e /]# ping -a 25a1ed6ef906 PING 25a1ed6ef906 (172.21.0.3) 56(84) 个字节的数据。来自 docker_mongo_1.docker_default (172.21.0.3) 的 64 个字节:icmp_seq=1 ttl=64 time=0.125 ms 两个容器在同一个 docker 网络下(“名称”:“docker_default”)。它看起来更像是一个 TLS 问题。我将加入 Slack。
    【解决方案2】:

    有两个问题:

    1. mongo 容器设置为可以访问 admin db,而我是 试图访问本地数据库。 如 mongo 的 docker hub 所示:

      MONGO_INITDB_ROOT_USERNAME, MONGO_INITDB_ROOT_PASSWORD 这些变量结合使用,创建一个新用户并设置 用户的密码。此用户是在 admin 身份验证中创建的 数据库并赋予 root 角色,这是一个“超级用户”角色。

    2. Presto 配置文件 (/usr/lib/presto/default/etc/catalog/mongodb.properties) 设置为 在 mongo 的容器没有的情况下使用 TLS。 更改属性 mongodb.ssl.enabled=false mongo 容器中的错误给出了线索​​:

      mongo_1 | 2019-12-19T12:51:45.607+0000 I NETWORK [conn2254] 接收来自客户端的请求时出错:SSLHandshakeFailed:收到 SSL 握手,但 服务器在没有 SSL 支持的情况下启动。结束来自 172.21.0.4:47702 的连接(连接 ID:2254)

    终于
    presto> SELECT COUNT(*) FROM mongodb.admin.liran;

    _col0

    29353 (1 行)

    查询 20191219_125214_00000_6983x,已完成,1 个节点 拆分:总共 18 个,完成 18 个 (100.00%) 0:02 [29.4K 行,681B] [17.8K 行/s,411B/s]

    【讨论】:

      猜你喜欢
      • 2019-01-12
      • 2017-10-13
      • 2021-02-02
      • 2016-06-13
      • 2019-02-16
      • 2022-11-11
      • 2017-11-10
      • 2019-08-27
      • 1970-01-01
      相关资源
      最近更新 更多