在 Docker 中将 Java Spring 应用程序连接到 ScyllaDB 集群时出错

Question

我在 docker 上成功运行了 ScyllaDB 集群，但是在尝试将我的 Spring Boot 应用程序连接到它时遇到了一些连接问题。

集群似乎很健康：

Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address     Load       Tokens       Owns    Host ID                               Rack
UN  172.19.0.3  205.31 KB  256          ?       c387bd38-cf37-4c70-a8a3-db586f6410c4  Rack1
UN  172.19.0.2  101.61 KB  256          ?       89dda41e-70c5-43a5-a4a7-ce126007a1e1  Rack1
UN  172.19.0.4  210.34 KB  256          ?       58dfe868-a7ac-47a5-b3e9-dae610ef0bc9  Rack1

CONTAINER ID   IMAGE                   COMMAND                  CREATED         STATUS         PORTS                                                            NAMES
b90d0357abee   scylladb/scylla:4.5.0   "/docker-entrypoint.…"   2 minutes ago   Up 2 minutes   22/tcp, 7000-7001/tcp, 9042/tcp, 9160/tcp, 9180/tcp, 10000/tcp   scylla-node2
19f19cb2e631   scylladb/scylla:4.5.0   "/docker-entrypoint.…"   2 minutes ago   Up 2 minutes   22/tcp, 7000-7001/tcp, 9042/tcp, 9160/tcp, 9180/tcp, 10000/tcp   scylla-node3
c98b0b1aa018   scylladb/scylla:4.5.0   "/docker-entrypoint.…"   2 minutes ago   Up 2 minutes   22/tcp, 7000-7001/tcp, 9042/tcp, 9160/tcp, 9180/tcp, 10000/tcp   scylla-node1

并且 CQL 可以正常工作：

docker exec -it scylla-node1 cqlsh
Connected to  at 172.19.0.2:9042.
[cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4]
Use HELP for help.
cqlsh>

我正在尝试使用 datastax 驱动程序（scylla-driver-core 版本 3.11.2.0）将我的 Java 应用程序连接到集群：

Cluster cluster = Cluster.builder().addContactPoints("172.19.0.3", "172.19.0.2", "172.19.0.4")
                .build();
System.out.println("Connected to cluster " + cluster.getMetadata().getClusterName());

但我收到以下异常：

2022-05-08 18:00:02.928  INFO 4760 --- [  restartedMain] com.datastax.driver.core.NettyUtil       : Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
2022-05-08 18:00:09.149  WARN 4760 --- [r1-nio-worker-0] com.datastax.driver.core.Connection      : Error creating netty channel to /172.19.0.4:9042

io.netty.channel.ConnectTimeoutException: connection timed out: /172.19.0.4:9042
    at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe$1.run(AbstractNioChannel.java:261) ~[netty-transport-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute$$$capture(AbstractEventExecutor.java:164) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:503) [netty-transport-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.75.Final.jar:4.1.75.Final]
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.75.Final.jar:4.1.75.Final]

Exception in thread "restartedMain" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /172.19.0.4:9042 (com.datastax.driver.core.exceptions.TransportException: [/172.19.0.4:9042] Cannot connect), /172.19.0.3:9042 (com.datastax.driver.core.exceptions.TransportException: [/172.19.0.3:9042] Cannot connect), /172.19.0.2:9042 (com.datastax.driver.core.exceptions.TransportException: [/172.19.0.2:9042] Cannot connect))
    at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:270)
    at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:109)
    at com.datastax.driver.core.Cluster$Manager.negotiateProtocolVersionAndConnect(Cluster.java:1921)
    at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1832)
    at com.datastax.driver.core.Cluster.getMetadata(Cluster.java:476)
    at com.dt.package.ScyllaConnection.connect_and_query(ScyllaConnection.java:15)
    at com.dt.package.MyApp.main(MyApp.java:12)
    ... 5 more

docker-compose 文件示例：

  scylla-node2:
    container_name: scylla-node2
    image: scylladb/scylla:4.5.0
    restart: always
    command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
    volumes:
      - "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
      - "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
    networks:
      web:

在尝试连接到 Scylla Cloud 集群（使用正确的 IP）时，同样的代码可以工作。我在这里遗漏了一些简单的东西吗？

Answer 1

问题可能是因为您尝试使用其 docker 内部 IP 地址连接到 ScyllaDB 节点。

如果您需要从内部 docker 网络外部访问您的节点，您需要正确地向 docker 主机公开适当的端口。

使用 docker compose，您可以使用 ports 字段导出容器端口。例如：

  scylla-node2:
    container_name: scylla-node2
    image: scylladb/scylla:4.5.0
    restart: always
    command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
    ports:
      - 9043:9042
    volumes:
      - "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
      - "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
    networks:
      web:

在示例中，我们将容器端口 9042 暴露给主机端口 9043。

您将需要与其余节点类似的配置：您为所有节点使用相同的主机，因此您需要为每个节点选择不同的端口。

  scylla-node1:
    ...
    ports:
      - 9042:9042
    ...
  scylla-node2:
    ...
    ports:
      - 9043:9042
    ...
  scylla-node3:
    ...
    ports:
      - 9044:9042
    ...

不同的节点将共享同一个主机，即运行 docker 容器的主机。要从 Spring 应用程序连接到它们，您需要以下内容：

Cluster cluster =
  Cluster.builder()
    .addContactPointsWithPorts(
      new InetSocketAddress("127.0.0.1", 9042),
      new InetSocketAddress("127.0.0.1", 9043),
      new InetSocketAddress("127.0.0.1", 9044)
    );

System.out.println("Connected to cluster " + cluster.getMetadata().getClusterName());

请注意在代码中使用方法addContactPointsWithPorts 而不是addContactPoints。

Answer 2

我也遇到了同样的问题，但是设置不同的端口仍然没有解决。