使用 MapDriver 测试 Hbase 连接

Question

我正在使用 MapDriver 来测试 Mapper 程序。在 Mapper - setup() 方法中，我正在实例化一个 Hbase 表，它正在尝试连接到 zookeeper。
在我的测试类中，我无法连接到 zookeeper，并且由于连接超时异常而一直失败。

SampleMapper.java
@Override
public void setup(Context context) throws IOException, InterruptedException {
    Configuration config = context.getConfiguration();
    table = new HTable(config, config.get("tableName"));
    Scan tableScan = new Scan();
    tableScan.addColumn(COLUMN_FAMILY_STUDENT, COLUMN_ID);
    tableScan.setCaching(NUMBER_OF_ROWS_TO_CACHE);
    tableScan.setCacheBlocks(false);
    super.setup(context);
}

sampleMapperTest.java
@Before
public void setUp() {
    mapDriver = MapDriver.newMapDriver(new SampleMapper());
    Configuration conf = mapDriver.getConfiguration();
    conf.set("tableName", "testTable");
    conf.setStrings("io.serializations",
            conf.get("io.serializations"), MutationSerialization.class.getName(), ResultSerialization.class.getName());

}


 [org.apache.zookeeper.ZooKeeper] Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection-0x625e134e, quorum=localhost:2181, baseZNode=/hbase
  main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
  main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
        04 Mar 2018 20:31 47,648 main-SendThread(0:0:0:0:0:0:0:1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 0:0:0:0:0:0:0:1/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
        04 Mar 2018 20:31 47,702 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
         04 Mar 2018 20:31 47,702 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 1000ms before retry #0...
        04 Mar 2018 20:31 48,651 main-SendThread(0:0:0:0:0:0:0:1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
        java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
            04 Mar 2018 20:31 48,752 main WARN  [org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper] Possibly transient ZooKeeper, quorum=localhost:2181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
       04 Mar 2018 20:31 48,752 main INFO  [org.apache.hadoop.hbase.util.RetryCounter] Sleeping 2000ms before retry #1...
       04 Mar 2018 20:31 49,753 main-SendThread(127.0.0.1:2181) INFO  [org.apache.zookeeper.ClientCnxn] Opening socket connection to server 127.0.0.1/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
       04 Mar 2018 20:31 50,755 main-SendThread(127.0.0.1:2181) WARN  [org.apache.zookeeper.ClientCnxn] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
    java.net.ConnectException: Connection refused: no further information

我不确定如何在这里模拟 zooker。我怎样才能获得与 Zookeeper 的连接？

有人可以帮忙吗？

Answer 1

main-SendThread(127.0.0.1:2181) 信息 [org.apache.zookeeper.ClientCnxn] 打开到服务器的套接字连接 127.0.0.1/127.0.0.1:2181。不会尝试使用 SASL 进行身份验证（未知错误） main-SendThread(127.0.0.1:2181) WARN [org.apache.zookeeper.ClientCnxn] 会话 0x0 用于服务器 null， 意外错误，关闭套接字连接并尝试重新连接 java.net.ConnectException：连接被拒绝：没有更多信息

您的错误表明您正在连接到 localhost (127.0.0.1)。如果您使用的是远程主机，请设置以下内容：

config.set("hbase.zookeeper.quorum", "remote ip");
config.set("hbase.zookeeper.property.clientPort", "2181"); //your port
config.set("hbase.master", "hbase_master_ip:16000"); //your port