将 Phoenix 与 Cloudera Hbase 一起使用（从 repo 安装）

Question

我可以让 Phoenix 在独立的 Apache Hbase

上工作

（注意，所有这些都是针对 RHEL6.5 上的 Hbase 1.0.0）

对于 Hbase 的 Cloudera 风格，但是如果没有抛出异常，我永远不会让它工作。 （甚至尝试将 RHEL7 最小化为 en OS）

Hbase 1.0 的 Phoenix 4.4 也会发生同样的情况。

hbase(main):001:0> version
1.0.0-cdh5.4.4, rUnknown, Mon Jul  6 16:59:55 PDT 2015

堆栈跟踪：

    [ec2-user@ip-172-31-60-109 phoenix-4.5.0-HBase-1.0-bin]$ bin/sqlline.py localhost:2181:/hbase
Setting property: [isolation, TRANSACTION_READ_COMMITTED]
issuing: !connect jdbc:phoenix:localhost:2181:/hbase none none org.apache.phoenix.jdbc.PhoenixDriver
Connecting to jdbc:phoenix:localhost:2181:/hbase
15/08/06 03:10:25 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/08/06 03:10:26 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-phoenix.properties,hadoop-metrics2.properties
15/08/06 03:10:27 WARN ipc.CoprocessorRpcChannel: Call failed on IOException
org.apache.hadoop.hbase.DoNotRetryIOException: org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:84)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1269)
    at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:11619)
    at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7054)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1746)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1728)
    at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:31447)
    at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2035)
    at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
    at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
    at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.buildDeletedTable(MetaDataEndpointImpl.java:966)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.loadTable(MetaDataEndpointImpl.java:1042)
    at org.apache.phoenix.coprocessor.MetaDataEndpointImpl.createTable(MetaDataEndpointImpl.java:1216)
    ... 10 more

    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
    at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
    at org.apache.hadoop.hbase.protobuf.ProtobufUtil.execService(ProtobufUtil.java:1609)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:92)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel$1.call(RegionCoprocessorRpcChannel.java:89)
    at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
    at org.apache.hadoop.hbase.ipc.RegionCoprocessorRpcChannel.callExecService(RegionCoprocessorRpcChannel.java:95)
    at org.apache.hadoop.hbase.ipc.CoprocessorRpcChannel.callMethod(CoprocessorRpcChannel.java:56)
    at org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService$Stub.createTable(MetaDataProtos.java:11799)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$6.call(ConnectionQueryServicesImpl.java:1273)
    at org.apache.phoenix.query.ConnectionQueryServicesImpl$6.call(ConnectionQueryServicesImpl.java:1261)
    at org.apache.hadoop.hbase.client.HTable$16.call(HTable.java:1737)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.DoNotRetryIOException): org.apache.hadoop.hbase.DoNotRetryIOException: SYSTEM.CATALOG: org.apache.hadoop.hbase.client.Scan.setRaw(Z)Lorg/apache/hadoop/hbase/client/Scan;
    at ... 

Answer 1

Cloudera 没有正式支持 Apache phoenix，它还在 cloudera Labs 中，所以您在 cloudera 存储库中找不到任何 Cloudera Pheonix tar.gz 文件，在 Cloudera 存储库中唯一可以找到 Phoenix 的地方是包裹 @987654321 @, 但是parcel只有通过cloudera manager安装才能使用，cloudera Phoenix的最新可用版本是4.3.0。

如果您想在 Cloudera Hadoop 发行版上执行 Phoenix 4.4 或 4.5 版本，您需要使用 CDH 依赖 jar 重新构建 phoenix 库。你不能简单地使用 apache Phoenix tar.gz

这里是步骤。

最近我发现 Andrew Purtell 在使 Phoenix 与 CDH 版本兼容方面做了大量工作。以下链接 github 页面中提供了相同的内容。从下面的 github 链接下载适当的分支。这样可以节省您的时间。

https://github.com/chiastic-security/phoenix-for-cloudera/branches

• 从 Apache repository 下载 Apache phoenix 4.5 源代码。 （如果你是从上面的github页面下载的，跳过这一步）

使用CDH依赖jar重新构建源代码——更新pom.xml和2个源文件如下（我的CDH版本是5.4.2）

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/pom.xml  phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/pom.xml
28c28
< <!--    <module>phoenix-pig</module> -->
---
>     <module>phoenix-pig</module>
37a38,41
>       <id>apache release</id>
>       <url>https://repository.apache.org/content/repositories/releases/</url>
>     </repository>
>     <repository>
42,43c46,50
<       <id>cloudera</id>
<       <url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
---
>       <id>apache snapshot</id>
>       <url>https://repository.apache.org/content/repositories/snapshots/</url>
>       <snapshots>
>         <enabled>true</enabled>
>       </snapshots>
45d51
<
54d59
<
77,81c82,83
<     <hbase.version>1.0.0-cdh5.4.2</hbase.version>
<     <hadoop-two.version>2.6.0-cdh5.4.2</hadoop-two.version>
/<     <hadoop.version>2.6.0-cdh5.4.2</hadoop.version>
<     <pig.version>0.12.0</pig.version>
<     <flume.version>1.5.0-cdh5.4.2</flume.version>
---
>     <hbase.version>1.0.1</hbase.version>
>     <hadoop-two.version>2.5.1</hadoop-two.version>
84a87,88
>     <hadoop.version>2.5.1</hadoop.version>
>     <pig.version>0.13.0</pig.version>
97a102
>     <flume.version>1.4.0</flume.version>
449,450c454
<
<   <dependency>
---
>       <dependency>
454c458
<       </dependency>
---
>       </dependency>

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java  phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/LocalIndexMerger.java
84c84
<                     rss.getServerName(), metaEntries,1);
---
>                     rss.getServerName(), metaEntries);

[h4ck3r@host1 phoenix]$ diff phoenix-4.5_Updated/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java phoenix-4.5_Orig/phoenix-4.5.0-HBase-1.0-src/phoenix-core/src/main/java/org/apache/hadoop/hbase/regionserver/IndexSplitTransaction.java
291c291
<                 daughterRegions.getSecond().getRegionInfo(), server.getServerName(),1);
---
>                 daughterRegions.getSecond().getRegionInfo(), server.getServerName());
978c978
< }
---
> }
\ No newline at end of file

• 上述构建将在每个子组件的目标目录下创建新的罐子。
• 从 Apache repository 下载 Apache phoenix 4.5 二进制文件

• 解压phoenix-4.5.0-HBase-1.0-bin.tar.gz 并用新的Jars 替换下面的Phoenix Jars

  • phoenix-4.5.0-HBase-1.0-client.jar
  • phoenix-4.5.0-HBase-1.0-server-without-antlr.jar
  • phoenix-4.5.0-HBase-1.0-client-minimal.jar
  • phoenix-assembly-4.5.0-HBase-1.0-tests.jar
  • phoenix-4.5.0-HBase-1.0-client-without-hbase.jar
  • phoenix-core-4.5.0-HBase-1.0.jar
  • phoenix-4.5.0-HBase-1.0-server.jar

• 在 hbase lib 位置替换 phoenix-4.5.0-HBase-1.0-server.jar 和 phoenix-core-4.5.0-HBase-1.0.jar 并重新启动 hbase。 （在 4.7 中只有 phoenix-4.7.0-cdh5.X.1-server.jar 被复制到 hbase lib）

• 从新更新的目录执行 phoenix 命令。

由于一些依赖问题 phoenix-pig 没有被处理，这只是一个解决方法。

Answer 2

New Apache Phoenix 4.5.2 Package from Cloudera Labs