【问题标题】:HORTONWORKS - Hbase/Phoenix - WALEditCodec - missingHORTONWORKS - Hbase/Phoenix - WALEditCodec - 缺失
【发布时间】:2017-11-07 05:50:02
【问题描述】:

尝试在 Hbase 上运行 Phoenix 时收到以下错误:

例外 #1:

2017-11-07 12:40:12,620 WARN  [RS_LOG_REPLAY_OPS-XXX:16020-0] 
regionserver.SplitLogWorker: log splitting of 
WALs/XXX.XXX.XXX.XXX,16020,1507179047656-
splitting/XXX.XXX.XXX.XXX%2C16020%2C1507179047656.default.1507179049782 failed, returning error
java.io.IOException: Cannot get log reader
    at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:355)
    at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:267)
    at org.apache.hadoop.hbase.wal.WALSplitter.getReader(WALSplitter.java:839)
    at org.apache.hadoop.hbase.wal.WALSplitter.getReader(WALSplitter.java:763)
    at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:297)
    at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:235)
    at org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
    at org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
    at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
:$
    at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:355)
    at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:267)
    at org.apache.hadoop.hbase.wal.WALSplitter.getReader(WALSplitter.java:839)
    at org.apache.hadoop.hbase.wal.WALSplitter.getReader(WALSplitter.java:763)
    at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:297)
    at org.apache.hadoop.hbase.wal.WALSplitter.splitLogFile(WALSplitter.java:235)
    at org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:104)
    at org.apache.hadoop.hbase.regionserver.handler.WALSplitterHandler.process(WALSplitterHandler.java:72)
    at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.UnsupportedOperationException: Unable to find  org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
    at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:36)
    at org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.create(WALCellCodec.java:103)
    at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.getCodec(ProtobufLogReader.java:297)
    at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:307)
    at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:82)
    at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.init(ProtobufLogReader.java:164)
    at org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:303)
    ... 11 more
Caused by: java.lang.ClassNotFoundException:  org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:32)
    ... 17 more

应用补丁#1: 我已通过 Ambari Web UI 为 Hortonworks 文档指定的高级 Hbase 配置应用了以下设置: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_command-line-upgrade/content/configure-phoenix-25.html

例外 #2

FATAL [RS_LOG_REPLAY_OPS-XXX:16020-1] conf.Configuration: error parsing conf core-site.xml
java.io.FileNotFoundException: /etc/hadoop/2.6.1.0-129/0/core-site.xml (Too many open files)

应用补丁#2 我检查了每个包含 Hbase 区域服务器的服务器上的每个“core-site.xml”文件,并确保它以 </configuration> 结尾。以及指定目录'/etc/hadoop/2.6.1.0-129/0/core-site.xml'中的core-site.xml文件

无法找到有关此问题的任何其他信息。

【问题讨论】:

    标签: hadoop hbase hortonworks-data-platform phoenix ambari


    【解决方案1】:

    我进入 HDFS 并使用以下命令删除了所有 WAL 拆分日志:

    hdfs dfs -rm -r /apps/hbase/data/WALs/*splitting*
    

    这解决了异常 #1。请记住,根据我所阅读的内容,这会导致数据丢失。

    对于异常 #2,我返回并检查了每个服务器的打开文件限制 (ulimit -n),并在适用的情况下根据 Hortonworks 文档进行了更新: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.6.2/bk_security/content/kerb-config-limits.html

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2016-04-08
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多