【问题标题】:Apache hadoop Installation on Windows 10在 Windows 10 上安装 Apache hadoop
【发布时间】:2017-05-27 08:33:10
【问题描述】:

在 windows 10 上设置没有 Cygwin 的单节点集群时,我遵循了具体的文档-Link for Hadoop installation in windows 10

我在使用 D:\hadoop-2.6.2.tar\hadoop-2.6.2\hadoop-2.6.2\sbin>start-dfs.cmd 启动 hdfs 时遇到以下错误

错误消息堆栈跟踪:

17/01/12 12:25:42 FATAL datanode.DataNode: Exception in secureMain java.lang.RuntimeException: Error while running command to get file permissions : ExitCodeException exitCode=-1073741515:
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
    at org.apache.hadoop.util.Shell.run(Shell.java:455)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:808)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:791)
    at org.apache.hadoop.fs.FileUtil.execCommand(FileUtil.java:1097)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:582)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:139)
    at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156)
    at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:2299)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2341)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2323)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2215)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2262)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2438)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2462)

    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.loadPermissionInfo(RawLocalFileSystem.java:620)
    at org.apache.hadoop.fs.RawLocalFileSystem$DeprecatedRawLocalFileStatus.getPermission(RawLocalFileSystem.java:557)
    at org.apache.hadoop.util.DiskChecker.mkdirsWithExistsAndPermissionCheck(DiskChecker.java:139)
    at org.apache.hadoop.util.DiskChecker.checkDir(DiskChecker.java:156)
    at org.apache.hadoop.hdfs.server.datanode.DataNode$DataNodeDiskChecker.checkDir(DataNode.java:2299)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.checkStorageLocations(DataNode.java:2341)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:2323)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:2215)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:2262)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:2438)
    at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:2462) 17/01/12 12:25:42 INFO util.ExitUtil: Exiting with status 1

还有这个关于启动namenode的错误信息:

17/01/12 12:25:43 FATAL namenode.NameNode: Failed to start namenode.
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Ljava/lang/String;I)Z
        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access0(Native Method)
        at org.apache.hadoop.io.nativeio.NativeIO$Windows.access(NativeIO.java:557)
        at org.apache.hadoop.fs.FileUtil.canWrite(FileUtil.java:996)
        at org.apache.hadoop.hdfs.server.common.Storage$StorageDirectory.analyzeStorage(Storage.java:490)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:309)
        at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:202)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:1022)
        at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:741)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:538)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:597)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:764)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:748)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1441)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1507)
17/01/12 12:25:43 INFO util.ExitUtil: Exiting with status 1

【问题讨论】:

  • 你通过上面的错误Error while running command to get file permissions了吗?

标签: apache hadoop windows-10 hadoop2


【解决方案1】:

[]问题分析]/data目录权限不够,NameNode无法启动。

【解决方案】

(1)在root下,将/data/目录的操作权限分配给hadoop用户;

(2) 清空/data目录文件;

(3)重新格式化NameNode,重启hadoop集群。

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2020-11-07
    • 2018-02-01
    • 1970-01-01
    • 1970-01-01
    • 2019-06-19
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多