【问题标题】:Building Apache Spark 2.1.0 from source fails从源代码构建 Apache Spark 2.1.0 失败
【发布时间】:2017-04-02 17:03:21
【问题描述】:

我正在尝试构建 Apache Spark 2.1.0 源代码,但是下面的这些错误让我感到困惑......

Hadoop 2.8.0 已安装并且正在运行 Scala 2.12.1 是在执行 Spark 安装之前安装的(这似乎会自动安装 Scala 2.11.8 ?!?)

我的构建线是:

build/mvn -Pyarn -Phadoop-2.7 -Dhadoop.version=2.7.0 -DskipTests clean package

有人知道我为什么会得到:

user@server:/usr/local/share/spark/spark-2.1.0$ sudo /usr/local/share/spark/spark-2.1.0/build/mvn -Pyarn -Phadoop-2.7 -Dhadoop.version=2.7.0 -DskipTests clean package
[sudo] password for user:
exec: curl --progress-bar -L https://downloads.typesafe.com/zinc/0.3.9/zinc-0.3.9.tgz
######################################################################## 100.0%
exec: curl --progress-bar -L https://downloads.typesafe.com/scala/2.11.8/scala-2.11.8.tgz
######################################################################## 100.0%
exec: curl --progress-bar -L https://www.apache.org/dyn/closer.lua?action=download&filename=/maven/maven-3/3.3.9/binaries/apache-maven-3.3.9-bin.tar.gz
######################################################################## 100.0%
Using `mvn` from path: /usr/local/share/spark/spark-2.1.0/build/apache-maven-3.3.9/bin/mvn
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=512M; support was removed in 8.0
[INFO] Scanning for projects...
Downloading: https://repo1.maven.org/maven2/org/apache/apache/14/apache-14.pom
[ERROR] [ERROR] Some problems were encountered while processing the POMs:
[FATAL] Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.1.0: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): repo1.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11
 @
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]
[ERROR]   The project org.apache.spark:spark-parent_2.11:2.1.0 (/usr/local/share/spark/spark-2.1.0/pom.xml) has 1 error
[ERROR]     Non-resolvable parent POM for org.apache.spark:spark-parent_2.11:2.1.0: Could not transfer artifact org.apache:apache:pom:14 from/to central (https://repo1.maven.org/maven2): repo1.maven.org: Name or service not known and 'parent.relativePath' points at wrong local POM @ line 22, column 11: Unknown host repo1.maven.org: Name or service not known -> [Help 2]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException

我手动测试了下载(查看是否导致错误),下载没有问题:

https://repo1.maven.org/maven2/org/apache/apache/14/apache-14.pom

我还测试了访问下面的 URL,它也显示了内容:

https://repo1.maven.org/maven2

希望聪明人知道如何解决这个问题...

【问题讨论】:

    标签: scala hadoop apache-spark


    【解决方案1】:

    我发现了问题所在:

    我必须在目录中的 SETTINGS.XML 中配置我们的代理设置:

    /usr/local/share/spark/spark-2.1.0/build/apache-maven-3.3.9/conf
    

    编辑文件后,构建没有任何问题:)

    希望这可以帮助遇到同样问题的其他人...

    编辑:为了更加清楚,仅在 bash 中具有有效的代理配置不足以使 Maven 构建成功。我能够手动从 bash 下载所有文件,但 Maven 还需要 SETTINGS.XML 文件中的代理配置......

    【讨论】:

      猜你喜欢
      • 2017-04-21
      • 2017-03-02
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2021-03-17
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多