在实现 spark Mllib 时警告 Scala 应用程序

Question

/* 这是使用 Scala IDE 运行 spark 应用程序时收到的警告 我需要你的帮助来解决这个警告 这里没有使用 Maven 项目 */

Exception in thread "main" java.lang.IllegalArgumentException: 
  java.net.URISyntaxException: Illegal character in scheme name at index 0: 
    ‪C:/Datasets/creditcard.csv
    at org.apache.hadoop.fs.Path.initialize(Path.java:205)
    at org.apache.hadoop.fs.Path.<init>(Path.java:171)
    at org.apache.hadoop.util.StringUtils.stringToPath(StringUtils.java:245)
    at org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:411)
    at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$30.apply(SparkContext.scala:1014)
    at org.apache.spark.SparkContext$$anonfun$hadoopFile$1$$anonfun$30.apply(SparkContext.scala:1014)
    at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:179)
    at org.apache.spark.rdd.HadoopRDD$$anonfun$getJobConf$6.apply(HadoopRDD.scala:179)
    at scala.Option.foreach(Option.scala:257)
    at org.apache.spark.rdd.HadoopRDD.getJobConf(HadoopRDD.scala:179)
    at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:198)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
    at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:252)
    at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:250)
    at scala.Option.getOrElse(Option.scala:121)
    at org.apache.spark.rdd.RDD.partitions(RDD.scala:250)
    at org.apache.spark.SparkContext.runJob(SparkContext.scala:1958)
    at org.apache.spark.rdd.RDD.count(RDD.scala:1157)
    at data_Classifier_package.DecisionTreeClassifier$.main(DecisionTreeClassifier.scala:46)
    at data_Classifier_package.DecisionTreeClassifier.main(DecisionTreeClassifier.scala)
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 0: ‪C:/Datasets/creditcard.csv
    at java.net.URI$Parser.fail(Unknown Source)
    at java.net.URI$Parser.checkChars(Unknown Source)
    at java.net.URI$Parser.checkChar(Unknown Source)
    at java.net.URI$Parser.parse(Unknown Source)
    at java.net.URI.<init>(Unknown Source)
    at org.apache.hadoop.fs.Path.initialize(Path.java:202)
    ... 23 more

Answer 1

你的错误信息很清楚。

索引 0 处的方案名称中有非法字符： ‪C:/Datasets/creditcard.csv

https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml 列出了有效的 URI 方案。

您可能想改用file:///C:/Datasets/creditcard.csv。

File URIs in Windows