我已经尝试过 Azul JDK 8。
我只想说,虽然 Azul JDK 在 Apple M1 上原生运行并且速度非常快,但仍然存在问题。尤其是一些Java代码需要调用C++代码的时候。
例如,我是一名大数据开发人员。我开始在我的开发工作流程中使用 Azul JDK。但我注意到某些测试在切换后开始失败。例如,当测试写入Parquet/Avro 文件时,它会失败。我认为这是因为有一些用 C++ 为 Parquet/Avro 编写的原生东西,它们不是为 M1 编译的。
由于这个特定的原因,我不得不使用非 M1 JDK,它很慢。那里没有问题。
以下是我在使用 Azul 时遇到的错误示例,而我在使用非 M1 JDK 时没有遇到:
- convert Base64 JSON back to rpo Avro *** FAILED ***
org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 10.0 failed 1 times, most recent failure: Lost task 0.0 in stage 10.0 (TID 14, localhost, executor driver): org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:331)
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:171)
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
at org.apache.avro.file.SnappyCodec.compress(SnappyCodec.java:43)
at org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:358)
at org.apache.avro.file.DataFileWriter.writeBlock(DataFileWriter.java:382)
at org.apache.avro.file.DataFileWriter.sync(DataFileWriter.java:401)
at org.apache.avro.file.DataFileWriter.flush(DataFileWriter.java:410)
at org.apache.avro.file.DataFileWriter.close(DataFileWriter.java:433)
at org.apache.avro.mapred.AvroOutputFormat$1.close(AvroOutputFormat.java:170)
at org.apache.spark.internal.io.SparkHadoopWriter.close(SparkHadoopWriter.scala:101)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12$$anonfun$apply$5.apply$mcV$sp(PairRDDFunctions.scala:1145)
at org.apache.spark.util.Utils$.tryWithSafeFinallyAndFailureCallbacks(Utils.scala:1393)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1145)
at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1125)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
at org.apache.spark.scheduler.Task.run(Task.scala:108)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:335)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Driver stacktrace:
at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1499)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1487)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1486)
at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)
at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1486)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814)
at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:814)
at scala.Option.foreach(Option.scala:257)
at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:814)
...
Cause: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
at org.xerial.snappy.SnappyLoader.findNativeLibrary(SnappyLoader.java:331)
at org.xerial.snappy.SnappyLoader.loadNativeLibrary(SnappyLoader.java:171)
at org.xerial.snappy.SnappyLoader.load(SnappyLoader.java:152)
at org.xerial.snappy.Snappy.<clinit>(Snappy.java:47)
at org.apache.avro.file.SnappyCodec.compress(SnappyCodec.java:43)
at org.apache.avro.file.DataFileStream$DataBlock.compressUsing(DataFileStream.java:358)
at org.apache.avro.file.DataFileWriter.writeBlock(DataFileWriter.java:382)
at org.apache.avro.file.DataFileWriter.sync(DataFileWriter.java:401)
at org.apache.avro.file.DataFileWriter.flush(DataFileWriter.java:410)
at org.apache.avro.file.DataFileWriter.close(DataFileWriter.java:433)
如你所见,上面写着: Cause: org.xerial.snappy.SnappyError: [FAILED_TO_LOAD_NATIVE_LIBRARY] no native library is found for os.name=Mac and os.arch=aarch64
我用谷歌搜索了这个问题,他们说原生库是为 Spark 的更高版本编译的,很遗憾。
这让我非常沮丧,我现在想要一台 Windows 笔记本电脑,哈哈。在 M1 芯片上运行 Intel JDK 有时会很慢,我不希望这样。
小心!
更新:
他们发布了支持 M1 的新版本库,我在项目中更新了它们,一切正常,感谢上帝。有时这些“本机代码错误”会以不同的异常表现出来,这是额外的 P.I.T.A.我必须处理,而我在 Windows 笔记本电脑上的同事不需要处理它。错误有时可能有点不清楚,但如果您在错误日志中看到有关本机代码的内容,或者诸如“jna”或“jni”之类的字眼,那么这是 M1 芯片问题。