Python版本不一致导致的,设置下环境变量即可。

Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run with different minor versions.Please check environment variables PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON are correctly set.

python --version 发现是2.6

echo $PYSPARK_PYTHON 值为空

echo $PYSPARK_DRIVER_PYTHON 值是ipython

which ipython发现路径是

/home/work/envs/python_2.7.13/bin/ipython

设置环境变量,问题解决:

export PYSPARK_PYTHON=/home/work/envs/python_2.7.13/bin/ipython

 

Exception: Python in worker has different version 2.6 than that in driver 2.7, PySpark cannot run

 

相关文章:

  • 2021-06-04
  • 2022-03-01
  • 2021-05-28
猜你喜欢
  • 2021-06-06
  • 2022-12-23
  • 2021-04-07
  • 2022-12-23
  • 2022-01-21
  • 2022-12-23
  • 2021-05-24
相关资源
相似解决方案