【发布时间】:2019-04-25 14:45:15
【问题描述】:
在将 xarray.DataArray 写入到_netcdf() 时,我得到了ValueError: Lock objects should only be shared between processes through inheritance。
一切正常,直到写入磁盘。但我找到了一种解决方法,即使用dask.config.set(scheduler='single-threaded')。
是否每个人都应该在写入磁盘之前使用
dask.config.set(scheduler='single-threaded')?我错过了什么吗?
我测试了两个调度器:
1)from dask.distributed import Client; client = Client()
2)import dask.multiprocessing; dask.config.set(scheduler=dask.multiprocessing.get)
python=2.7,xarray=0.10.9,回溯:
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataarray.py", line 1746, in to_netcdf
return dataset.to_netcdf(*args, **kwargs)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1254, in to_netcdf
compute=compute)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/backends/api.py", line 724, in to_netcdf
unlimited_dims=unlimited_dims, compute=compute)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1181, in dump_to_store
store.sync(compute=compute)
...
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/synchronize.py", line 95, in __getstate__
assert_spawning(self)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/forking.py", line 52, in assert_spawning
' through inheritance' % type(self).__name__
【问题讨论】:
-
我认为这里提到的问题已经在 Xarray 中得到修复。我建议更新到更新版本的 xarray (也许是 dask)。如果问题依旧,请在github上开一个issue:github.com/pydata/xarray/issues
标签: pickle python-multiprocessing dask python-xarray