【问题标题】:RuntimeError: Lock objects should only be shared between processes through inheritanceRuntimeError:锁对象只能通过继承在进程之间共享
【发布时间】:2019-04-25 14:45:15
【问题描述】:

在将 xarray.DataArray 写入到_netcdf() 时,我得到了ValueError: Lock objects should only be shared between processes through inheritance。 一切正常,直到写入磁盘。但我找到了一种解决方法,即使用dask.config.set(scheduler='single-threaded')

  • 是否每个人都应该在写入磁盘之前使用dask.config.set(scheduler='single-threaded')

  • 我错过了什么吗?

我测试了两个调度器:

1)from dask.distributed import Client; client = Client()

2)import dask.multiprocessing; dask.config.set(scheduler=dask.multiprocessing.get)

python=2.7,xarray=0.10.9,回溯:


  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataarray.py", line 1746, in to_netcdf
    return dataset.to_netcdf(*args, **kwargs)
  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1254, in to_netcdf
    compute=compute)
  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/backends/api.py", line 724, in to_netcdf
    unlimited_dims=unlimited_dims, compute=compute)
  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1181, in dump_to_store
    store.sync(compute=compute)
...
  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/synchronize.py", line 95, in __getstate__
    assert_spawning(self)
  File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/forking.py", line 52, in assert_spawning
    ' through inheritance' % type(self).__name__

【问题讨论】:

  • 我认为这里提到的问题已经在 Xarray 中得到修复。我建议更新到更新版本的 xarray (也许是 dask)。如果问题依旧,请在github上开一个issue:github.com/pydata/xarray/issues

标签: pickle python-multiprocessing dask python-xarray


【解决方案1】:

正如@jhamman 在 cmets 中提到的那样。这可能已在较新版本的 Xarray 中得到修复。

【讨论】:

    猜你喜欢
    • 2021-12-22
    • 2017-07-27
    • 2016-04-23
    • 2012-11-27
    • 2014-10-22
    • 2012-07-22
    • 2021-10-21
    • 2016-02-05
    • 1970-01-01
    相关资源
    最近更新 更多