Jupyter 服务器上的 Python 读/写访问问题答案

【问题标题】：Read/Write access issue Python on Jupyter ServerJupyter 服务器上的 Python 读/写访问问题
【发布时间】：2018-05-14 02:15:10
【问题描述】：

我正在远程处理 Jupyter 笔记本服务器，当我使用以下方式创建文件时：

file = open("test.txt","w") file.write("test") file.close()

一切都按预期工作，文件 test.txt 被写入工作目录。尝试使用 Pandas to_hfs 命令时出现了我的问题：

data.to_hdf('raw_data.h5','raw_data_santodomingo',mode='w',format='f',data_columns=True)

我收到以下错误：

Opening raw_data.h5 in read-only mode
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in open(self, mode, **kwargs)
    586         try:
--> 587             self._handle = tables.open_file(self._path, self._mode, **kwargs)
    588         except (IOError) as e:  # pragma: no cover

/opt/conda/lib/python3.6/site-packages/tables/file.py in open_file(filename, mode, title, root_uep, filters, **kwargs)
    319     # Finally, create the File instance, and return it
--> 320     return File(filename, mode, title, root_uep, filters, **kwargs)
    321 

/opt/conda/lib/python3.6/site-packages/tables/file.py in __init__(self, filename, mode, title, root_uep, filters, **kwargs)
    783         # Now, it is time to initialize the File extension
--> 784         self._g_new(filename, mode, **params)
    785 

tables/hdf5extension.pyx in tables.hdf5extension.File._g_new()

/opt/conda/lib/python3.6/site-packages/tables/utils.py in check_file_access(filename, mode)
    178                 raise IOError("directory ``%s`` exists but it can not be "
--> 179                               "written" % (parentname,))
    180     elif mode == 'a':

OSError: directory ``.`` exists but it can not be written

During handling of the above exception, another exception occurred:

OSError                                   Traceback (most recent call last)
<ipython-input-182-479f2e98ea81> in <module>()
----> 1 pre_clean_data.to_hdf('raw_data.h5','raw_data_santodomingo',mode='w',format='f',data_columns=True)

/opt/conda/lib/python3.6/site-packages/pandas/core/generic.py in to_hdf(self, path_or_buf, key, **kwargs)
   1136 
   1137         from pandas.io import pytables
-> 1138         return pytables.to_hdf(path_or_buf, key, self, **kwargs)
   1139 
   1140     def to_msgpack(self, path_or_buf=None, encoding='utf-8', **kwargs):

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in to_hdf(path_or_buf, key, value, mode, complevel, complib, append, **kwargs)
    267     if isinstance(path_or_buf, string_types):
    268         with HDFStore(path_or_buf, mode=mode, complevel=complevel,
--> 269                       complib=complib) as store:
    270             f(store)
    271     else:

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in __init__(self, path, mode, complevel, complib, fletcher32, **kwargs)
    446         self._fletcher32 = fletcher32
    447         self._filters = None
--> 448         self.open(mode=mode, **kwargs)
    449 
    450     @property

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in open(self, mode, **kwargs)
    589             if 'can not be written' in str(e):
    590                 print('Opening %s in read-only mode' % self._path)
--> 591                 self._handle = tables.open_file(self._path, 'r', **kwargs)
    592             else:
    593                 raise

/opt/conda/lib/python3.6/site-packages/tables/file.py in open_file(filename, mode, title, root_uep, filters, **kwargs)
    318 
    319     # Finally, create the File instance, and return it
--> 320     return File(filename, mode, title, root_uep, filters, **kwargs)
    321 
    322 

/opt/conda/lib/python3.6/site-packages/tables/file.py in __init__(self, filename, mode, title, root_uep, filters, **kwargs)
    782 
    783         # Now, it is time to initialize the File extension
--> 784         self._g_new(filename, mode, **params)
    785 
    786         # Check filters and set PyTables format version for new files.

tables/hdf5extension.pyx in tables.hdf5extension.File._g_new()

/opt/conda/lib/python3.6/site-packages/tables/utils.py in check_file_access(filename, mode)
    154         # The file should be readable.
    155         if not os.access(filename, os.F_OK):
--> 156             raise IOError("``%s`` does not exist" % (filename,))
    157         if not os.path.isfile(filename):
    158             raise IOError("``%s`` is not a regular file" % (filename,))

OSError: ``raw_data.h5`` does not exist

这些行看起来很中肯，让我觉得写权限是个问题：

/opt/conda/lib/python3.6/site-packages/tables/utils.py in check_file_access(filename, mode)
    154         # The file should be readable.
    155         if not os.access(filename, os.F_OK):
--> 156             raise IOError("``%s`` does not exist" % (filename,))
    157         if not os.path.isfile(filename):
    158             raise IOError("``%s`` is not a regular file" % (filename,))

和

/opt/conda/lib/python3.6/site-packages/tables/utils.py in check_file_access(filename, mode)
    154         # The file should be readable.
    155         if not os.access(filename, os.F_OK):
--> 156             raise IOError("``%s`` does not exist" % (filename,))
    157         if not os.path.isfile(filename):
    158             raise IOError("``%s`` is not a regular file" % (filename,))

OSError: ``raw_data.h5`` does not exist

但这会让我感到困惑，因为我可以在上面提到的工作目录中编写文本文件。感谢所有和任何帮助的尝试。

编辑：如果我使用完整路径：/home/joyvan/work/raw_data.h5'，我会得到不同的错误读数。

data.to_hdf('/home/joyvan/work/raw_data.h5','raw_data_santodomingo',mode='w',format='f',data_columns=True)

生产

OSError                                   Traceback (most recent call last)
<ipython-input-185-de493145e6a7> in <module>()
----> 1 pre_clean_data.to_hdf('/home/joyvan/work/raw_data.h5','raw_data_santodomingo',mode='w',format='f',data_columns=True)

/opt/conda/lib/python3.6/site-packages/pandas/core/generic.py in to_hdf(self, path_or_buf, key, **kwargs)
   1136 
   1137         from pandas.io import pytables
-> 1138         return pytables.to_hdf(path_or_buf, key, self, **kwargs)
   1139 
   1140     def to_msgpack(self, path_or_buf=None, encoding='utf-8', **kwargs):

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in to_hdf(path_or_buf, key, value, mode, complevel, complib, append, **kwargs)
    267     if isinstance(path_or_buf, string_types):
    268         with HDFStore(path_or_buf, mode=mode, complevel=complevel,
--> 269                       complib=complib) as store:
    270             f(store)
    271     else:

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in __init__(self, path, mode, complevel, complib, fletcher32, **kwargs)
    446         self._fletcher32 = fletcher32
    447         self._filters = None
--> 448         self.open(mode=mode, **kwargs)
    449 
    450     @property

/opt/conda/lib/python3.6/site-packages/pandas/io/pytables.py in open(self, mode, **kwargs)
    585 
    586         try:
--> 587             self._handle = tables.open_file(self._path, self._mode, **kwargs)
    588         except (IOError) as e:  # pragma: no cover
    589             if 'can not be written' in str(e):

/opt/conda/lib/python3.6/site-packages/tables/file.py in open_file(filename, mode, title, root_uep, filters, **kwargs)
    318 
    319     # Finally, create the File instance, and return it
--> 320     return File(filename, mode, title, root_uep, filters, **kwargs)
    321 
    322 

/opt/conda/lib/python3.6/site-packages/tables/file.py in __init__(self, filename, mode, title, root_uep, filters, **kwargs)
    782 
    783         # Now, it is time to initialize the File extension
--> 784         self._g_new(filename, mode, **params)
    785 
    786         # Check filters and set PyTables format version for new files.

tables/hdf5extension.pyx in tables.hdf5extension.File._g_new()

/opt/conda/lib/python3.6/site-packages/tables/utils.py in check_file_access(filename, mode)
    172                 parentname = '.'
    173             if not os.access(parentname, os.F_OK):
--> 174                 raise IOError("``%s`` does not exist" % (parentname,))
    175             if not os.path.isdir(parentname):
    176                 raise IOError("``%s`` is not a directory" % (parentname,))

OSError: ``/home/joyvan/work`` does not exist

【问题讨论】：

我认为当前目录是“。”在这两种情况下是不同的。使用绝对路径进行测试（例如/home/user/raw_data.h5 而不是raw_data.h5）
我已经尝试使用完整路径，如上面的编辑所示，我得到一个不同但类似的错误。
文件夹 /home/joyvan/work 是否存在？也许只有 /home/joyvan 存在？
是的 /home/joyvan/work 存在并且是我正在工作的目录，如果我创建一个文本文件，它会按预期工作。
我已经解决了这个问题，只使用要保存的数据文件的名称：'raw_data.h5'，例如，不写入目录，将其保存在当前目录中。感谢 Andrei 的帮助

标签： python pandas file-permissions jupyter

【解决方案1】：

我遇到了类似的问题。
事实证明，作为当前用户，我正在运行的文件没有足够的写入权限。
我用 root 用户运行了相同的脚本，它工作正常。

注意：这已经晚了，不是对 OP 问题的回答，但对我来说类似的情况和编写对我有用的解决方案。

【讨论】：