【问题标题】:shutil.rmtree doesn't work with Windows Libraryshutil.rmtree 不适用于 Windows 库
【发布时间】:2014-07-18 10:05:45
【问题描述】:

所以我正在构建一个简单的脚本,将某些文档备份到我的第二个硬盘驱动器(你永远不知道会发生什么!)。所以,我使用shutil.copytree 函数将我的数据复制到第二个驱动器上。它工作得很好,这不是问题。

如果目标已经存在,我使用shutil.rmtree 函数删除树。我会告诉你我的代码:

import shutil
import os

def overwrite(src, dest):
    if(not os.path.exists(src)):
        print(src, "does not exist, so nothing may be copied.")
        return

    if(os.path.exists(dest)):
        shutil.rmtree(dest)

    shutil.copytree(src, dest)
    print(dest, "overwritten with data from", src)
    print("")

overwrite(r"C:\Users\Centurion\Dropbox\Documents", r"D:\Backup\Dropbox Documents")
overwrite(r"C:\Users\Centurion\Pictures", r"D:\Backup\All Pictures")

print("Press ENTER to continue...")
input()

如您所见,一个简单的脚本。现在,当我第一次运行脚本时,一切都很好。图片和文档复制到我的D: 驱动器就好了。但是,当我第二次运行时,这是我的输出:

C:\Users\Centurion\Programming\Python>python cpdocsnpics.py
D:\Backup\Dropbox Documents overwritten with data from C:\Users\Centurion\Dropbox\Documents

Traceback (most recent call last):
  File "cpdocsnpics.py", line 17, in <module>
    overwrite(r"C:\Users\Centurion\Pictures", r"D:\Backup\All Pictures")
  File "cpdocsnpics.py", line 10, in overwrite
    shutil.rmtree(dest)
  File "C:\Python34\lib\shutil.py", line 477, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\Python34\lib\shutil.py", line 376, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\Python34\lib\shutil.py", line 374, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 5] Access is denied: 'D:\\Backup\\All Pictures'

该错误仅在我第一次复制Pictures 后发生;我假设它与成为图书馆有关。

我该怎么办?

【问题讨论】:

标签: python windows file shutil


【解决方案1】:

这是一个跨平台的一致性问题。 您已复制具有 readonly 属性的文件/目录。第一次“dest”不存在,因此不执行rmtree方法。但是,当您尝试运行“覆盖”功能时,我们可以注意到“目标”位置存在(及其子树),但它是以只读访问权限复制的。所以这里我们遇到了一个问题。 为了“修复”问题,您必须为shutil.rmtreeonerror 参数提供处理程序。只要您的问题与只读问题有关,解决方法就有点像这样:

def readonly_handler(func, path, execinfo): 
    os.chmod(path, 128) #or os.chmod(path, stat.S_IWRITE) from "stat" module
    func(path)

正如您在 python 文档中看到的那样,onerror 必须是一个可调用的,它接受三个参数:函数、路径和 excinfo。欲了解更多信息,read the docs.

def overwrite(src, dest):
    if(not os.path.exists(src)):
        print(src, "does not exist, so nothing may be copied.")
        return

    if(os.path.exists(dest)):  
        shutil.rmtree(dest, onerror=readonly_handler)

    shutil.copytree(src, dest)
    print(dest, "overwritten with data from", src)
    print("")

当然,这个处理程序简单而具体,但是如果发生其他错误,就会引发新的异常,这个处理程序可能无法修复它们!

注意: Tim Golden(Windows 的 Python 贡献者)一直在修补 shutil.rmtree 问题,看来它会在 Python 3.5 中得到解决(请参阅 issue 19643)。

【讨论】:

  • 有趣!是否所有函数都有onerror 处理程序?
  • @Ken 这取决于!在这种情况下,您的特定错误是由于只读文件中的访问错误造成的。只要您无法预测您的文件是否会被设置为“只读”属性的复制,所以我建议您将回调(作为 readonly_handler)传递给 shutil.rmtree
【解决方案2】:

我发现除了在 Windows 上使用shutil.rmtree 的只读文件之外的问题(在 Windows 7 上测试)。我使用shutil.rmtreeshutil.copytree 的组合在测试套件中创建测试夹具,因此在短时间内(shutil.rmtree 函数在返回调用程序时尚未完成,并且仅在一段时间后才能重新使用已删除的文件名。

TL;DR: 解决方案并不漂亮 - 概括地说,它会在删除目录之前重命名目录,但有一些问题需要处理,因为 Windows 文件系统似乎需要一些时间才能赶上操作散发出香味。实际代码会捕获各种失败条件,并在短暂延迟后重试失败操作的变体。

接下来是更长的讨论,最后是我的最终代码。

我的第一个想法是在删除之前尝试重命名目录树,以便可以立即重新使用原始目录名称。这似乎有帮助。为此,我为rmtree创建了一个替代品,其本质是这样的:

def removetree(tgt):
    def error_handler(func, path, execinfo):
        e = execinfo[1]
        if e.errno == errno.ENOENT or not os.path.exists(path):
            return              # path does not exist - treat as success
        if func in (os.rmdir, os.remove) and e.errno == errno.EACCES:
            os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
            func(path)          # read-only file; make writable and retry
        raise e
    tmp = os.path.join(os.path.dirname(tgt),"_removetree_tmp")
    os.rename(tgt, tmp)
    shutil.rmtree(tmp, onerror=error_handler)
    return

我发现此逻辑是一种改进,但它会导致 os.rename 操作出现不可预知的失败,并出现几个可能的错误之一。所以我还在os.rename周围添加了一些重试逻辑,因此:

def removetree(tgt):
    def error_handler(func, path, execinfo):
        # figure out recovery based on error...
        e = execinfo[1]
        if e.errno == errno.ENOENT or not os.path.exists(path):
            return              # path does not exist
        if func in (os.rmdir, os.remove) and e.errno == errno.EACCES:
            os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
            func(path)          # read-only file; make writable and retry
        raise e
    # Rename target directory to temporary value, then remove it
    count = 0 
    while count < 10:           # prevents indefinite loop
        count += 1
        tmp = os.path.join(os.path.dirname(tgt),"_removetree_tmp_%d"%(count))
        try:
            os.rename(tgt, tmp)
            shutil.rmtree(tmp, onerror=error_handler)
            break
        except OSError as e:
            time.sleep(1)       # Give file system some time to catch up
            if e.errno in [errno.EACCES, errno.ENOTEMPTY]:
                continue        # Try another temp name
            if e.errno == errno.EEXIST:
                shutil.rmtree(tmp, ignore_errors=True)  # Try to clean up old files
                continue        # Try another temp name
            if e.errno == errno.ENOENT:
                break           # 'src' does not exist(?)
            raise               # Other error - propagate
    return

上面的代码没有经过测试,但这里的总体思路似乎确实有效。我实际使用的完整代码如下,并使用了两个函数。它可能包含一些不必要的逻辑,但对我来说似乎更可靠(因为我的测试套件现在在 Windows 上反复通过,而以前它在大多数运行中都无法预料地失败):

def renametree_temp(src):
    """
    Rename tree to temporary name, and return that name, or 
    None if the source directory does not exist.
    """
    count = 0 
    while count < 10:      # prevents indefinite loop
        count += 1
        tmp = os.path.join(os.path.dirname(src),"_removetree_tmp_%d"%(count))
        try:
            os.rename(src, tmp)
            return tmp      # Success!
        except OSError as e:
            time.sleep(1)
            if e.errno == errno.EACCES:
                log.warning("util.renametree_temp: %s EACCES, retrying"%tmp)
                continue    # Try another temp name
            if e.errno == errno.ENOTEMPTY:
                log.warning("util.renametree_temp: %s ENOTEMPTY, retrying"%tmp)
                continue    # Try another temp name
            if e.errno == errno.EEXIST:
                log.warning("util.renametree_temp: %s EEXIST, retrying"%tmp)
                shutil.rmtree(tmp, ignore_errors=True)  # Try to clean up old files
                continue    # Try another temp name
            if e.errno == errno.ENOENT:
                log.warning("util.renametree_temp: %s ENOENT, skipping"%tmp)
                break       # 'src' does not exist(?)
            raise           # Other error: propagaee
    return None

def removetree(tgt):
    """
    Work-around for python problem with shutils tree remove functions on Windows.
    See:
        https://stackoverflow.com/questions/23924223/
        https://stackoverflow.com/questions/1213706/
        https://stackoverflow.com/questions/1889597/
        http://bugs.python.org/issue19643
    """
    # shutil.rmtree error handler that attempts recovery from attempts 
    # on Windows to remove a read-only file or directory (see links above).
    def error_handler(func, path, execinfo):
        e = execinfo[1]
        if e.errno == errno.ENOENT or not os.path.exists(path):
            return          # path does not exist: nothing to do
        if func in (os.rmdir, os.remove) and e.errno == errno.EACCES:
            try:
                os.chmod(path, stat.S_IRWXU| stat.S_IRWXG| stat.S_IRWXO) # 0777
            except Exception as che:
                log.warning("util.removetree: chmod failed: %s"%che)
            try:
                func(path)
            except Exception as rfe:
                log.warning("util.removetree: 'func' retry failed: %s"%rfe)
                if not os.path.exists(path):
                    return      # Gone, assume all is well
                raise
        if e.errno == errno.ENOTEMPTY:
            log.warning("util.removetree: Not empty: %s, %s"%(path, tgt))
            time.sleep(1)
            removetree(path)    # Retry complete removal
            return
        log.warning("util.removetree: rmtree path: %s, error: %s"%(path, repr(execinfo)))
        raise e
    # Try renaming to a new directory first, so that the tgt is immediately 
    # available for re-use.
    tmp = renametree_temp(tgt)
    if tmp:
        shutil.rmtree(tmp, onerror=error_handler)
    return

(以上代码合并了What user do python scripts run as in windows?的只读文件问题的解决方案,根据Deleting directory in Python进行了测试。我认为我没有遇到只读文件问题,所以假设没有在我的测试套件中测试。)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2011-07-09
    • 2013-12-11
    • 2017-09-07
    • 2013-11-04
    • 1970-01-01
    • 2012-07-02
    • 1970-01-01
    相关资源
    最近更新 更多