在 Windows 下如何共享日志文件？

Question

我有几个不同的进程，我希望它们都记录到同一个文件。这些进程在 Windows 7 系统上运行。有些是 python 脚本，有些是cmd 批处理文件。

在 Unix 下，您只需让每个人都以附加模式打开文件并写掉。只要每个进程在单个消息中写入的字节数少于PIPE_BUF，就可以保证每个write 调用不会与任何其他调用交错。

有没有办法在 Windows 下实现这一点？幼稚的类 Unix 方法失败了，因为 Windows 不喜欢默认情况下一次打开一个文件以供写入的多个进程。

Answer 1

可以让多个批处理进程安全地写入单个日志文件。我对 Python 一无所知，但我想这个答案中的概念可以与 Python 集成。

Windows 最多允许一个进程在任何时间点打开特定文件以进行写访问。这可用于实现基于文件的锁定机制，以保证事件在多个进程中被序列化。有关示例，请参阅 https://stackoverflow.com/a/9048097/1012053 和 http://www.dostips.com/forum/viewtopic.php?p=12454。

由于您要做的只是写入日志，因此您可以将日志文件本身用作锁。日志操作被封装在一个尝试以附加模式打开日志文件的子例程中。如果打开失败，则例程循环返回并再次尝试。一旦打开成功，日志将被写入然后关闭，并且例程返回给调用者。例程执行传递给它的任何命令，并且在例程中写入标准输出的任何内容都将重定向到日志。

这是一个测试批处理脚本，它创建了 5 个子进程，每个子进程写入日志文件 20 次。写入是安全交错的。

@echo off
setlocal
if "%~1" neq "" goto :test

:: Initialize
set log="myLog.log"
2>nul del %log%
2>nul del "test*.marker"
set procCount=5
set testCount=10

:: Launch %procCount% processes that write to the same log
for /l %%n in (1 1 %procCount%) do start "" /b "%~f0" %%n

:wait for child processes to finish
2>nul dir /b "test*.marker" | find /c "test" | >nul findstr /x "%procCount%" || goto :wait

:: Verify log results
for /l %%n in (1 1 %procCount%) do (
  <nul set /p "=Proc %%n log count = "
  find /c "Proc %%n: " <%log%
)

:: Cleanup
del "test*.marker"
exit /b

==============================================================================
:: code below is the process that writes to the log file

:test
set instance=%1
for /l %%n in (1 1 %testCount%) do (
  call :log echo Proc %instance% says hello!
  call :log dir "%~f0"
)
echo done >"test%1.marker"
exit

:log command args...
2>nul (
  >>%log% (
    echo ***********************************************************
    echo Proc %instance%: %date% %time%
    %*
    (call ) %= This odd syntax guarantees the inner block ends with success  =%
            %= We only want to loop back and try again if redirection failed =%
  )
) || goto :log
exit /b

这是证明每个进程的所有 20 次写入都成功的输出

Proc 1 log count = 20
Proc 2 log count = 20
Proc 3 log count = 20
Proc 4 log count = 20
Proc 5 log count = 20

您可以打开生成的“myLog.log”文件以查看写入是如何安全交错的。但是输出太大，无法在此处发布。

通过修改 :log 例程使其不会在失败时重试，很容易证明来自多个进程的同时写入可能会失败。

:log command args...
>>%log% (
  echo ***********************************************************
  echo Proc %instance%: %date% %time%
  %*
)
exit /b

以下是“破坏” :log 例程后的一些示例结果

The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
The process cannot access the file because it is being used by another process.
Proc 1 log count = 12
Proc 2 log count = 16
Proc 3 log count = 13
Proc 4 log count = 18
Proc 5 log count = 14

Answer 2

你可以试试这个 Python 模块： http://pypi.python.org/pypi/ConcurrentLogHandler

它提供了一个插入式替换 RotatingFileHandler，它允许多个进程同时记录到单个文件，而不会丢弃或破坏日志事件。

我没有使用过它，但是我在阅读 Python 中的一个相关错误 (Issue 4749) 时发现了它。

如果您实现自己的代码而不是使用该模块，请务必阅读该错误！

您可以像在 Bash 中一样在 Windows 上使用 output redirection。将批处理文件的输出通过管道传输到 Python 脚本，该脚本通过 ConcurrentLogHandler 记录日志。