如果测试用例失败，则将子流程的标准输出添加到 JSON 报告

Question

我正在研究添加到由pytest-json 或pytest-json-report 生成的JSON 报告的方法：我没有挂断这两个插件。到目前为止，我已经使用pytest-json 完成了大部分评估。因此，例如，JSON 对象有这个用于测试用例

{
    "name": "fixture_test.py::test_failure1",
    "duration": 0.0012421607971191406,
    "run_index": 2,
    "setup": {
        "name": "setup",
        "duration": 0.00011181831359863281,
        "outcome": "passed"
    },
    "call": {
        "name": "call",
        "duration": 0.0008759498596191406,
        "outcome": "failed",
        "longrepr": "def test_failure1():\n>       assert 3 == 4, \"3 always equals 3\"\nE       AssertionError: 3 always equals 3\nE       assert 3 == 4\n\nfixture_test.py:19: AssertionError"
    },
    "teardown": {
        "name": "teardown",
        "duration": 0.00014257431030273438,
        "outcome": "passed"
    },
    "outcome": "failed"
}

这是来自我正在尝试的实验。在实践中，一些测试用例是通过Popen 产生一个子进程来完成的，并且断言是某个字符串出现在标准输出中。如果测试用例失败，我需要向包含该子进程的标准输出的 call 字典添加一个键/值。到目前为止，我一直徒劳地尝试找到正确的固定装置或设备来完成此任务。似乎pytest_exception_interact 可能是要走的路，但到目前为止我还没有深入研究 JSON 结构。我需要做的就是在发生错误时添加/修改 JSON 结构。看来pytest_runtest_call太重了。

或者，有没有办法改变上面longrepr 的值？我一直无法找到执行这些操作的正确方法，现在该提问了。

Answer 1

看起来，pytest-json 项目已经失效。 pytest-json-report 的开发者/所有者有话要说（在 Related Tools at this link 下）。

pytest-json 有一些很棒的功能，但似乎没有维护。我从那里借鉴了一些想法和测试用例。

pytest-json-report 项目正好处理我需要的情况：从子流程中捕获标准输出并将其放入 JSON 报告中。这样做的粗略示例如下：

import subprocess as sp
import pytest
import sys
import re

def specialAssertHandler(str, assertMessage):
    # because pytest automatically captures stdout,stderr this is all that's needed
    # when the report is generated, this will be in a field named "stdout"
    print(str)
    return assertMessage

def test_subProcessStdoutCapture():
    # NOTE: if you're version of Python 3 is sufficiently mature, add text=True also
    proc = sp.Popen(['find', '.', '-name', '*.json'], stdout=sp.PIPE)

    # NOTE: I had this because on the Ubuntu I was using, this is the version of
    # Python and the return of proc.stdout.read() is a binary object not a string
    if sys.version[0] == 3 and sys.version[6]:
        output = proc.stdout.read().decode()
    elif sys.version[0] == 2:
        # The other version of Python I'm using is 2.7.15, it's exceedingly frustrating
        # that the Python language def changed so between 2 and 3.  In 2, the output
        # was already a string object
        output = proc.stdout.read()

    m = re.search('some string', output)
    assert m is not None, specialAssertHandler(output, "did not find 'some string' in output")

通过上述方法，使用pytest-json-report，子流程的完整输出被基础设施捕获并放入上述报告中。摘录如下：

        {
            "nodeid": "expirment_test.py::test_stdout",
            "lineno": 25,
            "outcome": "failed",
            "keywords": [
                "PyTest",
                "test_stdout",
                "expirment_test.py"
            ],
            "setup": {
                "duration": 0.0002694129943847656,
                "outcome": "passed"
            },
            "call": {
                "duration": 0.02718186378479004,
                "outcome": "failed",
                "crash": {
                    "path": "/home/afalanga/devel/PyTest/expirment_test.py",
                    "lineno": 32,
                    "message": "AssertionError: Expected to find always\nassert None is not None"
                },
                "traceback": [
                    {
                        "path": "expirment_test.py",
                        "lineno": 32,
                        "message": "AssertionError"
                    }
                ],
                "stdout": "./.report.json\n./report.json\n./report1.json\n./report2.json\n./simple_test.json\n./testing_addition.json\n\n",
                "longrepr": "..."
            },
            "teardown": {
                "duration": 0.0004875659942626953,
                "outcome": "passed"
            }
        }

longrepr 字段包含测试用例的全文，但为了简洁起见，它被省略了。在crash 字段中，放置了我的示例中assertMessage 的值。这表明可以在发生时将此类消息放入报告中，而不是进行后处理。

我认为可以使用我在原始问题pytest_exception_interact 中引用的钩子“巧妙地”处理这个问题。如果我发现是这样，我将通过演示更新此答案。