Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--output=json flag to allow external tooling to be developed on top of cptest #40

Open
kuredoro opened this issue Oct 7, 2021 · 1 comment
Milestone

Comments

@kuredoro
Copy link
Owner

kuredoro commented Oct 7, 2021

cptest produces go test -v-like reports, but if somebody wants to process the reports themselves (to generate html pages, etc.), they have no pain-less way to do it. This proposal aims to increase interoperability with other tools.

The format needs to be decided. It should interoperate with #6 too. This is minimal, I think

[
    {
        "input": "foo\n",
        "expected": "foo\n",
        "stdout": "bar\n",
        "stderr": "whaaa?\n",
        "exit_code": 0,
        "run_time": "1.04s",
        "verdict": "TL"
    },
    null,
    {
        // ...
    }
]
@kuredoro
Copy link
Owner Author

kuredoro commented Oct 10, 2021

So, okay, ALL of cptest output should be in json. Meaning that errors should go there too. Moreover, a struct is a better choice anyway, since it can be extended without breaking backwards compatibility.

v0.2

{
    "results": [
        {
            "input": "foo\n",
            "expected": "foo\n",
            "stdout": "bar\n",
            "stderr": "whaaa?\n",
            "exit_code": 0,
            "run_time": "1.04s",
            "verdict": "TL"
        },
        null,
        {
            // ...
        }
    ],
    "errors": [
        {
            // think about how to represent the error hierachy
        }
    ]    
}

@kuredoro kuredoro changed the title --output-json flag to allow external tooling to be developed on top of cptest --output=json flag to allow external tooling to be developed on top of cptest Oct 10, 2021
@kuredoro kuredoro added this to the ver2.04a milestone Sep 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant