Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make notebook test errors more readable #3620

Closed
balopat opened this issue Dec 18, 2020 · 0 comments · Fixed by #3751
Closed

Make notebook test errors more readable #3620

balopat opened this issue Dec 18, 2020 · 0 comments · Fixed by #3751
Labels
area/notebook-testing kind/health For CI/testing/release process/refactoring/technical debt items

Comments

@balopat
Copy link
Contributor

balopat commented Dec 18, 2020

As pointed out in #3604 (comment) test failures spit out a bunch of json that are hard to parse. We should make this better. Again treebeard might be the solution here.
@balopat balopat added kind/health For CI/testing/release process/refactoring/technical debt items area/notebook-testing labels Dec 18, 2020
@balopat balopat mentioned this issue Dec 18, 2020
Merged
@balopat balopat mentioned this issue Feb 4, 2021
Merged
CirqBot pushed a commit that referenced this issue Feb 5, 2021
@balopat
Notebook tests against both released Cirq and PRs (#3751)
d14eb7c 
This PR adds notebook testing based on the PR branch in a non-isolated environment (hence it's faster). The isolated notebook tests against the latest released version of Cirq are only run now on changed notebooks (hence it's much faster too). I also improved the notebook error messages output - it only prints the errors. If there is a notebook failure, the output notebooks are also downloadable on the Github Action workflow as an artifact.

Fixes #3620.
Helps with #3603. (in order to fix that we'll probably need to introduce parametrization of the notebooks, for minimal data, which papermill can do)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/notebook-testing kind/health For CI/testing/release process/refactoring/technical debt items
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Notebook tests against both released Cirq and PRs balopat/Cirq
1 participant
@balopat