Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds FAQ for tests failing in CI but passing locally #2571

Merged
merged 2 commits into from
Mar 5, 2020
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions source/faq/questions/dashboard-faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,21 @@ When a run happens and a test fails - instead of going and inspecting your CI pr

No, although we are looking to build an on-premise version of the Dashboard for use in private clouds. If you're interested in our on-premise version, please add your interest to our {% url "Dashboard Product Board" https://portal.productboard.com/cypress-io/1-cypress-dashboard %}!

## {% fa fa-angle-right %} Why do my Cypress tests pass locally but not in CI?

There are many reasons why tests may fail in CI but pass locally. Some of these include:

- There is a problem isolated to the Electron browser (Tests run in CI use the Electron browser)
- A test failure in CI could be highlighting a bug in your CI build process
- Variability in timing when running your application in CI (For example, network requests that resolve within the timeout locally may take longer in CI)

To troubleshoot why tests are failing in CI but passing locally, you can try these strategies:

- Test locally with Electron to identify if the issue is specific to the browser.
- Review your CI build process to ensure nothing is changing with your application that would result in failing tests.
- Remove time-sensitive variability in your tests. For example, ensure a network request has finished before looking for the DOM element that relies on the data from that network request. You can leverage {% url "aliasing" variables-and-aliases#Aliases %} for this.
- Ensure video recording and/or screenshots are enabled for the CI run and compare the recording to the Command Log when running the test locally.

## {% fa fa-angle-right %} Can I choose not to use the Dashboard?

Of course. The Dashboard Service is a separate service from the Test Runner and will always remain optional. We hope you'll find a tremendous amount of value in it, but it is not coupled to being able to run your tests.
Expand Down