From 2f6bce26efe0ba2312b9d0dcb69af03b9bc0feb1 Mon Sep 17 00:00:00 2001 From: Alex Birdsall Date: Wed, 31 Aug 2022 12:46:35 -0700 Subject: [PATCH 1/4] Update README with instructions --- airbyte-webapp-e2e-tests/README.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/airbyte-webapp-e2e-tests/README.md b/airbyte-webapp-e2e-tests/README.md index b3f96ca1f9dfa..0a1fc45cb7d20 100644 --- a/airbyte-webapp-e2e-tests/README.md +++ b/airbyte-webapp-e2e-tests/README.md @@ -1,5 +1,17 @@ -## Start Cypress +## Running Cypress tests locally +Except as noted, commands are written as if run from inside the `airbyte-webapp-e2e-tests/` directory. -### `npm install` +Prerequisites: +1) install the e2e test dependencies and tooling with `npm install` +2) build the OSS backend for the current commit with `SUB_BUILD=PLATFORM ../gradlew clean build` +3) create the test database: `npm run createdb` +4) start the OSS backend: `VERSION=dev docker-compose --file ../docker-compose.yaml up` -### `npm run cypress:open` +If you want the ability to change the frontend as you test (to add `data-testid` attributes to the components being tested, for example, or to practice test-driven development), there are a few additional steps. These are optional, but recommended: +5) run `docker-compose stop webapp` to turn off the dockerized frontend build (which uses a production-style build instead of a hot-reloading dev server) +6) the following two commands should be run separately, so open another terminal window and `cd` into the `airbyte-webapp/` +7) if you have not already done so, run `npm install` to install the frontend app's dependencies +8) start the frontend development server with `npm start` + +Now you're ready to start the test suite! Ensure you're back in the `airbyte-webapp-e2e-tests/` directory, and then +9) start the cypress test runner with `npm run cypress:open` From b5b84f45a449df64e5754cffc64076fb0eb6d1b5 Mon Sep 17 00:00:00 2001 From: Alex Birdsall Date: Wed, 31 Aug 2022 13:01:24 -0700 Subject: [PATCH 2/4] Prevent test failures from intentionally-thrown errors --- airbyte-webapp-e2e-tests/cypress/integration/base.spec.ts | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/airbyte-webapp-e2e-tests/cypress/integration/base.spec.ts b/airbyte-webapp-e2e-tests/cypress/integration/base.spec.ts index b70691a666f11..b4f59dd559d2d 100644 --- a/airbyte-webapp-e2e-tests/cypress/integration/base.spec.ts +++ b/airbyte-webapp-e2e-tests/cypress/integration/base.spec.ts @@ -8,6 +8,8 @@ describe("Error handling view", () => { }, }); + cy.on("uncaught:exception", () => false); + cy.visit("/"); cy.get("div").contains("Version mismatch between 0.0.1-ci and 0.0.2-ci.").should("exist"); @@ -19,6 +21,8 @@ describe("Error handling view", () => { body: "Failed to fetch", }); + cy.on("uncaught:exception", () => false); + cy.visit("/"); cy.get("div").contains("Cannot reach server. The server may still be starting up.").should("exist"); From 684a094d8bc1dcf4baf6a52015b4520c8280f222 Mon Sep 17 00:00:00 2001 From: Alex Birdsall Date: Fri, 2 Sep 2022 15:54:04 -0700 Subject: [PATCH 3/4] Rephrase cypress open instructions in README --- airbyte-webapp-e2e-tests/README.md | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/airbyte-webapp-e2e-tests/README.md b/airbyte-webapp-e2e-tests/README.md index 0a1fc45cb7d20..1bf42ea1c4b0d 100644 --- a/airbyte-webapp-e2e-tests/README.md +++ b/airbyte-webapp-e2e-tests/README.md @@ -1,17 +1,14 @@ -## Running Cypress tests locally -Except as noted, commands are written as if run from inside the `airbyte-webapp-e2e-tests/` directory. +## Running Cypress tests locally with `cypress open` +The most useful way to run tests locally is with the `cypress open` command. It opens a dispatcher window that lets you select which browser to run (including an Electron child window whose devtools will be very familiar to chrome users) and which tests to run in it. By default, this command is configured to visit page urls from port 3000 (as used by a locally-run dev server), not port 8000 (as used by docker-compose's `webapp` service). If you want to run tests against the dockerized UI instead, leave the `webapp` docker-compose service running in step 4) and start the test runner with `CYPRESS_BASE_URL=http://localhost:8000 npm run cypress:open` in step 8). -Prerequisites: -1) install the e2e test dependencies and tooling with `npm install` -2) build the OSS backend for the current commit with `SUB_BUILD=PLATFORM ../gradlew clean build` -3) create the test database: `npm run createdb` -4) start the OSS backend: `VERSION=dev docker-compose --file ../docker-compose.yaml up` - -If you want the ability to change the frontend as you test (to add `data-testid` attributes to the components being tested, for example, or to practice test-driven development), there are a few additional steps. These are optional, but recommended: -5) run `docker-compose stop webapp` to turn off the dockerized frontend build (which uses a production-style build instead of a hot-reloading dev server) -6) the following two commands should be run separately, so open another terminal window and `cd` into the `airbyte-webapp/` -7) if you have not already done so, run `npm install` to install the frontend app's dependencies -8) start the frontend development server with `npm start` +Except as noted, all commands are written as if run from inside the `airbyte-webapp-e2e-tests/` directory. -Now you're ready to start the test suite! Ensure you're back in the `airbyte-webapp-e2e-tests/` directory, and then -9) start the cypress test runner with `npm run cypress:open` +Prerequisites: +1) Install the e2e test dependencies and tooling with `npm install`. +2) Build the OSS backend for the current commit with `SUB_BUILD=PLATFORM ../gradlew clean build`. +3) Create the test database: `npm run createdb`. +4) Start the OSS backend: `VERSION=dev docker-compose --file ../docker-compose.yaml up`. If you want, follow this with `docker-compose stop webapp` to turn off the dockerized frontend build; interactive cypress sessions don't use it. +5) The following two commands will start a separate long-running server, so open another terminal window. In it, `cd` into the `airbyte-webapp/` directory. +6) If you have not already done so, run `npm install` to install the frontend app's dependencies. +7) Start the frontend development server with `npm start`. +8) Back in the `airbyte-webapp-e2e-tests/` directory, start the cypress test runner with `npm run cypress:open`. From dcd7543bb9526fd339fb23a0da2523454d716abc Mon Sep 17 00:00:00 2001 From: Alex Birdsall Date: Fri, 2 Sep 2022 16:17:07 -0700 Subject: [PATCH 4/4] Add CI repro documentation --- airbyte-webapp-e2e-tests/README.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/airbyte-webapp-e2e-tests/README.md b/airbyte-webapp-e2e-tests/README.md index 1bf42ea1c4b0d..81efe0fe527c7 100644 --- a/airbyte-webapp-e2e-tests/README.md +++ b/airbyte-webapp-e2e-tests/README.md @@ -1,10 +1,12 @@ -## Running Cypress tests locally with `cypress open` -The most useful way to run tests locally is with the `cypress open` command. It opens a dispatcher window that lets you select which browser to run (including an Electron child window whose devtools will be very familiar to chrome users) and which tests to run in it. By default, this command is configured to visit page urls from port 3000 (as used by a locally-run dev server), not port 8000 (as used by docker-compose's `webapp` service). If you want to run tests against the dockerized UI instead, leave the `webapp` docker-compose service running in step 4) and start the test runner with `CYPRESS_BASE_URL=http://localhost:8000 npm run cypress:open` in step 8). +## Running an interactive Cypress session with `npm run cypress:open` +The most useful way to run tests locally is with the `cypress open` command. It opens a dispatcher window that lets you select which tests and browser to run; the Electron browser (whose devtools will be very familiar to chrome users) opens a child window, and having both cypress windows grouped behaves nicely when switching between applications. In an interactive session, you can use `it.skip` and `it.only` to focus on the tests you care about; any change to the source file of a running test will cause tests to be automatically rerun. At the end of a test run, the web page is left "dangling" with all state present at the end of the last test; you can click around, "inspect element", and interact with the page however you wish, which makes it easy to incrementally develop tests. + +By default, this command is configured to visit page urls from port 3000 (as used by a locally-run dev server), not port 8000 (as used by docker-compose's `webapp` service). If you want to run tests against the dockerized UI instead, leave the `webapp` docker-compose service running in step 4) and start the test runner with `CYPRESS_BASE_URL=http://localhost:8000 npm run cypress:open` in step 8). Except as noted, all commands are written as if run from inside the `airbyte-webapp-e2e-tests/` directory. -Prerequisites: -1) Install the e2e test dependencies and tooling with `npm install`. +Steps: +1) If you have not already done so, run `npm install` to install the e2e test dependencies. 2) Build the OSS backend for the current commit with `SUB_BUILD=PLATFORM ../gradlew clean build`. 3) Create the test database: `npm run createdb`. 4) Start the OSS backend: `VERSION=dev docker-compose --file ../docker-compose.yaml up`. If you want, follow this with `docker-compose stop webapp` to turn off the dockerized frontend build; interactive cypress sessions don't use it. @@ -12,3 +14,15 @@ Prerequisites: 6) If you have not already done so, run `npm install` to install the frontend app's dependencies. 7) Start the frontend development server with `npm start`. 8) Back in the `airbyte-webapp-e2e-tests/` directory, start the cypress test runner with `npm run cypress:open`. + +## Reproducing CI test results with `npm run cypress:ci` or `npm run cypress:ci:record` +Unlike `npm run cypress:open`, `npm run cypress:ci` and `npm run cypress:ci:record` use the dockerized UI (i.e. they expect the UI at port 8000, rather than port 3000). If the OSS backend is running but you have run `docker-compose stop webapp`, you'll have to re-enable it with `docker-compose start webapp`. These trigger headless runs: you won't have a live browser to interact with, just terminal output. + +Except as noted, all commands are written as if run from inside the `airbyte-webapp-e2e-tests/` directory. + +Steps: +1) If you have not already done so, run `npm install` to install the e2e test dependencies. +2) Build the OSS backend for the current commit with `SUB_BUILD=PLATFORM ../gradlew clean build`. +3) Create the test database: `npm run createdb`. +4) Start the OSS backend: `VERSION=dev docker-compose --file ../docker-compose.yaml up`. +5) Start the cypress test run with `npm run cypress:ci` or `npm run cypress:ci:record`.