Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor localdb tests #5078

Merged
merged 66 commits into from
Feb 7, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
d806282
add e2e_localdb tests
zainasir Dec 16, 2024
64fb0f8
fix circleci config typo
zainasir Dec 16, 2024
4ef57e7
fix path to test script for e2e-localdb
zainasir Dec 16, 2024
ab27c07
update runner machine
zainasir Dec 17, 2024
21ccc02
remove docker caching
zainasir Dec 17, 2024
f66b2de
use correct yarn versions
zainasir Dec 17, 2024
fd5a055
explicitly call build script
zainasir Dec 17, 2024
ff3dce4
install yarn dependencies
zainasir Dec 17, 2024
265bba9
testing ubuntu machine
zainasir Dec 18, 2024
dee380d
break e2e_localdb tests into smaller jobs
zainasir Dec 19, 2024
fc6b4b4
testing independent steps
zainasir Dec 19, 2024
3edeff7
fix circleci config schema
zainasir Dec 19, 2024
daad43e
fix typo in test codebase git url
zainasir Dec 19, 2024
c020cff
set up yarn env
zainasir Dec 19, 2024
921733e
add instructions for running new e2e-localdb tests
zainasir Jan 2, 2025
6fa5b37
fix e2e-localdb instructions
zainasir Jan 2, 2025
ed657c6
remove instructions for old e2e-localdb tests
zainasir Jan 2, 2025
a4f888a
use new database image
zainasir Jan 9, 2025
88d64bc
fix job name
zainasir Jan 9, 2025
b6f883b
use https for frontend server
zainasir Jan 10, 2025
aa31e7b
add keycloak auth to localdb tests
zainasir Jan 17, 2025
8db0333
support keycloak for circleci e2e tests
zainasir Jan 21, 2025
0d47ade
fix typo
zainasir Jan 21, 2025
7f06b88
fix typo
zainasir Jan 21, 2025
2a150af
support configurable frontend setup for localdb tests
zainasir Jan 22, 2025
3aa0148
update circleci localdb tests
zainasir Jan 22, 2025
46c810c
use custom application properties for localdb tests
zainasir Jan 27, 2025
eb1c750
use latest mysql test database
zainasir Jan 29, 2025
9fa1a2b
support custom backend image
zainasir Jan 29, 2025
be4c347
use master backend image
zainasir Jan 29, 2025
04cb4b9
store docker compose logs as artifact
zainasir Jan 29, 2025
1e61dda
add spec parameter to localdb tests
zainasir Jan 30, 2025
413a9fe
store artifacts and screenshots
zainasir Jan 30, 2025
ce50ee5
update screenshot and junit dir paths
zainasir Jan 30, 2025
5473bf3
update screenshot directory
zainasir Jan 30, 2025
ee025ed
use webdriver manager local
zainasir Jan 31, 2025
ed36132
use webdriver manager local
zainasir Jan 31, 2025
0406bff
update build frontend commands
zainasir Jan 31, 2025
a5b031e
update artifact paths
zainasir Jan 31, 2025
5aa6dd7
typo
zainasir Jan 31, 2025
cd52348
test docker version of localdb tests
zainasir Feb 3, 2025
370b118
remove frontend server steps
zainasir Feb 3, 2025
1c8b7d3
update local screenshots path
zainasir Feb 3, 2025
66f220a
use relative paths
zainasir Feb 3, 2025
ebae083
add prettier to new tests
zainasir Feb 3, 2025
852df9c
fix prettier workspace path
zainasir Feb 3, 2025
1e9fbbe
update dependency tree for prettier tests
zainasir Feb 3, 2025
b6d31b7
move api_sync to new tests
zainasir Feb 3, 2025
402adaa
cache build and dependencies
zainasir Feb 3, 2025
b11e2d1
update paths for checksum keys
zainasir Feb 3, 2025
f55545a
update checksum path
zainasir Feb 3, 2025
164640b
update cache version
zainasir Feb 3, 2025
500db0e
move import statement check to new tests
zainasir Feb 3, 2025
9d4642c
remove old tests
zainasir Feb 3, 2025
4a2b776
fix end-to-end-tests
zainasir Feb 4, 2025
6219260
force linux dependencies to always succeed
zainasir Feb 4, 2025
e398771
serve frontend in the background
zainasir Feb 4, 2025
29b8a61
use insecure connection check for frontend
zainasir Feb 4, 2025
d21d205
use https
zainasir Feb 4, 2025
95d8ba1
start frontend server in a single shell
zainasir Feb 4, 2025
1011385
use absolute paths
zainasir Feb 4, 2025
6b25383
add status badge for circleci tests
zainasir Feb 4, 2025
e3a105e
cleanup and remove old localdb files
zainasir Feb 7, 2025
1a54df2
update local db docs
zainasir Feb 7, 2025
48315da
track screenshots in localdb
zainasir Feb 7, 2025
3e265bc
remove circleci status badge
zainasir Feb 7, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
540 changes: 269 additions & 271 deletions .circleci/config.yml

Large diffs are not rendered by default.

67 changes: 8 additions & 59 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ A custom frontend can be tested against any backend in the web browser using a l
## Run of `localdb` e2e-tests
To enable e2e-tests on for features that depend on data that are not included in studies served by the public cBioPortal instance, cbioportal-frontend provides the `e2e local database` (refered to as _e2e-localdb_ or _local e2e_ in this text) facility that allows developers to load custom studies in any backend version used for e2e-tests. CircleCI runs the `e2e-localdb` tests as a separate job.

Files for the local database e2e-tests are located in the `./end-to-end-test/local` directory of cbioportal-frontend. The directory structure of `./end-to-end-test/local` is comparable to that of the `./end-to-end-test/remote` directory used for e2e-tests against remote public cBioPortal instances.
The script that can be used to run e2e-localdb tests is located at [./scripts/e2e.sh](./scripts/e2e.sh).

### Running `localdb` e2e-tests for development

Expand All @@ -195,67 +195,16 @@ Files for the local database e2e-tests are located in the `./end-to-end-test/loc

3. You need to have a global version of Maven installed.

In a terminal, start the frontend dev server

```
export BRANCH_ENV=custom
yarn install --frozen-lockfile // only necessary first time
yarn buildDLL:dev // only necessary first tiem
yarn start
```

3. Install dev dependencies:
```bash
cd end-to-end-test
yarn
```
5. In a second terminal at project root, spinup the backend (api) instance:

```
// if you are running for first time, you will need to build the docker containers.
// Answer yes when it prompts you to do so. This will take at least 20 minutes depending
// on your system speed.
// Once you have done this, you can answer no on subsequent attempts

In a terminal, run the following commands from root directory.
```shell
# Start backend and frontend servers
yarn run e2e:spinup
```

6. When backend instance is operational, you can run tests. Upon executing
the command below, a browser should open and you should see your tests execute.

# Run tests
yarn run e2e:local
```
//grep accepts fragments of file name,
//but you MUST using trailing *
//you need only match the file name, not path

yarn run e2e:local --grep=some.spec*

```



### Running e2e-localdb tests _CircleCI_ or _CircleCI+PR_ context
E2e-tests on _CircleCI_ and _CircleCI+PR_ context are triggered via _hooks_ configured on GitHub. Configuration of hooks falls beyond the scope of this manual.

#### cBioPortal-backend version
E2e-testing against a local database removes dependence on data provided by public cbioportal instances for testing. This makes it possible to test features for data types that are not provided by public cbioportal instances or test features that depend on a backend feature not yet integrated in public cbioportal instances. E2e-localdb tests make use of the `BACKEND` environmental variable to test against a specific backend version. Depending on the running context (see section above) setting the `BACKEND` environmental variable is required or optional (see table below).

Requirement for setting the BACKEND variable depends on the context of the job:

| **context** | **BACKEND var** | **comments** |
|------------------------ | ----------------- | ------------ |
| _Local_ | mandatory | |
| _CircleCI_ | mandatory for feature branches | not for `master` or `rc` builds |
| _CircleCI+PR_ | optional | 1. When specified, GitHub PR must be of 'draft' state.<br>2. When not-specified, backend branch will mirror frontend branch (rc frontend vs. rc backend) |

The value of the `BACKEND` variable must be formatted as `<BACKEND_GITHUB_USER>:<BACKEND_BRANCH>`. For example, when the /env/custom.sh file contains `export BACKEND=thehyve:uwe7872A` this is interpreted as a requirement for the commit `uwe7872A` of the _github.com/thehyve/cbioportal_ repository.

#### BACKEND environmental variable in _CircleCI+PR_ context
Using the `BACKEND` variable e2e-localdb tests can be conducted against any backend version. This poses a risk when testing in _CircleCI+PR_ context, i.e., tests show up as succesful but should have failed against the backend version that compatible with the target cbioportal-frontend branch. To guard against this and prevent merging of incompatible branches into cbioportal-frontend the e2e-localdb tests enforce the use of _draft_ pull requests (see [here](https://help.github.com/en/articles/creating-a-pull-request) for more info). When a cBioPortal backend version is specified (i.e., may require a not yet merged backend branch) and the branch is part of a pull request, the pull request must be in state _draft_. Only when the `BACKEND` variable is not defined a (non-_draft_) e2e-localdb tests will be conducted on branches that are part of pull requests. Needles to say, pull request should for this and other reasons only be merged when the e2e-localdb tests succeed!

When the `BACKEND` variable is not set, the backend version will be set to the target branch of the pull request, i.e. a pull request to 'rc' branch will be tested against the 'rc' branch of the backend.

#### Writing e2e tests
### Writing e2e tests
Some random remarks on e2e-test development
- Screenshot tests and DOM-based tests are contained in files that end with *.screenshot.spec.js or *.spec.js, respectively.
- Screenshot tests should only be used to test components that cannot be accessed via the DOM.
Expand All @@ -282,7 +231,7 @@ Making e2e-tests follows the current procedure for the e2e-tests:
* Gene panel and gene set matrix data of custom studies must comply with gene panel/sets imported as part of study_es_0.
* Imports of custom seed data for gene panels and gene sets are not implemented at the moment of this writing.
* In order to minimize time of local database e2e-tests the size of custom studies should be kept as small as possible.
* When developing in _Local_ context port 8081 can be used to access the cbioportal instance ('http://localhost:8081/cbioportal').
* When developing in _Local_ context port 8080 can be used to access the cbioportal instance ('http://localhost:8080').

#### Debugging help
Here are some errors that have been encountered and are hard to debug.
Expand Down
17 changes: 0 additions & 17 deletions end-to-end-test/local/docker_compose/build.sh

This file was deleted.

15 changes: 0 additions & 15 deletions end-to-end-test/local/docker_compose/cbioportal-custombranch.yml

This file was deleted.

20 changes: 0 additions & 20 deletions end-to-end-test/local/docker_compose/cbioportal.yml

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
FROM circleci/node:15.2.1-browsers
MAINTAINER Pim van Nierop <pim@thehyve.nl>

LABEL Description="End-2-end test image for cBioPortal frontend"

#echo INSTALL cavas deps [necessary for visual regression framework]
#RUN apt-get install build-essential libcairo2-dev libpango1.0-dev libjpeg-dev libgif-dev librsvg2-dev

USER root

CMD /cbioportal-frontend/end-to-end-test/local/docker_compose/docker/screenshottest.sh
5 changes: 0 additions & 5 deletions end-to-end-test/local/docker_compose/docker/screenshottest.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,9 @@ set -e
set -u # unset variables throw error
set -o pipefail # pipes fail when partial command fails

#echo INSTALL SCREENSHOT DEPENDENCIES
#cd /cbioportal-frontend/end-to-end-test
#yarn install --frozen-lockfile

export FRONTEND_TEST_USE_LOCAL_DIST=false
export HEADLESS_CHROME=false

#echo START SERVE_DIST
cd /cbioportal-frontend
yarn serveDistLocalDb &

Expand Down
82 changes: 0 additions & 82 deletions end-to-end-test/local/docker_compose/initdb.sh

This file was deleted.

50 changes: 0 additions & 50 deletions end-to-end-test/local/docker_compose/keycloak.yml

This file was deleted.

Loading