Run the upgrade and downgrade tests in the upgrade prow job #300

houshengbo · 2020-02-17T16:37:27Z

Fixes #310

Proposed Changes

This PR adds the execution of upgrade and downgrade tests into the operator prow job.
This PR refactors the e2e-common script, to accommodate the changes in upgrade tests.
This PR adds the deployment verification for Knative in operator's upgrade tests.

Release Note

knative-prow-robot

@houshengbo: 0 warnings.

In response to this:

Fixes #

Proposed Changes

Release Note

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

knative-prow-robot · 2020-02-17T16:38:00Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: houshengbo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [houshengbo]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

We change the test namespace, because all the tests in serving repository are running under the namespace knative-serving. We will call the upgrade and downgrade tests directly in serving. We need to make sure all the tests are running under the same namespace.

…ages

* We do not need to delete the operator CR, becuase we will verify the services in serving. * We do not need to create the opertor CR, because we will install a previous version of the operator and create CR before reaching this test. * We need to verify the status of all the deployments, making sure they match the expected deployments in terms of number and names, and reach the ready status.

aliok · 2020-03-02T07:43:28Z

test/e2e-common.sh

-readonly TEST_NAMESPACE="operator-tests"
+readonly TEST_NAMESPACE="knative-serving"
+
+OPERATOR_DIR=$(dirname $0)/..


Suggested change

OPERATOR_DIR=$(dirname $0)/..

OPERATOR_DIR=$(cd $(dirname "$0")/.. && pwd)

OPERATOR_DIR=$(dirname $0)/.. resolves to ./...
Changing it with OPERATOR_DIR=$(cd $(dirname "$0")/.. && pwd) results in OPERATOR_DIR being e.g. /home/ali/go/src/knative.dev/

aliok · 2020-03-02T11:09:54Z

@houshengbo any instructions to run this test locally? I couldn't on Minikube and on OpenShift

houshengbo · 2020-03-02T18:13:37Z

@aliok this is good question, but it seems to me that running the tests against GKE is the ONLY way to verify the tests.

aliok · 2020-03-03T06:35:18Z

but it seems to me that running the tests against GKE is the ONLY way to verify the tests.

then it lgtm as things pass on GKE. I created #317 so that I don't forget it and hopefully make things working later.

/lgtm

houshengbo changed the title ~~Add the upgrade tests in Serving~~ WIP: Add the upgrade tests in Serving Feb 17, 2020

knative-prow-robot added the do-not-merge/work-in-progress label Feb 17, 2020

knative-prow-robot reviewed Feb 17, 2020

View reviewed changes

knative-prow-robot requested review from bbrowning and trshafer February 17, 2020 16:37

knative-prow-robot added the size/XXL label Feb 17, 2020

googlebot added the cla: yes Author(s) signed a CLA. label Feb 17, 2020