Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable Model Registry test in Smoke #1883

Merged
merged 10 commits into from
Oct 2, 2024
Merged

Enable Model Registry test in Smoke #1883

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
625811a
Enable component if needed
lugi0 Oct 2, 2024
e65a75a
update tags
lugi0 Oct 2, 2024
48c67f6
wait for namespace to become active after enabling component
lugi0 Oct 2, 2024
9c03d9c
Remove sleep after code execution in Jupyter
lugi0 Oct 2, 2024
1f73cbb
Upload files after accessing the workbench rather than before
lugi0 Oct 2, 2024
69d626c
Ensure namespace exists before waiting
lugi0 Oct 2, 2024
db54663
change if condition
lugi0 Oct 2, 2024
cc35c5e
refresh dashboard if model registry not present
lugi0 Oct 2, 2024
be24676
change while loop behaviour
lugi0 Oct 2, 2024
048a252
add small sleep in jupyter
lugi0 Oct 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions ods_ci/tests/Resources/OCP.resource
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -302,3 +302,18 @@ Get Resource Attribute
Should Be Equal "${rc}" "0" msg=${value}
Log To Console ${resource_type} ${resource_name} in the namespace ${namespace} attribute ${attribute_path} value is ${value}
RETURN ${value}

Wait For Namespace To Be Active
[Documentation] Waits for a given namespace to become active
[Arguments] ${namespace} ${timeout}=30s
WHILE True limit=${timeout}
${rc}= Run And Return Rc oc get namespace/${namespace}
IF "${rc}" == "0"
BREAK
END
END
${rc} ${value}= Run And Return Rc And Output
... oc wait --for jsonpath='{.status.phase}=Active' --timeout=${timeout} namespace/${namespace}
Log ${rc}
Log ${value}
Should Be Equal As Integers ${rc} 0
43 changes: 35 additions & 8 deletions ods_ci/tests/Tests/1300__model_registry/1302_model_registry_model_serving.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,6 @@
${JUPYTER_NOTEBOOK_FILEPATH}= ${MODELREGISTRY_BASE_FOLDER}/${JUPYTER_NOTEBOOK}
${DC_S3_TYPE}= Object storage
${NAMESPACE_ISTIO}= istio-system
# ${NAMESPACE_MODEL_REGISTRY}= odh-model-registries
${SECRET_PART_NAME_1}= modelregistry-sample-rest
${SECRET_PART_NAME_2}= modelregistry-sample-grpc
${SECRET_PART_NAME_3}= model-registry-db
Expand All @@ -44,13 +43,14 @@
${MR_REGISTERED_MODEL_AUTHOR}= Tony
${MR_TABLE_XPATH}= //table[@data-testid="registered-model-table"]
${MR_VERSION_TABLE_XPATH}= //table[@data-testid="model-versions-table"]
${DISABLE_COMPONENT}= ${False}


*** Test Cases ***
# robocop: disable:line-too-long
Verify Model Registry Integration With Secured-DB
[Documentation] Verifies the Integartion of Model Registry operator with Jupyter Notebook
[Tags] OpenDataHub MRMS1302 Smoke ExcludeOnRhoai
[Tags] Smoke MRMS1302
Create Workbench workbench_title=${WORKBENCH_TITLE} workbench_description=Registry test
... prj_title=${PRJ_TITLE} image_name=Minimal Python deployment_size=${NONE}
... storage=Persistent pv_existent=${NONE}
Expand All @@ -64,16 +64,15 @@
Data Connection Should Be Listed name=${DC_S3_NAME} type=${DC_S3_TYPE} connected_workbench=${workbenches}
Open Data Science Project Details Page project_title=${prj_title} tab_id=workbenches
Wait Until Workbench Is Started workbench_title=${WORKBENCH_TITLE}
Sleep 5s reason=Sometimes the pod/container is not found by oc, let's give it some time
${handle}= Launch And Access Workbench workbench_title=${WORKBENCH_TITLE}
... username=${TEST_USER.USERNAME} password=${TEST_USER.PASSWORD}
... auth_type=${TEST_USER.AUTH_TYPE}
Upload File In The Workbench filepath=${SAMPLE_ONNX_MODEL} workbench_title=${WORKBENCH_TITLE}
... workbench_namespace=${PRJ_TITLE}
Upload File In The Workbench filepath=${JUPYTER_NOTEBOOK_FILEPATH} workbench_title=${WORKBENCH_TITLE}
... workbench_namespace=${PRJ_TITLE}
Download Python Client Dependencies ${MR_PYTHON_CLIENT_FILES} ${MR_PYTHON_CLIENT_WHL_VERSION}
Upload Python Client Files In The Workbench ${MR_PYTHON_CLIENT_FILES}
${handle}= Launch And Access Workbench workbench_title=${WORKBENCH_TITLE}
... username=${TEST_USER.USERNAME} password=${TEST_USER.PASSWORD}
... auth_type=${TEST_USER.AUTH_TYPE}
Upload Certificate To Jupyter Notebook ${CERTS_DIRECTORY}/domain.crt
Upload Certificate To Jupyter Notebook openshift_ca.crt
Jupyter Notebook Can Query Model Registry ${JUPYTER_NOTEBOOK}
Expand All @@ -93,6 +92,7 @@
[Documentation] Suite setup steps for testing Model Registry.
Set Library Search Order SeleniumLibrary
RHOSi Setup
Enable Model Registry If Needed
Component Should Be Enabled modelregistry
Launch Dashboard ${TEST_USER.USERNAME} ${TEST_USER.PASSWORD} ${TEST_USER.AUTH_TYPE}
... ${ODH_DASHBOARD_URL} ${BROWSER.NAME} ${BROWSER.OPTIONS}
Expand Down Expand Up @@ -121,6 +121,7 @@
Remove Model Registry
Remove Deployment Files ${CERTS_DIRECTORY}
Remove Deployment Files ${MODELREGISTRY_BASE_FOLDER}/Python_Dependencies
Disable Model Registry If Needed
RHOSi Teardown

Get Cluster Domain And Token
Expand Down Expand Up @@ -210,8 +211,9 @@
Open With JupyterLab Menu Run Restart Kernel and Run All Cells…
Wait Until Page Contains Element xpath=//div[contains(text(),"Restart") and @class="jp-Dialog-buttonLabel"]
Click Element xpath=//div[contains(text(),"Restart") and @class="jp-Dialog-buttonLabel"]
# Somehow misses the check on the active cells without the sleep
Sleep 5s
Wait Until JupyterLab Code Cell Is Not Active timeout=120s
Sleep 2m msg=Waits until the jupyter notebook has completed execution of all cells
JupyterLab Code Cell Error Output Should Not Be Visible
SeleniumLibrary.Capture Page Screenshot

Expand Down Expand Up @@ -321,7 +323,14 @@

Open Model Registry Dashboard Page
[Documentation] Opens the Model Registry page from the dashboard nav bar
SeleniumLibrary.Wait Until Page Contains Model Registry
${mr_present}= Run Keyword And Return Status SeleniumLibrary.Page Should Contain Model Registry
WHILE ${mr_present}!=${TRUE} limit=120s
SeleniumLibrary.Reload Page
SeleniumLibrary.Wait Until Page Contains Home
Maybe Wait For Dashboard Loading Spinner Page
${mr_present}= Run Keyword And Return Status
... SeleniumLibrary.Page Should Contain Model Registry
END
SeleniumLibrary.Click Link Model Registry
Wait For RHODS Dashboard To Load wait_for_cards=${FALSE} expected_page=Model Registry
SeleniumLibrary.Wait Until Page Contains Select a model registry to view and manage your registered models.
Expand All @@ -347,3 +356,21 @@
${ns}= Get Substring ${ns} 1 -1
Log ${ns}
RETURN ${ns}

Enable Model Registry If Needed
[Documentation] While in tech preview the component will not be enabled by default. This keyword enables it.
${management_state}= Get DSC Component State default-dsc modelregistry ${OPERATOR_NAMESPACE}
IF "${management_state}" != "Managed"
Set Component State modelregistry Managed
Set Suite Variable ${DISABLE_COMPONENT} ${True}
${ns}= Get Model Registry Namespace From DSC
Wait For Namespace To Be Active ${ns} timeout=120s
END

Disable Model Registry If Needed
[Documentation] If we had to enable the component before the test run, let's also disable it at the end to leave
... the cluster in the same state we found it in
IF ${DISABLE_COMPONENT}==${True}
Set Component State modelregistry Removed
Run And Verify Command oc delete namespace ${NAMESPACE_MODEL_REGISTRY} --force
END
Loading