Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add test for ELYZA-japanese-Llama-2-7b-instruct #1253

Merged
merged 6 commits into from
Mar 7, 2024
Merged

Add test for ELYZA-japanese-Llama-2-7b-instruct #1253

merged 6 commits into from
Mar 7, 2024

Conversation

bdattoma
Copy link
Contributor

@bdattoma bdattoma commented Mar 5, 2024 •
edited
Loading

  1. adding the test case for Elyza llama model
  2. adding model_path variable to unbind the model path from model name (wasn't working for this model)
  3. making keyword to read json files and string reading data as "raw" - this model is returning "\n" in the responses (like other models as well) and this was breaking the code.

PR validation (including regression):
rhods-ci-pr-test/2561 - regression OK - the failures are automation bug not related to this PR
rhods-ci-pr-test/2565 PASS - only Elyza TC

@bdattoma bdattoma added needs testing Needs to be tested in Jenkins new test New test(s) added (PR will be listed in release-notes) labels Mar 5, 2024
@bdattoma bdattoma self-assigned this Mar 5, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Mar 5, 2024
View reviewed changes
...i/tests/Tests/400__ods_dashboard/420__model_serving/LLMs/422__model_serving_llm_models.robot
... namespace=${test_namespace}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${model_name}
... namespace=${test_namespace} timeout=900s
Run Keyword If "${KSERVE_MODE}"=="RawDeployment"

Check warning

Code scanning / Robocop

'{{ statement_name }}' is deprecated since Robot Framework version {{ version }}, use '{{ alternative }}' instead Warning test

'Run Keyword If' is deprecated since Robot Framework version 5.*, use 'IF' instead
...i/tests/Tests/400__ods_dashboard/420__model_serving/LLMs/422__model_serving_llm_models.robot
@@ -53,25 +53,60 @@
... AND
... Run Keyword If "${KSERVE_MODE}"=="RawDeployment" Terminate Process llm-query-process kill=true

Verify User Can Serve And Query A elyza/elyza-japanese-llama-2-7b-instruct Model

Check warning

Code scanning / Robocop

Test case '{{ test_name }}' is too long ({{ test_length }}/{{ allowed_length }}) Warning test

Test case 'Verify User Can Serve And Query A elyza/elyza-japanese-llama-2-7b-instruct Model' is too long (59/20)
...i/tests/Tests/400__ods_dashboard/420__model_serving/LLMs/422__model_serving_llm_models.robot
Set Test Variable ${model_name}
${models_names}= Create List ${model_name}
Set Test Variable ${models_names}
Set Test Variable ${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR
ods_ci/tests/Resources/CLI/ModelServing/llm.resource
@@ -59,13 +59,13 @@
[Arguments] ${namespace} ${enable_metrics}=${FALSE} ${runtime}=caikit-tgis-runtime ${protocol}=grpc
... ${access_key_id}=${S3.AWS_ACCESS_KEY_ID} ${access_key}=${S3.AWS_SECRET_ACCESS_KEY}
... ${endpoint}=${MODELS_BUCKET.ENDPOINT} ${verify_ssl}=${TRUE}
... ${download_in_pvc}=${FALSE} ${storage_size}=70Gi ${model_name}=${NONE} ${download_timeout}=600s
... ${download_in_pvc}=${FALSE} ${storage_size}=70Gi ${model_name}=${NONE} ${model_path}=${NONE} ${download_timeout}=600s

Check warning

Code scanning / Robocop

Line is too long ({{ line_length }}/{{ allowed_length }}) Warning test

Line is too long (148/120)
ods_ci/tests/Resources/CLI/ModelServing/llm.resource
@@ -59,13 +59,13 @@
[Arguments] ${namespace} ${enable_metrics}=${FALSE} ${runtime}=caikit-tgis-runtime ${protocol}=grpc
... ${access_key_id}=${S3.AWS_ACCESS_KEY_ID} ${access_key}=${S3.AWS_SECRET_ACCESS_KEY}
... ${endpoint}=${MODELS_BUCKET.ENDPOINT} ${verify_ssl}=${TRUE}
... ${download_in_pvc}=${FALSE} ${storage_size}=70Gi ${model_name}=${NONE} ${download_timeout}=600s
... ${download_in_pvc}=${FALSE} ${storage_size}=70Gi ${model_name}=${NONE} ${model_path}=${NONE} ${download_timeout}=600s

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (5 / 1)
ods_ci/tests/Resources/CLI/ModelServing/llm.resource
@@ -640,6 +640,7 @@
Set Test Variable ${use_https}
Set Test Variable ${namespace}
Set Test Variable ${storage_size}
Set Test Variable ${model_path}

Check notice

Code scanning / Robocop

{{ set_variable_keyword }} can be replaced with VAR Note test

Set Test Variable can be replaced with VAR
ods_ci/tests/Resources/CLI/ModelServing/llm.resource
@@ -640,6 +640,7 @@
Set Test Variable ${use_https}
Set Test Variable ${namespace}
Set Test Variable ${storage_size}
Set Test Variable ${model_path}

Check warning

Code scanning / Robocop

Test, suite and global variables should be uppercase Warning test

Test, suite and global variables should be uppercase
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Mar 5, 2024 •
edited
Loading

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
442 0 0 442 100

mwaykole
mwaykole previously approved these changes Mar 6, 2024
View reviewed changes
@bdattoma bdattoma requested a review from mwaykole March 6, 2024 13:36
@bdattoma bdattoma added verified This PR has been tested with Jenkins and removed needs testing Needs to be tested in Jenkins labels Mar 6, 2024
github-advanced-security[bot]
github-advanced-security bot found potential problems Mar 6, 2024
View reviewed changes
...i/tests/Tests/400__ods_dashboard/420__model_serving/LLMs/422__model_serving_llm_models.robot
... namespace=${test_namespace}
Wait For Pods To Be Ready label_selector=serving.kserve.io/inferenceservice=${model_name}
... namespace=${test_namespace} timeout=900s
Run Keyword If "${KSERVE_MODE}"=="RawDeployment"

Check warning

Code scanning / Robocop

'{{ statement_name }}' is deprecated since Robot Framework version {{ version }}, use '{{ alternative }}' instead Warning test

'Run Keyword If' is deprecated since Robot Framework version 5.*, use 'IF' instead
ods_ci/tests/Resources/CLI/ModelServing/llm.resource
@@ -629,7 +629,7 @@
[Arguments] ${model_name} ${bucket_name} ${endpoint}
... ${region} ${access_key_id} ${access_key}
... ${use_https} ${namespace} ${storage_size}
... ${download_timeout}=500s
... ${model_path} ${download_timeout}=500s

Check notice

Code scanning / Robocop

There is too many arguments per continuation line ({{ arguments_count }} / {{ max_arguments_count }}) Note test

There is too many arguments per continuation line (2 / 1)
lugi0
lugi0 previously approved these changes Mar 6, 2024
View reviewed changes
@bdattoma bdattoma requested a review from lugi0 March 6, 2024 15:40
@sonarqubecloud SonarQubeCloud
Copy link

sonarqubecloud bot commented Mar 6, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud

@tarukumar tarukumar merged commit 6c5050b into red-hat-data-services:master Mar 7, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mwaykole mwaykole mwaykole approved these changes

@lugi0 lugi0 lugi0 approved these changes

@tarukumar tarukumar tarukumar approved these changes

@jstourac jstourac Awaiting requested review from jstourac

Assignees

@bdattoma bdattoma

Labels
new test New test(s) added (PR will be listed in release-notes) verified This PR has been tested with Jenkins
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@bdattoma @mwaykole @lugi0 @tarukumar