Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add test for deploying and querying a prompt-tuned google/flan-t5-xl LLM #1272

Merged
merged 10 commits into from
Mar 12, 2024

Conversation

bdattoma
Copy link
Contributor

@bdattoma bdattoma commented Mar 8, 2024

Besides adding the test for the model the PR introduces

  • the usage of kustomize to compose the final InferenceService YAML. This could be extended in the future to other variants of the YAML (e.g., requests/limits)
  • the possibility to add parameters in the request body, e.g., max_new_tokens

@bdattoma bdattoma requested review from lugi0, mwaykole and tarukumar March 8, 2024 18:16
@bdattoma bdattoma self-assigned this Mar 8, 2024
@bdattoma bdattoma added needs testing Needs to be tested in Jenkins new test New test(s) added (PR will be listed in release-notes) labels Mar 8, 2024
- mountPath: "/mnt/prompts/"
name: model-volume
containers:
- name: download-model-prompts

Check warning

Code scanning / SonarCloud

CPU limits should be enforced Medium test

Specify a CPU limit for this container. See more on SonarCloud
- mountPath: "/mnt/prompts/"
name: model-volume
containers:
- name: download-model-prompts

Check warning

Code scanning / SonarCloud

Memory limits should be enforced Medium test

Specify a memory limit for this container. See more on SonarCloud
name: ${model_name}-download-prompts
labels:
name: ${model_name}-download-prompts
spec:

Check warning

Code scanning / SonarCloud

Service account tokens should not be mounted in pods Medium test

Set automountServiceAccountToken to false for this specification of kind Pod. See more on SonarCloud
- mountPath: "/mnt/prompts/"
name: model-volume
containers:
- name: download-model-prompts

Check warning

Code scanning / SonarCloud

Storage limits should be enforced Medium test

Specify a storage limit for this container. See more on SonarCloud
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robocop found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

Copy link
Contributor

github-actions bot commented Mar 8, 2024

Robot Results

✅ Passed ❌ Failed ⏭️ Skipped Total Pass %
446 0 0 446 100

lugi0
lugi0 previously approved these changes Mar 11, 2024
@bdattoma
Copy link
Contributor Author

bdattoma commented Mar 11, 2024

validation: rhods-ci-pr-test/2590 1 check failed, mistakes in the model responses json file. Retesting prompt-tuned model only here rhods-ci-pr-test/2592/ PASS

@bdattoma bdattoma added verified This PR has been tested with Jenkins and removed needs testing Needs to be tested in Jenkins labels Mar 11, 2024
@bdattoma bdattoma requested a review from lugi0 March 11, 2024 11:18
Copy link

Quality Gate Failed Quality Gate failed

Failed conditions
C Security Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

@tarukumar tarukumar merged commit fbc5b4c into red-hat-data-services:master Mar 12, 2024
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new test New test(s) added (PR will be listed in release-notes) verified This PR has been tested with Jenkins
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants