Docker Image Change #1207

BinamB · 2024-12-13T16:27:39Z

Related PR: https://github.com/uc-cdis/cloud-automation/pull/2682/files

Without the cloud-auto PR, we're not able to successfully deploy fence.

Improvements

Update to use new Amazon Linux base image and use the same structure as our other python services.
Utilizing "gen3" user instead of "root" for more secure containers
Moving to Poetry to manage our virtual environments
Multi-stage Docker builds for smaller images
Move to Gunicorn

github-actions · 2024-12-13T17:24:04Z

Please find the ci env pod logs here

github-actions · 2024-12-15T08:03:04Z

Please find the ci env pod logs here

github-actions · 2024-12-15T14:49:02Z

Please find the ci env pod logs here

nss10

The changes look good. Just a couple of questions, and will approve.

nss10 · 2024-12-16T15:38:26Z

Dockerfile

+        libxcrypt-compat-4.4.33 \
+        libpq-15.0 && \
+    echo "Installing RPM"; \
+    rpm -i https://ccrypt.sourceforge.net/download/1.11/ccrypt-1.11-1.x86_64.rpm; \


I noticed we are using an x86 binary for ARM. Will this be emulated for ARM during use? Has any testing been done to confirm its functionality?

nss10 · 2024-12-16T15:40:56Z

pyproject.toml

@@ -22,7 +22,7 @@ cached_property = "^1.5.1"
 cdiserrors = "<2.0.0"
 cdislogging = "^1.0.0"
 cdispyutils = "^2.0.1"
-flask = ">=3.0.0"
+flask = "^2.2.3"


Could you clarify the reason for downgrading Flask?

github-actions · 2025-02-12T19:03:18Z

filepath	$$\textcolor{#f14c4c}{\tt{failed}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-14T15:49:56Z

Please find the ci env pod logs here

github-actions · 2025-02-14T23:16:57Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#ffa500}{\tt{skipped}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{15}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_centralized\_auth.py}}$$	$$\textcolor{#23d18b}{\tt{16}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{16}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_data\_upload.py}}$$	$$\textcolor{#23d18b}{\tt{8}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_presigned\_url.py}}$$	$$\textcolor{#23d18b}{\tt{7}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{7}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_dbgap.py}}$$	$$\textcolor{#23d18b}{\tt{4}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_drs\_endpoint.py}}$$	$$\textcolor{#23d18b}{\tt{4}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oidc\_client.py}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{2}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_google\_data\_access.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_audit\_service.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_client\_credentials.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{2}}$$	$$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{65}}$$	$$\textcolor{#f14c4c}{\tt{2}}$$	$$\textcolor{#ffa500}{\tt{4}}$$	$$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

k-burt-uch · 2025-02-17T19:30:24Z

I manually verified the RAS AuthN test cases that are failing as follows:

Updated qa-dcp to use this PR's branch
Changed fence-config secret to use the RAS BDC Staging New clientId and clientSecret for the ras OPENID_CONNECT (In keeper)
gen3 kube-setup-fence
Verified I can log in
Changed fence-config secret to remove the ga4gh_passport from scope (this simulates the second failing RAS AuthN test case)
gen3 kube-setup-fence
Verified I can log in.

I've reverted the changes on qa-dcp.

github-actions · 2025-02-18T16:57:59Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#ffa500}{\tt{skipped}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_centralized\_auth.py}}$$	$$\textcolor{#23d18b}{\tt{16}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{16}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#f14c4c}{\tt{10}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_data\_upload.py}}$$	$$\textcolor{#23d18b}{\tt{8}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{9}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_presigned\_url.py}}$$	$$\textcolor{#23d18b}{\tt{7}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{7}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_user\_token.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_drs\_endpoint.py}}$$	$$\textcolor{#23d18b}{\tt{4}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{4}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_oidc\_client.py}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{2}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_audit\_service.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_client\_credentials.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_google\_data\_access.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#ffa500}{\tt{tests/test\_register\_user.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{2}}$$	$$\textcolor{#ffa500}{\tt{2}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{52}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$	$$\textcolor{#ffa500}{\tt{4}}$$	$$\textcolor{#f14c4c}{\tt{71}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-18T20:00:06Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#ffa500}{\tt{skipped}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#f14c4c}{\tt{10}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_google\_data\_access.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{10}}$$	$$\textcolor{#f14c4c}{\tt{13}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{24}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-18T22:53:52Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#ffa500}{\tt{skipped}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_dbgap.py}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{5}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#f14c4c}{\tt{10}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#23d18b}{\tt{2}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{3}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_google\_data\_access.py}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{10}}$$	$$\textcolor{#f14c4c}{\tt{13}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{24}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-19T15:22:29Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	$$\textcolor{#ffa500}{\tt{skipped}}$$	SUBTOTAL
$$\textcolor{#ffa500}{\tt{tests/test\_dbgap.py}}$$	$$\textcolor{#23d18b}{\tt{4}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{5}}$$
$$\textcolor{#f14c4c}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{14}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_ras\_authn.py}}$$	$$\textcolor{#23d18b}{\tt{3}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{3}}$$
$$\textcolor{#23d18b}{\tt{tests/test\_google\_data\_access.py}}$$	$$\textcolor{#23d18b}{\tt{1}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#666666}{\tt{0}}$$	$$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{22}}$$	$$\textcolor{#f14c4c}{\tt{1}}$$	$$\textcolor{#ffa500}{\tt{1}}$$	$$\textcolor{#f14c4c}{\tt{24}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-19T16:54:40Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	$$\textcolor{#f14c4c}{\tt{failed}}$$	SUBTOTAL
$$\textcolor{#f14c4c}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#f14c4c}{\tt{10}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$
$$\textcolor{#f14c4c}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{5}}$$	$$\textcolor{#f14c4c}{\tt{10}}$$	$$\textcolor{#f14c4c}{\tt{15}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

github-actions · 2025-02-19T17:38:27Z

filepath	$$\textcolor{#23d18b}{\tt{passed}}$$	SUBTOTAL
$$\textcolor{#23d18b}{\tt{tests/test\_oauth2.py}}$$	$$\textcolor{#23d18b}{\tt{15}}$$	$$\textcolor{#23d18b}{\tt{15}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$	$$\textcolor{#23d18b}{\tt{15}}$$	$$\textcolor{#23d18b}{\tt{15}}$$

Please find the detailed integration test report here

Please find the ci env pod logs here

* Update to use new Amazon Linux base image and use the same structure as our other python services. * Utilizing "gen3" user instead of "root" for more secure containers * Moving to Poetry to manage our virtual environments * Multi-stage Docker builds for smaller images * Move to Gunicorn --------- Co-authored-by: Jawad Qureshi <qureshi@uchicago.edu> Co-authored-by: EliseCastle23 <109446148+EliseCastle23@users.noreply.github.com> Co-authored-by: Alexander VanTol <avantol@uchicago.edu> Co-authored-by: Alexander VanTol <Avantol13@users.noreply.github.com> Co-authored-by: Sai Shanmukha <nss10@outlook.com> Co-authored-by: J. Q. <55899496+jawadqur@users.noreply.github.com>

jawadqur and others added 23 commits September 4, 2024 15:59

Add new AL2 based image and gunicorn

16aa5b4

Update poetry

84e63b9

updating the command

9883f05

Merge branch 'master' into feat/al2

a243631

updating poetry lock

0238816

fixing link

016636e

fixing deadlinks

0fe5859

fixing case

6708a84

Merge branch 'master' into feat/al2

c514982

Merge branch 'master' into feat/al2

9fc2412

feat(docker): update to use new base image

2031c74

feat(mcrypt): add wip work to have 2 images

cc0e908

Merge branch 'master' into feat/al2

351d6e5

WIP

c82765b

Update ci.yaml

a179a04

Changing the basic dockerfile

2502803

Merge branch 'master' into chore/ccrypt_usersync

43a98f4

update for arm

f198e66

Fix downloads

f8b848b

add y

fab15db

fix docker

5937738

single image

2bb0b1e

fix image

31e50ae

BinamB added 2 commits December 15, 2024 00:01

add poetry run to gunicorn

8e664c0

update poetry lock

d448672

update dockerfile

63217d7

nss10 reviewed Dec 16, 2024

View reviewed changes

nss10 removed the TestDbgap label Feb 12, 2025

nss10 removed the TestRasAuthN label Feb 14, 2025

Avantol13 approved these changes Feb 14, 2025

View reviewed changes

Merge branch 'master' into chore/ccrypt_usersync

4780e10

PlanXCyborg added TestDbgap TestGoogleDataAccess TestOauth2 TestRasAuthN labels Feb 18, 2025

PlanXCyborg added TestGoogleDataAccess TestRasAuthN TestDbgap and removed TestGoogleDataAccess TestDbgap TestRasAuthN labels Feb 18, 2025

PlanXCyborg removed TestGoogleDataAccess TestDbgap TestRasAuthN labels Feb 19, 2025

nss10 merged commit b2fa0ae into master Feb 19, 2025
9 checks passed

nss10 deleted the chore/ccrypt_usersync branch February 19, 2025 18:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker Image Change #1207

Docker Image Change #1207

BinamB commented Dec 13, 2024 •

edited

Loading

github-actions bot commented Dec 13, 2024

github-actions bot commented Dec 15, 2024

github-actions bot commented Dec 15, 2024

nss10 left a comment

nss10 Dec 16, 2024

nss10 Dec 16, 2024

github-actions bot commented Feb 12, 2025

github-actions bot commented Feb 14, 2025

github-actions bot commented Feb 14, 2025

k-burt-uch commented Feb 17, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 19, 2025

github-actions bot commented Feb 19, 2025

github-actions bot commented Feb 19, 2025

Docker Image Change #1207

Docker Image Change #1207

Conversation

BinamB commented Dec 13, 2024 • edited Loading

Improvements

github-actions bot commented Dec 13, 2024

github-actions bot commented Dec 15, 2024

github-actions bot commented Dec 15, 2024

nss10 left a comment

Choose a reason for hiding this comment

nss10 Dec 16, 2024

Choose a reason for hiding this comment

nss10 Dec 16, 2024

Choose a reason for hiding this comment

github-actions bot commented Feb 12, 2025

github-actions bot commented Feb 14, 2025

github-actions bot commented Feb 14, 2025

k-burt-uch commented Feb 17, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 18, 2025

github-actions bot commented Feb 19, 2025

github-actions bot commented Feb 19, 2025

github-actions bot commented Feb 19, 2025

BinamB commented Dec 13, 2024 •

edited

Loading