Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: retryable http client #398

Merged
merged 3 commits into from
Jan 17, 2023
Merged

feat!: retryable http client #398

merged 3 commits into from
Jan 17, 2023

Conversation

souleb
Copy link
Contributor

@souleb souleb commented Jan 11, 2023

fixes #147

If implemented, this will provide a default http client with retry.

The retry function is an exponential back off 0.25s * 2^n ± 10% and max 5 attempts.

The client is the default client of auth.Client

Co-authored-by: Shiwei Zhang shizh@microsoft.com
Signed-off-by: Soule BA bah.soule@gmail.com

cc @shizhMSFT

@souleb souleb marked this pull request as draft January 11, 2023 22:53
@shizhMSFT shizhMSFT added this to the v2.0.0 milestone Jan 12, 2023
@shizhMSFT shizhMSFT changed the title Retryable http client feat!: retryable http client Jan 12, 2023
Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work. Just left some initial review comments. I will continue reviewing later.

@souleb
Copy link
Contributor Author

souleb commented Jan 12, 2023

The examples tests are failing due to some TLS misconfiguration when I set the retry.DefaultClient as client for auth.Client. Can you tell me where to look?

@shizhMSFT
Copy link
Contributor

The examples tests are failing due to some TLS misconfiguration when I set the retry.DefaultClient as client for auth.Client. Can you tell me where to look?

@souleb The root cause of the error is that retry.Client relies on the default http transport. Therefore, we need to override the default http transport instead of the client for the mocks.

Please check the following changes:

diff --git a/example_test.go b/example_test.go
index c1e4c20..82fa126 100644
--- a/example_test.go
+++ b/example_test.go
@@ -182,7 +182,7 @@ func TestMain(m *testing.M) {
                panic(err)
        }
        remoteHost = u.Host
-       http.DefaultClient = httpsServer.Client()
+       http.DefaultTransport = httpsServer.Client().Transport
 
        os.Exit(m.Run())
 }
diff --git a/registry/example_test.go b/registry/example_test.go
index 70f102c..2463667 100644
--- a/registry/example_test.go
+++ b/registry/example_test.go
@@ -65,7 +65,7 @@ func TestMain(m *testing.M) {
                panic(err)
        }
        host = u.Host
-       http.DefaultClient = ts.Client()
+       http.DefaultTransport = ts.Client().Transport
 
        os.Exit(m.Run())
 }
diff --git a/registry/remote/example_test.go b/registry/remote/example_test.go
index bbd292e..370486d 100644
--- a/registry/remote/example_test.go
+++ b/registry/remote/example_test.go
@@ -214,7 +214,7 @@ func TestMain(m *testing.M) {
                panic(err)
        }
        host = u.Host
-       http.DefaultClient = ts.Client()
+       http.DefaultTransport = ts.Client().Transport
 
        os.Exit(m.Run())
 }

@souleb souleb marked this pull request as ready for review January 12, 2023 16:15
@codecov-commenter
Copy link

codecov-commenter commented Jan 13, 2023

Codecov Report

Merging #398 (afe215f) into main (76382aa) will increase coverage by 0.02%.
The diff coverage is 72.63%.

@@            Coverage Diff             @@
##             main     #398      +/-   ##
==========================================
+ Coverage   72.31%   72.34%   +0.02%     
==========================================
  Files          46       48       +2     
  Lines        4349     4444      +95     
==========================================
+ Hits         3145     3215      +70     
- Misses        903      921      +18     
- Partials      301      308       +7     
Impacted Files Coverage Δ
registry/remote/auth/client.go 72.46% <ø> (+0.40%) ⬆️
registry/remote/retry/client.go 66.66% <66.66%> (ø)
registry/remote/retry/policy.go 79.54% <79.54%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@souleb souleb force-pushed the pkg-retry branch 2 times, most recently from 5c7b9d7 to afe215f Compare January 13, 2023 13:56
@souleb souleb requested review from shizhMSFT and removed request for Wwwsylvia January 13, 2023 14:08
@shizhMSFT shizhMSFT requested a review from Wwwsylvia January 13, 2023 19:22
@souleb souleb force-pushed the pkg-retry branch 2 times, most recently from ae4410d to f9a9bc8 Compare January 16, 2023 23:19
souleb and others added 3 commits January 17, 2023 11:38
If implemented, this will provide a default http client with retry.

The retry function is an exponential back off 0.25s * 2^n ± 10% and max 5 attempts.

The client is the default client of `auth.Client`

Co-authored-by: Shiwei Zhang <shizh@microsoft.com>
Signed-off-by: Soule BA <bah.soule@gmail.com>
…ultClient

Signed-off-by: Soule BA <bah.soule@gmail.com>
Better backoff function that rely accounts for ± 10% jitter

Signed-off-by: Soule BA <bah.soule@gmail.com>
Copy link
Contributor

@shizhMSFT shizhMSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@Wwwsylvia Wwwsylvia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Wwwsylvia Wwwsylvia merged commit 5a2e692 into oras-project:main Jan 17, 2023
@souleb souleb deleted the pkg-retry branch January 17, 2023 12:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Support for retry?
4 participants