Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decouple ObjectStore from Reqwest / Generic HTTP Client Support #6056

Closed
Tracked by #6818
tustvold opened this issue Jul 15, 2024 · 3 comments · Fixed by #7183
Closed
Tracked by #6818

Decouple ObjectStore from Reqwest / Generic HTTP Client Support #6056

tustvold opened this issue Jul 15, 2024 · 3 comments · Fixed by #7183
Assignees
Labels
enhancement Any new improvement worthy of a entry in the changelog

Comments

@tustvold
Copy link
Contributor

This is a high-level tracking ticket and is not really actionable

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

We currently depend extensively on reqwest for our HTTP client. This has served us very well, however, there has been some community interest in being able to either use different HTTP clients (#6051), or alter the behaviour of the client in non-trivial ways (#5990).

Describe the solution you'd like

Users provide a custom client factory that takes a ClientOptions and returns some generic HTTP Client abstraction. There are a lot of unknowns about how this would work, in particular for request retries, streaming requests, or more advanced configurations around TLS, proxies, etc... but this would all need to be worked out

Describe alternatives you've considered

This could be considered out of scope and too much scope creep for this crate, users with more advanced requirements should instead provide their own implementation of the ObjectStore trait.

Additional context

@tustvold tustvold added the enhancement Any new improvement worthy of a entry in the changelog label Jul 15, 2024
@tustvold tustvold changed the title Decouple HTTP Client Decouple ObjectStore from Reqwest / Generic HTTP Client Support Jul 15, 2024
@alamb
Copy link
Contributor

alamb commented Jul 15, 2024

Another alternative might be to help improve reqwest upstream so that it doesn't require tokio

@tustvold
Copy link
Contributor Author

The maintainers have at least previously indicated they do not wish to support runtimes other than tokio, I probably should have included this in the additional context - seanmonstar/reqwest#719

@tustvold
Copy link
Contributor Author

It occurs to me that this might provide a reasonable injection mechanism for dispatching tokio work to another thread pool, e.g. as required by apache/datafusion#14286

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Any new improvement worthy of a entry in the changelog
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants