-
Notifications
You must be signed in to change notification settings - Fork 454
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: expand ListingSchemaProvider to support register and deregister table #3150
base: main
Are you sure you want to change the base?
feat: expand ListingSchemaProvider to support register and deregister table #3150
Conversation
Open for discussion on this one. Have had this locally for a long time and wanted to share the branch. I use this quite a lot in my project and others may find it useful to their project. |
@hntd187 can you take a look? :) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm supportive of this change, but I would like some changes to the internals.
…n register, and added a load_table function Signed-off-by: Alexander Falk <alexfalk7@gmail.com>
…able Signed-off-by: Alexander Falk <alexfalk7@gmail.com>
Description
The
ListingSchemaProvider
was missing the implementations of the register and deregister traits from theSchemaProvider
. These are now implemented, and to meet requirements from remote object stores, theDeltaTableBuilder
is now used to pass along the storage options.The
tables
property has been modified to instead of havingDashMap<String, String>
, it is now usingMutex<DashMap<String, Arc<dyn TableProvider>>>
to supportdowncast_ref
and mutability elsewhere.Inspiration from: Datafusion ListingSchemaProvider
Related Issue(s)
There is no related issues. I have a clone of this with extra modifications in my own project. I find it useful to have when adding a new schema and wanted to share this with others 👍
Documentation
None