You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The existing S3 connector offers a couple of options for configuring connections for JSONL file types. To ensure backwards compatibility, we'll want to update the config adapter to handle them.
In #28131, we're creating a new S3 FileBasedConfig object. This ticket involves extending that object to handle JSONL-specific options, and will also require the creation of a custom parser that handles the old options.
Verify that all options that we still support are appropriately mapped to the name in the file-based CDK:
newlines_in_values
unexpected_field_behavior: Options provided to Pyarrow for handling types that aren't in the schema used by a handful of connectors. This should be mapped to the new schema validation policies.
Acceptance Criteria
The existing JSONL config options are mapped and handled appropriately by the file-based CDK.
Any options that we cannot support are identified, along with the connectors that will be impacted.
The text was updated successfully, but these errors were encountered:
The existing S3 connector offers a couple of options for configuring connections for JSONL file types. To ensure backwards compatibility, we'll want to update the config adapter to handle them.
In #28131, we're creating a new S3 FileBasedConfig object. This ticket involves extending that object to handle JSONL-specific options, and will also require the creation of a custom parser that handles the old options.
newlines_in_values
unexpected_field_behavior
: Options provided to Pyarrow for handling types that aren't in the schema used by a handful of connectors. This should be mapped to the new schema validation policies.Acceptance Criteria
The text was updated successfully, but these errors were encountered: