🪟 🎉 Clear user-defined primary key(s) and cursor field(s) if the field is removed

[teallarson]

What

closes #19387

Clears the primary key and/or cursor properties on the stream.config object if any of the fields referenced there are removed during a refresh.

How

• useInitialValues (in formConfig) identifies if there is a breaking change on the connection. If there is, it will traverse the diff and pull out the transforms for the field(s) that were removed
• the transforms get sent along to calculateInitialValues which is already used to calculate and override config fields
• as each stream is evaluated, if it is included in the list of streams with broken changes, a cleanup function is called that will find a match on either a primary key (or a partial of a compound primary key) or cursor field. If a match is found for the removed field, we clear that property on the config object.

Recommended reading order

1. calculateInitialCatalog.test.tsx (for expected behaviors)
2. formConfig.tsx
3. calculateInitialCatalog.ts

Testing

This is not behind the feature flag, but should not affect anything negatively. Open to discussion if we'd rather keep it behind the flag. Either way, useful to test both with REACT_APP_AUTO_DETECT_SCHEMA_CHANGES as true and as false

At bare minimum:

• removing fields for user-configured primary keys and cursors
• adding and removing fields for source-defined primary keys and cursors

These are also covered by unit tests.

Next steps

These files are due for a refactor.

[teallarson]

it's possible that lodash intersection could tidy this up, but i haven't figured out how

[teallarson]

no changes to this logic, just cleaning up some naming here for clarity

[teallarson]

I don't love that this check happens on readonly connections. Presumably readonly connections wouldn't ever have a breaking change so should return immediately when clearBreakingFieldChanges() is called. It's not the only place we're doing something like this. I have proposed some cleanup in #20233.

[edmundito]

Discussed this with @tealjulia over Slack. We agree that it could be improved but it would require a lot of work outside of the scope of solving the current problem.

[edmundito]

Discussed this with @tealjulia over Slack. We agree that it could be improved but it would require a lot of work outside of the scope of solving the current problem.