Destination mysql: Switch to dv2 raw tables #37322
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎ 1 Ignored Deployment
|
octavia-squidington-iii
added
area/connectors
This was referenced Apr 15, 2024
This stack of pull requests is managed by Graphite. Learn more about stacking. |
edgao
commented
Apr 15, 2024
| @@ -132,23 +136,22 @@ protected List<JsonNode> retrieveRecords(final TestDestinationEnv testEnv, | |||
| } | |||
|
|
|||
| private List<JsonNode> retrieveRecordsFromTable(final String tableName, final String schemaName) throws SQLException { | |||
| try (final DSLContext dslContext = DSLContextFactory.create( | |||
| final DSLContext dslContext = DSLContextFactory.create( | |||
There was a problem hiding this comment.
I think jooq removed the autocloseable interface from DslContext?
There was a problem hiding this comment.
yeah had the same experience in Oracle
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
94a3dc0 to
dd57516
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
7a3147d to
b15916f
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
dd57516 to
97d1fa3
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
2 times, most recently
from
a8dc68e to
0a27153
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
97d1fa3 to
4e09b56
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
0a27153 to
81b0ca0
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
2 times, most recently
from
bd83d63 to
f6e353f
Compare
edgao
commented
Apr 16, 2024
| @@ -43,6 +43,7 @@ | |||
| import org.junit.jupiter.api.Timeout; | |||
| import org.testcontainers.containers.MySQLContainer; | |||
|
|
|||
| @Disabled | |||
There was a problem hiding this comment.
everything starts failing because we're writing to a different raw table. Disable the legacy DAT suite.
... https://github.com/airbytehq/airbyte/issues/29818 continues to exist
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
3 times, most recently
from
4255c6e to
f3ee677
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
41a159a to
ecffdd8
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
2 times, most recently
from
807fef1 to
19bb0fc
Compare
edgao
commented
Apr 22, 2024
| /* | ||
| * We want to generate a query like: | ||
| * | ||
| * LOAD DATA LOCAL INFILE '/a/b/c' INTO TABLE foo.bar FIELDS TERMINATED BY ',' ENCLOSED BY |
There was a problem hiding this comment.
.... thanks, autoformat
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
ecffdd8 to
307c476
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
19bb0fc to
60079a8
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
307c476 to
6f1f8b4
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
60079a8 to
4f77fe7
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
6f1f8b4 to
4be8c70
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
4f77fe7 to
b7f83d8
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
4be8c70 to
8e32610
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
2 times, most recently
from
bafc309 to
38c8d07
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
8e32610 to
1004c52
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
3 times, most recently
from
3b4f4fc to
33935e7
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
1004c52 to
7ae481a
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
33935e7 to
cf60f60
Compare
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
7ae481a to
2067931
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
cf60f60 to
2870be2
Compare
This was referenced Apr 26, 2024
edgao
force-pushed
the
dv2/td_tests_allow_unsafe_cast_2
branch
from
2067931 to
be524ad
Compare
edgao
force-pushed
the
dv2/mysql_raw_tables
branch
from
2870be2 to
f4cc766
Compare
|
closing this PR in favor of #36936 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Switch the raw tables to DV2 format. See #36936 for implementing T+D. This PR sets a few "disable T+D" booleans for the sake of being able to run tests; that PR unsets them.
I'll merge #36936 into this branch and release them as a single major version bump - not planning to publish a version of destination-mysql with just dv2 raw tables.
There's some weird stuff going on in MysqlNameTransformer, which I didn't try to solve here. I mostly just don't understand what all the methods are supposed to do (e.g. how does getIdentifier relate to convertStreamName?). MysqlNameTransformer behaves differently from PostgresNameTransformer, and I elected to work around that inconsistency. IMO we should probably revamp that class to be less opinionated about what tables we create.