Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented SchemaTransforms for SingleStoreIO #24290

Merged
merged 21 commits into from
Dec 19, 2022
Merged

Implemented SchemaTransforms for SingleStoreIO #24290

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e97f319
Implemented SchemaTransforms for SingleStoreIO
AdalbertMemSQL Nov 14, 2022
69a1d21
Fixed nullable errors
AdalbertMemSQL Nov 24, 2022
08b3132
Changed to don't use .* form of import
AdalbertMemSQL Nov 24, 2022
caf46da
Changed formatter field to be transient
AdalbertMemSQL Nov 25, 2022
302ceac
Nit reformatting
AdalbertMemSQL Nov 25, 2022
b5e9543
Fixed bugs in tests
AdalbertMemSQL Nov 25, 2022
4e2e7bc
Moved schema transform classes to the separate folder
AdalbertMemSQL Nov 28, 2022
d59828a
Removed unused imports
AdalbertMemSQL Nov 28, 2022
04432dd
Added package-info file
AdalbertMemSQL Nov 28, 2022
103e4a8
check point
AdalbertMemSQL Dec 6, 2022
e67f055
check point
AdalbertMemSQL Dec 6, 2022
7be63d7
Resolved comments
AdalbertMemSQL Dec 6, 2022
266ccc0
Changed identation
AdalbertMemSQL Dec 6, 2022
b6b6af0
Fixed build by adding a cast
AdalbertMemSQL Dec 14, 2022
3ffaa49
Reformatted code
AdalbertMemSQL Dec 14, 2022
faa2416
Added an assertion that convertLogicalTypeFieldToString is called onl…
AdalbertMemSQL Dec 14, 2022
bdc5329
Refactored code to delete ReadRows and ReadRowsWithPartitions classes
AdalbertMemSQL Dec 15, 2022
b9d253e
Update .test-infra/jenkins/job_PostCommit_Java_SingleStoreIO_IT.groovy
AdalbertMemSQL Dec 15, 2022
7c9b4f3
Fixed bug where env variable name was used instead of the value
AdalbertMemSQL Dec 15, 2022
891508b
Changed to use checkArgument instead of assert
AdalbertMemSQL Dec 15, 2022
98a25bd
Added appropriate error message
AdalbertMemSQL Dec 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .test-infra/jenkins/job_PostCommit_Java_SingleStoreIO_IT.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ PostcommitJobBuilder.postCommitJob(jobName,
tempRoot : 'gs://temp-storage-for-perf-tests',
project : 'apache-beam-testing',
runner : 'DataflowRunner',
singleStoreServerName : singlestoreHostName,
singleStoreServerName : "\$${singlestoreHostName}",
singleStoreUsername : "admin",
singleStorePassword : "secretpass",
singleStorePort: "3306",
Expand Down
4 changes: 3 additions & 1 deletion ...re/src/main/java/org/apache/beam/sdk/io/singlestore/SingleStoreDefaultUserDataMapper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@
*/
package org.apache.beam.sdk.io.singlestore;

import static org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkArgument;

import java.math.BigDecimal;
import java.nio.charset.StandardCharsets;
import java.util.ArrayList;
Expand All @@ -37,7 +39,7 @@ final class SingleStoreDefaultUserDataMapper implements SingleStoreIO.UserDataMa
DateTimeFormat.forPattern("yyyy-MM-DD' 'HH:mm:ss.SSS");

private String convertLogicalTypeFieldToString(Schema.FieldType type, Object value) {
assert type.getTypeName().isLogicalType();
checkArgument(type.getTypeName().isLogicalType(), "<appropriate error message>");
Copy link
Contributor

@ahmedabu98 ahmedabu98 Dec 19, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you replace it with an appropriate error message?

P.S.
Doing another read of this I realize this check isn't necessary heh, sorry for suggesting it earlier.


Schema.LogicalType<Object, Object> logicalType =
(Schema.LogicalType<Object, Object>) type.getLogicalType();
Expand Down