Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sync jackson versions #60

Merged
merged 2 commits into from
Apr 28, 2023
Merged

sync jackson versions #60

merged 2 commits into from
Apr 28, 2023

Conversation

luster
Copy link
Contributor

@luster luster commented Apr 27, 2023

there's an incompatibility within scala module and jackson-databind causing a runtime error

on a fresh install of gcs-proto-tools,

$ proto-tools tojson file.protobuf.avro

Exception in thread "main" java.lang.ExceptionInInitializerError
	at me.lyh.protobuf.generic.Schema$.fromJson(Schema.scala:43)
	at org.apache.avro.tool.ProtobufReader.<init>(ProtobufReader.scala:8)
	at org.apache.avro.tool.ProtoToJsonTool.run(ProtoToJsonTool.java:61)
	at org.apache.avro.tool.ProtoMain.run(ProtoMain.java:64)
	at org.apache.avro.tool.ProtoMain.main(ProtoMain.java:53)
Caused by: com.fasterxml.jackson.databind.JsonMappingException: Scala module 2.11.2 requires Jackson Databind version >= 2.11.0 and < 2.12.0
	at com.fasterxml.jackson.module.scala.JacksonModule.setupModule(JacksonModule.scala:61)
	at com.fasterxml.jackson.module.scala.JacksonModule.setupModule$(JacksonModule.scala:46)
	at com.fasterxml.jackson.module.scala.DefaultScalaModule.setupModule(DefaultScalaModule.scala:17)
	at com.fasterxml.jackson.databind.ObjectMapper.registerModule(ObjectMapper.java:835)
	at me.lyh.protobuf.generic.SchemaMapper$.<clinit>(Schema.scala:127)
	... 5 more

on main branch, i get the same runtime error when running sbt assembly and invoking directly from the jar, as below.

when making the proposed change to build.sbt locally, i was able to run the following successfully:

$ sbt clean assembly
$ java -jar proto-tools/target/scala-2.13/proto-tools-3.22.2.jar tojson file.protobuf.avro

while investigating these changes, i noticed that there are some wider incompatibilities between scala versions across the various dependencies. i'm not a scala/sbt expert, but it seems strange that we declare a scala module version but then revert down to a lower version because of a dependency we pull in. not sure what the longer term change would be in managing this project, but just wanted to call out

there's an incompatibility within scala module and jackson-databind causing a runtime error
build.sbt Outdated
@@ -92,7 +92,8 @@ lazy val protoTools = project
"org.apache.hadoop" % "hadoop-common" % hadoopVersion,
"org.apache.hadoop" % "hadoop-client" % hadoopVersion,
"com.google.cloud.bigdataoss" % "gcs-connector" % gcsVersion
)
),
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.11.2"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of pinning jackson-databind on the jackson-module-scala, we should make sure all jackson libs are in sync (and on the latest if possible)

@RustedBones RustedBones changed the title pin jackson-databind version to 2.11.2 sync jackson versions Apr 28, 2023
@RustedBones RustedBones merged commit df3ef61 into spotify:main Apr 28, 2023
@luster luster deleted the patch-1 branch May 4, 2023 00:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants