Experimental arm64 support for Cassandra 3.11.6 #36

johntrimble · 2020-07-28T08:39:45Z

Turns the Cassandra 3.11.6 w/ Management API built into a multiarch build supporting both arm64 and amd64. Currently, disables metrics collector on arm64 as our collectd fork does not presently support arm64.

Closes #35

…etty" This reverts commit 8039c7c.

johntrimble · 2020-07-28T08:46:09Z

Dockerfile-oss

@@ -0,0 +1,115 @@
+FROM --platform=$BUILDPLATFORM maven:3.6.3-jdk-8-slim as builder


When using buildx, there isn't a way to make a build of an image in one invocation of buildx available to a subsequent invocation of buildx without the use of a registry. This means our traditional approach of having a 'build' docker image that we then reference in subsequent docker files will not work (unless we push it to a registry). To get around this, we use one multi-stage docker file that contains the build stages. A further discussion of this issue in buildx can be found here: docker/buildx#156

Also, note the --platform=$BUILDPLATFORM. This is because we don't have to build the jar for each arch we support since its java. This indicates that this stage should only be built on whatever the build platform is (probably linux-amd64).

johntrimble · 2020-07-28T08:48:54Z

Dockerfile-oss

+# Netty arm64 epoll support was not added until 4.1.50 (https://github.com/netty/netty/pull/9804)
+# Only replace this dependency for arm64 to avoid regressions
+RUN rm /opt/cassandra/lib/netty-all-*.jar
+COPY --from=netty4150 /root/.m2/repository/io/netty/netty-all/4.1.50.Final/netty-all-4.1.50.Final.jar /opt/cassandra/lib/netty-all-4.1.50.Final.jar


The better way to do this is probably to have the management api agent contain a shaded version of netty at the appropriate version. That will take further experimentation. However, we are only swapping out this jar for arm64, so this should not impact any existing users.

johntrimble · 2020-07-28T08:51:41Z

Dockerfile-oss

+COPY --from=builder /build/management-api-shim-4.x/target/datastax-mgmtapi-shim-4.x-0.1.0-SNAPSHOT.jar /opt/mgmtapi/
+
+ENV TINI_VERSION v0.18.0
+ADD https://github.com/krallin/tini/releases/download/${TINI_VERSION}/tini-${TARGETARCH} /tini


Note the change here to get tini for the appropriate architecture.

johntrimble · 2020-07-28T08:53:06Z

Dockerfile-oss

+# NOTE: Presently, OSS doesn't have official Cassandra 4.0 builds on dockerhub
+# and our build at datastax/cassandra:4.0 is not a multiarch image like the
+# official ones. Once one of those issues is fixed, the following targets can
+# be used to build Cassandra 4.0 with Management API.


Half way through this I realized we didn't have an appropriate multiarch cassandra 4.0 image. We could just remove all of this for now instead of commenting out. I thought it might be useful for reference, but I have no strong feelings either way.

johntrimble · 2020-07-28T08:53:33Z

management-api-common/pom.xml

@@ -14,7 +14,7 @@
    <properties>
        <slf4j.version>1.7.25</slf4j.version>
        <logback.version>1.2.3</logback.version>
-        <netty.version>4.1.45.Final</netty.version>
+        <netty.version>4.1.50.Final</netty.version>


This one provides a build of epoll for arm64.

johntrimble · 2020-07-28T08:55:07Z

scripts/docker-entrypoint.sh

+_metrics_collector_supported() {
+	# currently, metrics collector does not work on arm64
+	[ "$(uname -m)" != "aarch64" ]
+}


I don't know the best way to handle this. Somehow we need to disable the metrics collector for arm64. This appears to be working, but I'm open to suggestions.

tjake

Could you push the branch to this repo so the CI can run and open a new PR?

tjake

Seems like it running now.

…-multiarch-pr

johntrimble · 2020-08-03T22:46:07Z

@tjake yeah, I created the branch in the datastax repo, and since both branches pointed to the same commit, the jobs just started running. Tests are passing now, but I'm putting this back into draft briefly to test something.

This reverts commit fd5139b.

This reverts commit 1a40ff1.

johntrimble · 2020-08-04T04:14:51Z

.github/workflows/ci.yaml

+            --tag mgmtapi-3_11 \
+            --file Dockerfile-oss \
+            --target oss311 \
+            --platform linux/amd64 .


Only building here for "linux/amd64" as a multiarch image cannot be loaded into the local docker images. This should be good enough for running the integration tests.

johntrimble · 2020-08-04T08:05:05Z

@tjake The ci tests are passing now, but that's a bit deceptive. I just noticed that we use a docker-java library to build the image in our integration tests:

https://github.com/datastax/management-api-for-apache-cassandra/blob/cassandra-mgmtapi-multiarch-pr/management-api-server/src/test/java/com/datastax/mgmtapi/helpers/DockerHelper.java#L203-L208

Currently, this is using the Dockerfile-3_11 docker file. However, I updated the build to use Dockerfile-oss to create the multiarch images:

https://github.com/datastax/management-api-for-apache-cassandra/blob/cassandra-mgmtapi-multiarch-pr/.github/workflows/docker-release.yaml#L62-L69

This dockerfile requires the use of buildkit, which one can use by either using buildx (as is done to generate the multiarch images) or by setting the environment variable DOCKER_BUILDKIT=1 when calling docker build. Unfortunately, this docker-java library has no support for buildx and doesn't seem to have any intention of supporting buildkit: docker-java/docker-java#1381

Is there any reason why we cannot just invoke the docker executable to do the build instead of using this library?

tjake · 2020-08-05T15:09:40Z

Is there any reason why we cannot just invoke the docker executable to do the build instead of using this library?

That should be fine, can you do that?

johntrimble · 2020-08-05T16:55:10Z

That should be fine, can you do that?

Yes, I'll take care of it. Just wanted to make sure it was palatable first. Thanks for the feedback!

This reverts commit e64005d.

johntrimble · 2020-08-11T18:08:18Z

@tjake Okay, integration tests are now passing! My Java is a little rusty, so let me know if I should be doing anything differently there.

…-multiarch-pr

tjake

Awesome job! Thanks for this

johntrimble added 8 commits July 26, 2020 11:39

Update netty for epoll arm64 support

f04398d

Disable metrics monitor for arm64 since we have no binary for it

3c50ce3

Update build for testing multiarch

84f12ca

Fix workflow typo

9ca101b

Update branch for testing

7332499

Try only adding aarch64 epoll binary instead of replace all netty

8039c7c

Revert "Try only adding aarch64 epoll binary instead of replace all n…

6bef7da

…etty" This reverts commit 8039c7c.

Fix release workflow for merging

3d002fa

johntrimble commented Jul 28, 2020

View reviewed changes

johntrimble marked this pull request as ready for review July 28, 2020 08:57

johntrimble requested a review from tjake July 28, 2020 08:57

tjake suggested changes Aug 3, 2020

View reviewed changes

tjake reviewed Aug 3, 2020

View reviewed changes

Merge remote-tracking branch 'datastax/master' into cassandra-mgmtapi…

e66fa77

…-multiarch-pr

johntrimble marked this pull request as draft August 3, 2020 22:46

johntrimble added 3 commits August 3, 2020 16:21

Update ci to build 3.11 same as for release

fd5139b

Revert "Update ci to build 3.11 same as for release"

1a40ff1

This reverts commit fd5139b.

Revert "Revert "Update ci to build 3.11 same as for release""

893ef42

This reverts commit 1a40ff1.

johntrimble commented Aug 4, 2020

View reviewed changes

Update ci to build for amd64 and arm64

8ea8cf4

johntrimble marked this pull request as ready for review August 4, 2020 07:44

johntrimble added 3 commits August 11, 2020 00:50

Add use of buildx in integration test

c9167aa

Temporarily only test 3.11

e64005d

Revert "Temporarily only test 3.11"

7422adf

This reverts commit e64005d.

johntrimble added 2 commits August 22, 2020 15:38

Merge remote-tracking branch 'datastax/master' into cassandra-mgmtapi…

abdea73

…-multiarch-pr

Update to use 3.11.7 explicitly

8177539

tjake approved these changes Aug 26, 2020

View reviewed changes

tjake merged commit 5256425 into k8ssandra:master Aug 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Experimental arm64 support for Cassandra 3.11.6 #36

Experimental arm64 support for Cassandra 3.11.6 #36

johntrimble commented Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

johntrimble Jul 28, 2020

tjake left a comment

tjake left a comment

johntrimble commented Aug 3, 2020

johntrimble Aug 4, 2020

johntrimble commented Aug 4, 2020

tjake commented Aug 5, 2020

johntrimble commented Aug 5, 2020

johntrimble commented Aug 11, 2020

tjake left a comment

		@@ -0,0 +1,115 @@
		FROM --platform=$BUILDPLATFORM maven:3.6.3-jdk-8-slim as builder

Experimental arm64 support for Cassandra 3.11.6 #36

Experimental arm64 support for Cassandra 3.11.6 #36

Conversation

johntrimble commented Jul 28, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tjake left a comment

Choose a reason for hiding this comment

tjake left a comment

Choose a reason for hiding this comment

johntrimble commented Aug 3, 2020

Choose a reason for hiding this comment

johntrimble commented Aug 4, 2020

tjake commented Aug 5, 2020

johntrimble commented Aug 5, 2020

johntrimble commented Aug 11, 2020

tjake left a comment

Choose a reason for hiding this comment