Adding support to build cassandra-reaper docker images #76

chrislbs · 2017-04-04T01:36:13Z

Resolves #75

Including environment variable substitution functionality for
application yml
Including spotify docker build dependency
Adding initial Dockerfile and supporting startup shell scripts

There are some additional changes I can do but I wanted to get some feedback first. Some ideas that I can include in this or future pull requests.

Including a docker-compose.yml example that demonstrates multiple configurations
Update documentation to include all possible environment variable tweaks

- Including environment variable substitution functionality for application yml - Including spotify docker build dependency - Adding initial Dockerfile and supporting startup shell scripts

chrislbs · 2017-04-04T02:01:49Z

I just realized that #46 is perhaps a superset of some of the work I did here. @joaquincasares @adejanovski Do you think it might be worth combining some of the work between the 2 PRs?

joaquincasares · 2017-04-04T03:01:11Z

Oh nice! Thanks for getting that plugin to work! I seriously couldn't get it for the life of me! :)

Yeah, merging the two would be great! My purpose was primarily around getting an environment together to simply build packages without mucking up my system.

It seems your intent is geared towards an actual Docker use case, which is what was missing from my PR, hence why it's still pending.

I hope to get back to these 2 PRs sometime this week, but feel free to merge them together if you get a chance to do so before I do and I can review that as well.

Thanks again for your contribution! 👍

chrislbs · 2017-04-04T15:18:22Z

@joaquincasares I'm thinking perhaps we have 2 separate pull requests for this work. One that is building the cassandra-reaper image (this one). And another that utilizes docker for building the debian and rpm packages #46 .

If you'd prefer to keep it as a single PR I can work on merging them tonight.

chrislbs · 2017-05-02T15:09:01Z

@adejanovski Do you think we can go ahead and merge this?

joaquincasares

Apologies for the extremely long delay! Alex was out for ~3 weeks for conferences and vacation and I was pretty busy for ~2 weeks in preparation for my own wedding.

I'm fine to merge this in today and I can work on my other Docker branch later. This is definitely the ideal path forward for a Docker-first sort of deployment model (I just couldn't figure out that Spotify plugin).

My other branch is more geared towards a build system for building Debian and RHEL packages as Chris noted, but no need to get both of these into a single PR.

Thanks again @chrislbs and major apologies for the delay!

joaquincasares · 2017-05-02T23:07:33Z

pom.xml

@@ -266,6 +266,25 @@
                            </execution>
                        </executions>
                    </plugin>
+                    <plugin>
+                        <groupId>com.spotify</groupId>
+                        <artifactId>docker-maven-plugin</artifactId>


I'm so glad you got this figured out! I couldn't, for the life of me, get this plugin to work correctly! :)

joaquincasares · 2017-05-02T23:07:58Z

src/main/docker/Dockerfile

+
+ARG SHADED_JAR
+
+ENV DW_REAPER_SEGMENT_COUNT=200 \


Why the DW_ prefix?

So, that was just for "Dropwizard" but since the application is effectively REAPER we could just drop the prefix as that is likely clearer.

joaquincasares · 2017-05-02T23:09:32Z

src/main/java/com/spotify/reaper/ReaperApplication.java

+    // enable using environment variables in yml files
+    final SubstitutingSourceProvider envSourceProvider = new SubstitutingSourceProvider(
+            bootstrap.getConfigurationSourceProvider(), new EnvironmentVariableSubstitutor(false));
+    bootstrap.setConfigurationSourceProvider(envSourceProvider);


Very nice configuration management extension!

joaquincasares · 2017-05-02T23:10:29Z

src/main/docker/entrypoint.sh

+
+if [ "$1" = 'cassandra-reaper' ]; then
+    /root/append-persistence.sh
+    exec java -jar ${JAVA_OPTS} /root/cassandra-reaper.jar server /root/cassandra-reaper.yml


Using gosu might be most secure, but I can handle that at a later date.

Agreed. I can do it as part of this pull request and refactor to remove the DW_

joaquincasares · 2017-05-02T23:19:10Z

Once we get a documented answer for the DW_ prefix, we're good to merge.

@adejanovski depending on when the answer comes in, mind if I merge this branch tomorrow if you're already offline?

chrislbs · 2017-05-03T02:05:52Z

@joaquincasares I used su-exec as it was an alpine base image and gosu seemed more cumbersome. Also, I removed the DW_ prefix from all the environment variables.

joaquincasares

Thanks @chrislbs those changes look great.

I caught the DB_REAPER issue and the other two are strictly nitpicks.

I forgot that @adejanovski would be out at a client on-site this week. Once the next patch comes in, I'll just merge it in myself.

I also hadn't realized you were based out of Houston. Let me know if you're ever in Austin and want to grab some drinks and/or present at the Austin Cassandra User Group!

Perhaps you could do a presentation on Reaper. hint hint :)

joaquincasares · 2017-05-03T15:30:11Z

src/main/docker/Dockerfile

+    REAPER_DB_DRIVER_CLASS="org.h2.Driver" \
+    REAPER_DB_URL="jdbc:h2:/var/lib/cassandra-reaper/db;MODE=PostgreSQL" \
+    DB_REAPER_DB_USERNAME="" \
+    DB_REAPER_DB_PASSWORD="" \


These two envars were left as DB_REAPER, while append-persistence.sh was correctly updated.

joaquincasares · 2017-05-03T15:30:31Z

src/main/docker/Dockerfile

+    chown reaper:reaper /usr/local/bin/entrypoint.sh && \
+    chown reaper:reaper /usr/local/bin/append-persistence.sh && \
+    chmod u+x /usr/local/bin/entrypoint.sh && \
+    chmod u+x /usr/local/bin/append-persistence.sh


We could combine the above chmod commands.

joaquincasares · 2017-05-03T15:30:45Z

src/main/docker/Dockerfile

+    chown reaper:reaper /etc/cassandra-reaper.yml && \
+    chown reaper:reaper /var/lib/cassandra-reaper && \
+    chown reaper:reaper /usr/local/bin/entrypoint.sh && \
+    chown reaper:reaper /usr/local/bin/append-persistence.sh && \


We could combine the above chown commands.

chrislbs · 2017-05-03T15:49:23Z

I might just take you up on that invitation @joaquincasares . Thanks for the feedback and the great tool.

joaquincasares · 2017-05-03T17:17:47Z

Thanks for the additional commit and yes, any time you're in Austin. :)

I only help with the Docker stuff. The Spotify team, other forks, @adejanovski , and @rustyrazorblade are more to blame for this repo of awesomeness. :D

Thanks again!

joaquincasares · 2017-07-08T01:30:17Z

Hey @chrislbs , I just finished that old #46 PR by breaking it out into: #134, #135, and #136 . Thanks again for your Docker image build setup! It ran just perfectly with ease and using the ideal way to do so. (Unlike my hacky attempt within #46. :p)

Thanks again! 👍

Adding support to build cassandra-reaper docker images

2588b9d

- Including environment variable substitution functionality for application yml - Including spotify docker build dependency - Adding initial Dockerfile and supporting startup shell scripts

joaquincasares reviewed May 2, 2017

View reviewed changes

Using su-exec and removing env var prefix

6195670

joaquincasares reviewed May 3, 2017

View reviewed changes

Correcting ENV DB prefix; Inlining chwon/chmod commands

8639ea2

joaquincasares merged commit fd3cf9c into thelastpickle:master May 3, 2017

This was referenced Jul 7, 2017

Add docker-build process. #134

Merged

Attempt building debian, jar, and rpm packages using Dockerfiles #46

Closed

Build Docker Images #75

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding support to build cassandra-reaper docker images #76

Adding support to build cassandra-reaper docker images #76

chrislbs commented Apr 4, 2017 •

edited

Loading

chrislbs commented Apr 4, 2017

joaquincasares commented Apr 4, 2017

chrislbs commented Apr 4, 2017

chrislbs commented May 2, 2017

joaquincasares left a comment

joaquincasares May 2, 2017

joaquincasares May 2, 2017

chrislbs May 3, 2017

joaquincasares May 2, 2017

joaquincasares May 2, 2017

chrislbs May 3, 2017

joaquincasares commented May 2, 2017

chrislbs commented May 3, 2017

joaquincasares left a comment

joaquincasares May 3, 2017

joaquincasares May 3, 2017

joaquincasares May 3, 2017

chrislbs commented May 3, 2017

joaquincasares commented May 3, 2017

joaquincasares commented Jul 8, 2017

Adding support to build cassandra-reaper docker images #76

Adding support to build cassandra-reaper docker images #76

Conversation

chrislbs commented Apr 4, 2017 • edited Loading

chrislbs commented Apr 4, 2017

joaquincasares commented Apr 4, 2017

chrislbs commented Apr 4, 2017

chrislbs commented May 2, 2017

joaquincasares left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaquincasares commented May 2, 2017

chrislbs commented May 3, 2017

joaquincasares left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chrislbs commented May 3, 2017

joaquincasares commented May 3, 2017

joaquincasares commented Jul 8, 2017

chrislbs commented Apr 4, 2017 •

edited

Loading