added docker image

[heywoodlh]

I would love to have something like warpgate running in my K8s cluster which is why I packaged it as a Docker image.

Right now, it's a bit hackey to use expect to write the password to a variable but if you implement #15 I think the expect stuff would not be necessary.

In regard to current usage something like this would be sufficient:

docker run -it -e ADMIN_USER='heywoodlh' -e ADMIN_PASS='heywoodlh' -p 8888:8888 -p 2222:2222 --rm warpgate run

[heywoodlh]

Oh, also, before this gets merged -- the web interface wasn't working and I assume it is because I didn't build the API client (it was failing to build for some reason when I ran 'just openapi-client').

So if you have any pointers on what I am missing in the Dockerfile I would appreciate it.

[Eugeny]

Looks pretty good! I've posted a workaround in #15 (pipe output through a cat).

Quick q: how does one add users/targets to the config file in this setup? I suppose I could mount a new config file over /etc/warpgate.yaml, but that would also overwrite the admin user info generated from the env vars.

I could also add an option to set a secondary config "overlay" file for the user-editable part.

[heywoodlh]

I don't have any current way to add users/targets into the config. I know for the sake of the image it would be nice to have an overlay but I could probably come up with something a bit nicer (like storing the config in the /data directory or something).

…

On Apr 11, 2022, at 01:16, Eugeny ***@***.***> wrote:  Looks pretty good! I've posted a workaround in #15 (pipe output through a cat). Quick q: how does one add users/targets to the config file in this setup? I suppose I could mount a new config file over /etc/warpgate.yaml, but that would also overwrite the admin user info generated from the env vars. I could also add an option to set a secondary config "overlay" file for the user-editable part. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[Eugeny]

I'll work adding config overlay support in the next days then - seems like a "cleaner" way and probably useful outside of Docker too

[heywoodlh]

This is the issue I'm running into:

#0 101.7 error: Justfile does not contain recipe `openapi-client`.
--------------------

[Eugeny]

My bad, it's just openapi

[heywoodlh]

So I updated the Dockerfile in 50af5fe to build relative to the root of the warpgate repo. So you would build like this (assuming you're in the root of the repo):

docker build -t warpgate -f docker/Dockerfile .

---

However, even after building openapi with just I still get a 404 error on the admin interface:

Any ideas what I'm missing?

[apiening]

I've created a minimalistic docker-compose.yml file to try this:

version: '3'
services:
  warpgate:
    build: .
    command: "run"
    ports:
      - "8888:8888"
      - "2222:2222"
    environment:
      - ADMIN_USER=testadmin
      - ADMIN_PASS=securepassword
    volumes:
      - ./data:/data

The build and startup works fine, but I'm running in the same 404 Not Found issue like @heywoodlh.

Regarding the config file: I think putting it into /data is a good idea. Because the config file could be set with -c /data/worpgate.yaml it should work right away.
Because of configuration auto update, there is not even the need to restart the container after a configuration update.

[Eugeny]

@heywoodlh are you getting any errors during build?

[apiening]

I've created a build.log file with the full output of the build process from

docker-compose build --no-cache --progress=plain 2>&1 | tee build.log

I have attached the file.

Can't spot any errors that may explain why the assets are missing or whatever causes the 404.
However, the Dockerfile copies the debug target from /opt/warpgate/target/debug/warpgate. Is this correct?
Maybe we should add the --release parameter to cargo build?

@Eugeny may this cause the assets to not be included?

build.log

[Eugeny]

Yes, that's it! In --debug mode, content isn't embedded into the binary and is instead read from the source tree.

[apiening]

I can confirm that this works.

This is what I did to get this up and running:

git clone https://github.com/Eugeny/warpgate.git
gh pr checkout 16
mv docker/Dockerfile .
mkdir data
vim docker-compose.yml #Insert the content from the post https://github.com/Eugeny/warpgate/pull/16#issuecomment-1096438318
vim Dockerfile #Change the source of the copy command from /opt/warpgate/target/debug/warpgate to /opt/warpgate/target/release/warpgate
docker-compose build --no-cache
docker-compose up -d --force-recreate && docker-compose logs -f

[apiening]

Moving the config from /etc/warpgate.yaml to /data/warpgate.yaml worked fine for me.

Here is my modified docker/run.sh:

#!/usr/bin/env bash

[[ -n ${ADMIN_USER} ]] || ADMIN_USER='admin'
[[ -n ${ADMIN_PASS} ]] || ADMIN_PASS='admin'

[[ -e /data/web-admin.certificate.pem ]] || openssl req -x509 -nodes -days 7300 -newkey rsa:4096 -keyout /data/web-admin.key.pem -out /data/web-admin.certificate.pem -subj "/C=PE/ST=Lima/L=Lima/O=Acme Inc. /OU=IT Department/CN=acme.com"

password_hash=$(echo -n "${ADMIN_PASS}" | warpgate hash | cat)


[[ -f '/data/warpgate.yaml'  ]] || cat << EOF > /data/warpgate.yaml
---
targets:
  - name: web-admin
    allow_roles:
      - "warpgate:admin"
    web_admin: {}
users:
  - username: ${ADMIN_USER}
    credentials:
      - type: password
        hash: "${password_hash}"
    roles:
      - "warpgate:admin"
roles:
  - name: "warpgate:admin"
recordings:
  enable: true
  path: /data/recordings
web_admin:
  enable: true
  listen: "0.0.0.0:8888"
  certificate: /data/web-admin.certificate.pem
  key: /data/web-admin.key.pem
database_url: "sqlite:/data/db"
ssh:
  listen: "0.0.0.0:2222"
  keys: /data/ssh-keys
  client_key: "./client_key"
retention: 7days
EOF

[[ -L /etc/warpgate.yaml  ]] || ln -s /data/warpgate.yaml /etc/warpgate.yaml

warpgate -c /data/warpgate.yaml $@

The check in front of the cat command makes sure, that the config file is not overwritten when the container is restarted or recreated.

The symbolic link at the end makes it possible to omit the -c /data/warpgate.yaml when doing a

docker-compose exec warpgate warpgate check
14:22:02  INFO Using config: "/etc/warpgate.yaml" (users: 1, targets: 3, roles: 1)
14:22:02  INFO No problems found

[heywoodlh]

Great, thanks for the info! I'll update the Dockerfile and run.sh in the next hour to reflect the changes stated!

[Eugeny]

I've looked into config overlays and there's no proper support for it in config-rs - not for merging nested arrays anyway (here: targets/users/roles). I think you guys are actually on the right track with just letting the user modify the config file after the creation!

IMO we can just tell the users to:

• Run the container once with ADMIN_USER and ADMIN_PASS
• After that, just manage the config file manually and don't set ADMIN_USER/PASS

What do you think?

[heywoodlh]

The changes have been added in cf9d78e and I think this is at a place where this could be pulled in as everything seems to be functional for me now.

[heywoodlh]

I've looked into config overlays and there's no proper support for it in config-rs - not for merging nested arrays anyway (here: targets/users/roles). I think you guys are actually on the right track with just letting the user modify the config file after the creation!

IMO we can just tell the users to:

• Run the container once with ADMIN_USER and ADMIN_PASS
• After that, just manage the config file manually and don't set ADMIN_USER/PASS

What do you think?

Sorry, I didn't see this response beforehand. And yeah, I think that's fine to just deal with updating the config manually for now.

[heywoodlh]

@Eugeny at the moment there is a bug in cargo with QEMU/Docker on ARM that prevents cargo from working properly. This means that we can't build the Docker image on 32 bit ARM (Raspberry Pi users are the ones I am thinking would be most impacted).

ARM64 and x86_64 the builds work just fine.

Would you like me to see if I can implement a workaround? Or do you think that for 32 bit Raspberry Pi users (or other 32 bit ARM users) they should just stick with the binary you make available?

[Eugeny]

I've got no experience running Docker on RPi - is it usable in reality? If so, I'll be happy to merge an ARM64 build if you've got time for it!

[heywoodlh]

So it works just fine on ARM64 -- I tested on my M1 Macbook and it works perfectly.

In regard to Docker on a Pi it works really well.

I've created a Github Action to build the container once a day on ARM64 and AMD64 and push it to Docker Hub:

https://github.com/heywoodlh/actions/blob/master/.github/workflows/warpgate-buildx.yml

If you want I could create another pull request with the Github Action (you would just have to create your own DOCKERHUB_USERNAME and DOCKERHUB_TOKEN secret for this repo).

[Eugeny]

Oh I've already got the arm64 build running here: https://github.com/warp-tech/warpgate/blob/main/.github/workflows/docker.yml - do you think you could turn it into a build matrix job and make a PR?

[heywoodlh]

Huh, I've never done a build matrix job before so it'd be a good learning experience! In this case, what would be the point in making a build matrix job? Not asking because I disagree -- I'm asking because I'm ignorant.

Also, the compilation is super slow now that I've added the --release argument. Is there any recommendation to speed it up?

[Eugeny]

No problem - a matrix job is just a job with parameters, e.g. architecture. Here's how I'm doing it in Tabby: https://github.com/Eugeny/tabby/blob/master/.github/workflows/build.yml#L238-L315 . I'm setting the arch list and then using matrix.arch to set $ARCH env var - but it can also be fed directly to an action as parameter.

The build time difference between debug and release profiles in Rust is a real thing, but release builds are heavily optimized and in this case, also 4x smaller.

[Eugeny]

Github then produces two separate jobs: