Hosts with Multi-Factor auth fail

[heywoodlh]

I have one host using Duo's PAM module to provide multi factor authentication and another using Jumpcloud for the same purpose. Through Warpgate it fails despite having the ~/.ssh/authorized_keys file configured properly.

Connection failed  Authentication failed
channel 0: protocol error: close rcvd twice

Here's what the entire workflow looks like on the host using Duo:

ssh heywoodlh:arch-firewall.wireguard@warpgate.kube
heywoodlh:arch-firewall.wireguard@warpgate.kube's password:
 Warpgate  Selected target: arch-firewall.wireguard
 Warpgate  Host key ...
 Connection failed  Authentication failed
channel 0: protocol error: close rcvd twice
Connection to warpgate.kube closed.

And here's what it looks like for the host with Jumpcloud (I changed the hostname in this output):

ssh heywoodlh:example-host@warpgate.kube
heywoodlh:example-host@warpgate.kube's password:
channel 0: protocol error: close rcvd twice
 Warpgate  Selected target: example-host
 Connection failed  Connection refused (os error 111)
Connection to warpgate.kube closed.

As a sanity check, it seems to work just fine with my other machines not using multi-factor auth:

ssh heywoodlh:boba.wireguard@warpgate.kube
heywoodlh:boba.wireguard@warpgate.kube's password:
 Warpgate  Selected target: boba.wireguard
 Warpgate  Host key ...
 ✓ Warpgate connected

Last login: Wed Apr 13 15:09:11 2022 from 10.50.50.38
[heywoodlh@boba ~]$

[Eugeny]

Warpgate doesn't support keyboard-interactive auth on the targets or passing the interactive prompts to the client by design - my plan is to add 2FA support to WG directly, so that admins only need to handle 2FA config in one place.

The second error (Connection refused) looks like an incorrect host/port though.

[bram-pkg]

I think I'm experiencing the same issue, I have added a password, public key and otp to a user.
After adding require: [publickey, otp], I cannot log in anymore.
Warpgate prompts for a password, and then gives a permission denied (publickey,keyboard-interactive)

I'm not sure how to fix this?

When removing the password from my config, it still asks for one.
Is there a way to allow authentication with publickey only?

[Eugeny]

@bram-pkg does it work if you only set require: [publickey]? If your client is OpenSSH and it's asking for password, that means that the public key auth has already failed. You can also observe the auth flow with ssh -v

[bram-pkg]

It does not, my config looks like this:

users:
  # default admin...
  - username: bram
    credentials:
      #- type: password
      #  hash: "$argon2id$v.........."
      - type: publickey
        key: ssh-rsa blablablabla
      - type: otp
        key: long-otp-key
    require: [publickey]
    roles:
      - "backups"

And it still asks for a password. Note that the password is commented out right now.

Running ssh -i ~/.ssh/my-identity -p 2222 bram:target@ip -v it shows the following:

OpenSSH_9.0p1, OpenSSL 1.1.1o  3 May 2022
debug1: Reading configuration data /home/bram/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to <redacted> [<redacted>] port 2222.
debug1: Connection established.
debug1: identity file /home/bram/.ssh/my-identity type 0
debug1: identity file /home/bram/.ssh/my-identity-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.0
debug1: Remote protocol version 2.0, remote software version russh_0.34.0-beta.2
debug1: compat_banner: no match: russh_0.34.0-beta.2
debug1: Authenticating to <redacted>:2222 as 'bram:target'
debug1: load_hostkeys: fopen /home/bram/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256@libssh.org
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:my-server-key
debug1: load_hostkeys: fopen /home/bram/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[<redacted>]:2222' is known and matches the ED25519 host key.
debug1: Found key in /home/bram/.ssh/known_hosts:52
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/bram/.ssh/my-identity RSA SHA256:my-fingerprint explicit
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password,publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering public key: /home/bram/.ssh/my-identity RSA SHA256:my-fingerprint explicit
debug1: send_pubkey_test: no mutual signature algorithm
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: password,publickey,keyboard-interactive
debug1: Next authentication method: password
bram:target@<redacted>'s password:

[bram-pkg]

Hope that's enough information 😅

[Eugeny]

This seems to be the problem:

debug1: send_pubkey_test: no mutual signature algorithm

Could you generate a new private key with the exact same ssh-keygen settings and post it or send it to inbox@null.page?

In the meanwhile, a workaround could be to generate and use a separate Ed25519 key.

[bram-pkg]

I will try an Ed25519 key.

I will send you an RSA key in the format I used for this.

[bram-pkg]

Sent them to you in an email.

[bram-pkg]

I generated an SSH key with the following command now:

ssh-keygen -t ed25519

Added it to Warpgate, and now it doesn't ask for a password anymore.

After added otp to the require: [publickey, otp] list, it is also asking for the One-time password. Everything seems to be functioning now!

Apart from the small RSA key issue, ofcourse.

Thanks for your help!

[bram-pkg]

Correction, the connection freezes after entering my OTP code 😅

(bram:target@<redacted>) One-time password: 123456
Authenticated to <redacted> ([<redacted>]:2222) using "keyboard-interactive".
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: filesystem

And then it hangs. I'm forced to shut down my terminal and start a new session, Ctrl+C doesn't disconnect.

[Eugeny]

Could you please run Warpgate as RUST_LOG=debug warpgate run on the server side, try connecting again and post the resulting log?

[bram-pkg]

Sure thing, give me a minute.

[bram-pkg]

Could I email the log to you? It's quite big.
Same for the other ticket #139

[Eugeny]

Sure - same address

[bram-pkg]

I sent them to you, hope I named the files in a clear enough way.

[Eugeny]

Released in 0.2.4: https://github.com/warp-tech/warpgate/releases/tag/v0.2.4

[samtoxie]

I actually think the issues of @bram-pkg and @heywoodlh are actually two different issues. I'm running in to the same problem as heywoodlh.

The problem is when a target server has 2fa optionally enabled (for example like so: https://www.digitalocean.com/community/tutorials/how-to-set-up-multi-factor-authentication-for-ssh-on-ubuntu-20-04), as even when a user (such as the one used by warpgate) does not have otp configured the sshd will still force a keyboard-interactive auth.

In the example below I directly connected to the target using my own key, and not warpgate. This shows the sshd offering keyboard-interactive with will not actually prompt anything and instantly succeed:

debug1: Offering public key: /home/user/.ssh/id_ed25519 ED25519 SHA256:MYKEY explicit agent
debug1: Server accepts key: /home/user/.ssh/id_ed25519 ED25519 SHA256:MYKEY explicit agent
Authenticated using "publickey" with partial success.
debug1: Authentications that can continue: keyboard-interactive
debug1: Next authentication method: keyboard-interactive
Authenticated to with-optional-otp.my.net ([10.69.224.135]:22) using "keyboard-interactive".
debug1: channel 0: new session [client-session] (inactive timeout: 0)

Compared to a target that does not have otp optionally configured at all:

debug1: Offering public key: /home/user/.ssh/id_ed25519 ED25519 SHA256:MYKEY agent
debug1: Server accepts key: /home/user/.ssh/id_ed25519 ED25519 SHA256:MYKEY agent
Authenticated to without-optional-otp.my.net ([10.69.218.146]:22) using "publickey".
debug1: channel 0: new session [client-session] (inactive timeout: 0)

@Eugeny would it be possible to implement something in the SSH client to just silently accept this (maybe with a 1 or 2 second timeout)? This way warpgate and OTP can be used side by side in different users on a target.