Client:Sent SSH_MSG_USERAUTH_REQUEST twice even PASSWORD_PROMPTS=1

[ghost]

Version

2.9.0

Bug description

Trying to find out if the reported in

https://issues.apache.org/jira/browse/SSHD-1159

has been fixed or not. Don't have ASF Jira account, so cannot update the information there.

For SSHD-1159, comment by Lyor Goldstein says

"Version 2.2.0 is quite old - please try 2.7.0"

I tried with version 2.9.0 and see the same behavior. SSH server is reporting two login attempts even though we run the following program only once. sshd daemon is configured with two auth methods, Publickey,keyboard-interactive

package sshclient;

import java.io.IOException;
import java.util.Collections;
import java.util.concurrent.TimeUnit;
import org.apache.sshd.client.SshClient;
import org.apache.sshd.client.auth.keyboard.UserAuthKeyboardInteractiveFactory;
import org.apache.sshd.client.session.ClientSession;
import org.apache.sshd.common.PropertyResolverUtils;
import org.apache.sshd.core.CoreModuleProperties;

public class ApacheSshDemo {

public static void main(String[] args) throws Exception {
    ApacheSshDemo demo = new ApacheSshDemo();
    System.err.print("starting apache sshd demo");
    demo.newSession("localhost", "_v_10.0.1.1_v_admin", "blahblahblah", 830, 30);
}

public void newSession(String host, String username,
String password, int port, long defaultTimeout) throws IOException {
SshClient client = SshClient.setUpDefaultClient();
PropertyResolverUtils.updateProperty(client, CoreModuleProperties.PASSWORD_PROMPTS.getName(), 1);
client.start();
client.setUserAuthFactories(Collections.singletonList(UserAuthKeyboardInteractiveFactory.INSTANCE));
try (ClientSession session = client.connect(username, host, port)
.verify(defaultTimeout, TimeUnit.SECONDS)
.getSession()) {
session.addPasswordIdentity(password);
session.auth()
.verify(defaultTimeout, TimeUnit.SECONDS);
System.err.println("ssh connection was successful");
} catch (Exception e) {
System.err.println("ssh exception is "+e);
} finally {
client.stop();
}
}
}

This is sshd dependency specified in pom.xml

org.apache.sshd sshd-core 2.9.0 Actual behavior

Utility on SSH server is indicating two login attempts.

vm12:/var/log# !fail
faillock --dir /var/log/faillock --user admin
admin:
When Type Source Valid
2022-12-08 13:54:25 RHOST 10.0.1.1 V
2022-12-08 13:54:29 RHOST 10.0.1.1 V

Expected behavior

vm12:/var/log# !fail
faillock --dir /var/log/faillock --user admin
admin:
When Type Source Valid
2022-12-08 13:54:25 RHOST 10.0.1.1 V

Relevant log output

No response

Other information

OpenSSH_8.2

[tomaswolf]

I cannot reproduce this, and I have no idea what your server-side utility shows.

Run your test program with debug logging on and examine the client log. You will see that it will send an initial SSH_MSG_USERAUTH_REQUEST for authentication method "none". The server will reply with a failure, and will tell in that failure reply the client which authentication schemes are enabled on the server. Then the client will choose keyboard-interactive and send the real authentication request SSH_MSG_USERAUTH_REQUEST for "keyboard-interactive" with the password.

As far as I can tell everything is working exactly as it should. Maybe teach that server-side utility not to report SSH_MSG_USERAUTH_REQUEST "none". See RFC 4252, section 5.2.

[ghost]

Thank you Thomas very much for promptly looking into this. I will check the behavior on the server to see if it is counting none as a failed login attempt.

One quick question. How do I enable debug logs for apace mina sshd library. Are there instructions anywhere on how to do this?

[tomaswolf]

You would include a logging back-end with your test application, for instance slf4j-simple, and then configure it.

[ghost]

Server side log shows that initial userauth-request with ssh-connection method none is not counted as a failure attempt. However, there are two attempts with ssh-connection method=keyboard-interactive. This is for the client program given in the bug description.

I will enable debug logs on the client side to see why do we have this disparity.

Dec 8 18:55:55 vm12 sshd[11757]: debug1: userauth-request for user _v_10.0.1.1_v_admin service ssh-connection method none [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: attempt 0 failures 0 [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: Processing for user _v_10.0.1.1_v_admin. rhost (10.0.1.1) (8) (_v_admin)
Dec 8 18:55:55 vm12 sshd[11757]: username_raw:_v_10.0.1.1_v_admin, username:admin, remote_ip:10.0.1.1
Dec 8 18:55:55 vm12 sshd[11757]: debug1: connection from 127.0.0.1 matched 'LocalPort 830' at line 26
Dec 8 18:55:55 vm12 sshd[11757]: debug1: PAM: initializing for "admin"
Dec 8 18:55:55 vm12 sshd[11757]: debug1: PAM: setting PAM_RHOST to "10.0.1.1"
Dec 8 18:55:55 vm12 sshd[11757]: debug1: PAM: setting PAM_TTY to "ssh"
Dec 8 18:55:55 vm12 sshd[11757]: debug1: userauth_send_banner: sent [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: userauth-request for user _v_10.0.1.1_v_admin service ssh-connection method keyboard-interactive [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: attempt 1 failures 0 [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: keyboard-interactive devs [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: auth2_challenge: user=_v_10.0.1.1_v_admin devs= [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: kbdint_alloc: devices 'pam' [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth]
Dec 8 18:55:55 vm12 sshd[11757]: Postponed keyboard-interactive for _v_10.0.1.1_v_admin from 127.0.0.1 port 51184 ssh2 [preauth]
Dec 8 18:55:55 vm12 sshd[11774]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=10.0.1.1 user=admin
Dec 8 18:55:59 vm12 sshd[11757]: error: PAM: Authentication failure for admin from 10.0.1.1
Dec 8 18:55:59 vm12 sshd[11757]: Failed keyboard-interactive/pam for admin from 10.0.1.1 port 51184 ssh2
Dec 8 18:55:59 vm12 sshd[11757]: debug1: userauth-request for user _v_10.0.1.1_v_admin service ssh-connection method keyboard-interactive [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: debug1: attempt 2 failures 1 [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: debug1: keyboard-interactive devs [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: debug1: auth2_challenge: user=_v_10.0.1.1_v_admin devs= [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: debug1: kbdint_alloc: devices 'pam' [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: debug1: auth2_challenge_start: trying authentication method 'pam' [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: Postponed keyboard-interactive for _v_10.0.1.1_v_admin from 127.0.0.1 port 51184 ssh2 [preauth]
Dec 8 18:55:59 vm12 sshd[11757]: Connection closed by authenticating user _v_10.0.1.1_v_admin 127.0.0.1 port 51184 [preauth]

[tomaswolf]

Yes, I see now where this goes wrong.

[ghost]

Hi Thomas, do mean that you can see the issue in the sshd library and it is sending two user auth requests instead of one (excluding the initial request with connection-method=none)?

[tomaswolf]

I labeled this issue as a "bug": yes, there is something wrong in Apache MINA SSHD.

On the surface, it's just an off-by-one error in the case that all passwords were refused. But actually the problem is a little bit deeper and may need a bit more work to fix properly.

[ghost]

Thanks Thomas. I see that you have marked it to be fixed in 2.9.3. Do we have a rough estimate on when 2.9.3 will be released. Also, do the bug fixes get back ported to earlier sshd versions (like 2.4 for instance), or we have pretty much have to upgrade to 2.9.3 to pick up the fix?

[tomaswolf]

2.9.3 just means "next release". No, there's is no timeline. All maintainers are unpaid volunteers doing this in their spare time. We do not backport fixes.

[tomaswolf]

@gnodet : do you remember what the original intent of this property CoreModuleProperties.PASSWORD_PROMPTS was?

When I follow the git history, I end up at commit 65e0320 and SSHD-302, both from 2014. They don't give any context or motivation.

Currently it appears to be a global count of SSH_MSG_USERAUTH_INFO_REQUEST received from the server, and thus a guard against RFC 4256, section 3.2: "The server may send as many requests as are necessary to authenticate the client;..." (a malicious or broken server might just keep sending them). However, the implementation also uses it to limit the number of times a SSH_MSG_USERAUTH_REQUEST message is sent. Which has an off-by-one error, and which makes only sense if the server sends only one SSH_MSG_USERAUTH_INFO_REQUEST with prompts per request.

It does not appear an implementation of the ssh config NumberOfPasswordPrompts, as both the reporter and me obviously had thought. (Despite the name, OpenSSH uses the NumberOfPasswordPrompts to limit the number of password or keyboard-interactive attempts, i.e., the sending of SSH_MSG_USERAUTH_REQUEST messages.) Apache MINA SSHD doesn't seem to have anything for that. UserAuthPassword also doesn't use the CoreModuleProperties.PASSWORD_PROMPTS at all.

So what was the intention here?

If we want to guard against a malicious server in keyboard-interactive auth, then CoreModuleProperties.PASSWORD_PROMPTS should keep counting and limiting handling SSH_MSG_USERAUTH_INFO_REQUEST. But then there should be a separate CoreModuleProperties.PASSWORD_ATTEMPTS to limit the number of attempts (sending of SSH_MSG_USERAUTH_REQUEST for both password and keyboard-interactive, counted separately). The latter would then indeed correspond to the OpenSSH NumberOfPasswordPrompts ssh config.

[tomaswolf]

Absent an answer, I've pushed a fix to PR #284. It implements CoreModuleProperties.PASSWORD_PROMPTSas the equivalent of the OpenSSH config NumberOfPasswordPrompts. Only interactive authentication attempts count towards the limit.

There is no special guard against a server that keeps sending challenges in keyboard-interactive authentication. Normally such behavior would be caught by either of the following:

• supplied passwords for non-interactively authenticating being exhausted,
• the user cancelling authentication on the umpteenth interactive dialog for a challenge,
• or the client's authentication time-out expiring.