Bump sshd-core to 2.6.0 and update tests

[res0nance]

See JENKINS-XXXXX.

Supersedes: #5203 the code has been updated to keep the same legacy behaviour

Proposed changelog entries

• Update Jenkins CLI to SSHD Core from 1.7.0 to 2.6.0
• ...

Proposed upgrade guidelines

N/A

Submitter checklist

• (If applicable) Jira issue is well described
• Changelog entries and upgrade guidelines are appropriate for the audience affected by the change (users or developer, depending on the change). Examples
  • Fill-in the Proposed changelog entries section only if there are breaking changes or other changes which may require extra steps from users during the upgrade
• Appropriate autotests or explanation to why this change has no tests
• For dependency updates: links to external changelogs and, if possible, full diffs

Desired reviewers

@mention

Maintainer checklist

Before the changes are marked as ready-for-merge:

• There are at least 2 approvals for the pull request and no outstanding requests for change
• Conversations in the pull request are over OR it is explicit that a reviewer does not block the change
• Changelog entries in the PR title and/or Proposed changelog entries are correct
• Proper changelog labels are set so that the changelog can be generated automatically
• If the change needs additional upgrade steps from users, upgrade-guide-needed label is set and there is a Proposed upgrade guidelines section in the PR title. (example)
• If it would make sense to backport the change to LTS, a Jira issue must exist, be a Bug or Improvement, and be labeled as lts-candidate to be considered (see query).

[oleg-nenashev]

Spliterator is definitely an overkill here, but it is not a hot spot in CLI.
Thanks for the patch!

[jvz]

PKCS12 support, nice!

[timja]

nice

[res0nance]

This seems to break for some unknown reason, I tried debugging this but, I'm not certain why its broken

[oleg-nenashev]

Likely https://github.com/jenkinsci/sshd-module needs to be updated as well.

Tests fail with:

WARNING: Unknown host key for localhost/127.0.0.1:34717
io.jenkins.cli.shaded.org.apache.sshd.common.SshException: No more authentication methods available
	at io.jenkins.cli.shaded.org.apache.sshd.common.future.AbstractSshFuture.verifyResult(AbstractSshFuture.java:126)
	at io.jenkins.cli.shaded.org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:39)
	at io.jenkins.cli.shaded.org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:32)
	at hudson.cli.SSHCLI.sshConnection(SSHCLI.java:108)
	at hudson.cli.CLI._main(CLI.java:285)
	at hudson.cli.CLI.main(CLI.java:96)
Caused by: io.jenkins.cli.shaded.org.apache.sshd.common.SshException: No more authentication methods available
	at io.jenkins.cli.shaded.org.apache.sshd.client.session.ClientUserAuthService.tryNext(ClientUserAuthService.java:342)

AND

WARNING: Unknown host key for localhost/127.0.0.1:34569
io.jenkins.cli.shaded.org.apache.sshd.common.SshException: Server key did not validate   4.093 [id=166]	WARNING	o.a.s.c.s.h.AbstractSession#exceptionCaught: exceptionCaught(ServerSessionImpl[null@/127.0.0.1:39108])[state=Opened] IOException: Connection reset by peer
	at io.jenkins.cli.shaded.org.apache.sshd.common.future.AbstractSshFuture.verifyResult(AbstractSshFuture.java:126)
	at io.jenkins.cli.shaded.org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:39)
	at io.jenkins.cli.shaded.org.apache.sshd.client.future.DefaultAuthFuture.verify(DefaultAuthFuture.java:32)
	at hudson.cli.SSHCLI.sshConnection(SSHCLI.java:108)
	at hudson.cli.CLI._main(CLI.java:285)
	at hudson.cli.CLI.main(CLI.java:96)
Caused by: io.jenkins.cli.shaded.org.apache.sshd.common.SshException: Server key did not validate
	at io.jenkins.cli.shaded.org.apache.sshd.client.session.AbstractClientSession.checkKeys(AbstractClientSession.java:583)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSession.handleKexMessage(AbstractSession.java:611)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSession.doHandleMessage(AbstractSession.java:500)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSession.handleMessage(AbstractSession.java:428)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSession.decode(AbstractSession.java:1463)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSession.messageReceived(AbstractSession.java:388)
	at io.jenkins.cli.shaded.org.apache.sshd.common.session.helpers.AbstractSessionIoHandler.messageReceived(AbstractSessionIoHandler.java:64)
	at io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Session.handleReadCycleCompletion(Nio2Session.java:358)
	at io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:335)
	at io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2Session$1.onCompleted(Nio2Session.java:332)
	at io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2CompletionHandler.lambda$completed$0(Nio2CompletionHandler.java:38)
	at java.security.AccessController.doPrivileged(Native Method)
	at io.jenkins.cli.shaded.org.apache.sshd.common.io.nio2.Nio2CompletionHandler.completed(Nio2CompletionHandler.java:37)
	at sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:126)
	at sun.nio.ch.Invoker$2.run(Invoker.java:218)
	at sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:112)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

[timja]

the tests pass on my machine 🤷

there's work here to detach the ssh module: #5049

[oleg-nenashev]

We may merge it in 24 hours if there is no negative feedback. Please see the merge process documentation for more information about the merge process

[andrealai75]

I raised this: https://issues.jenkins.io/browse/JENKINS-65273 as since the upgrade we are getting "No more authentication methods available". I am not sure if it is related to this change but 2.280 was working fine with SSH Credentials Plugin version 1.18.1

[andrealai75]

We downgraded to 2.283 and the problem is gone so I presume 2.284 has a fault or a change in settings that we are missing

[batmat]

@timja @res0nance it seems like this bump caused many issues. Not sure we'll be able to find a fix for the forthcoming LTS, but worth keeping in the radar I think.

FYI I raised this one during the gov meeting yesterday.

Thanks!

[timja]

Hmm may be why I couldn't re-produce jenkinsci/azure-ad-plugin#128, possibly I was using weekly

---

FYI I raised this one during the gov meeting yesterday.

Much more visible to raise it in the mailing list. I can't attend those meetings due to the time they are at.