Mailing List Archive

[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Shreenidhi Shedi <sshedi@vmware.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |djm@mindrot.org,
| |dtucker@dtucker.net

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #1 from Darren Tucker <dtucker@dtucker.net> ---
(In reply to Shreenidhi Shedi from comment #0)
[...]
> When fips is enabled at server end and server has the following
> cipher set,
>
> ```
> root@phdev:~ $ sshd -T | grep ciphers
> ciphers
> chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,
> aes128-gcm@openssh.com,aes256-gcm@openssh.com

This indicates your FIPS modifications to the server are incorrect.

The server seems to be advertising the chacha20 cipher but seems unable
or unwilling to actually use it. If it's not going to support them it
should not offer them, and if the server's config explicitly includes
it then it should either refuse to start or remove them and log a
warning.

> root@phdev:~ $ rpm -q openssh
> openssh-9.1p1-10.ph5.x86_64 (this happens with 9.4p1 as well)
> ```
>
> The handshake with client starts with chacha20-poly1305 and this
> cipher is not fips complaint.
>
> I'm not sure what the intention was but in this commit:
> https://github.com/openssh/openssh-portable/commit/
> a22b9ef21285e81775732436f7c84a27bd3f71e0
>
> chacha20 cipher was promoted.

That was the intent, and it's fine. Simplifying a bit, the way it's
supposed to work is that the client picks the cipher from the list that
the server offers that it likes most.

Your server is lying about what ciphers it supports, and when the
client takes it up on its offer for chacha20, it doesn't work (from
your description, probably the server is aborting, but you'd have to
check your server logs).

Fix your server and it should work fine.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #2 from Shreenidhi Shedi <sshedi@vmware.com> ---
> Your server is lying about what ciphers it supports

This is the concern I have here. We are not explicitly setting these in
sshd_config and using defaults. Why does default cipher list show
chacha20 when it is not supporting it?

Or is it the suggested method from upstream that all downstream
instances should set cipherlist explicitly omitting chacha20 when fips
is enabled?

This list is coming from here:
https://github.com/openssh/openssh-portable/blob/master/myproposal.h#L59

I'm certain that we are not adding any cipher list explicitly.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #3 from Darren Tucker <dtucker@dtucker.net> ---
(In reply to Shreenidhi Shedi from comment #2)
> > Your server is lying about what ciphers it supports
>
> This is the concern I have here. We are not explicitly setting these
> in sshd_config and using defaults. Why does default cipher list show
> chacha20 when it is not supporting it?

Because your modifications to the server are insufficient. They should
remove it but don't.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #4 from Shreenidhi Shedi <sshedi@vmware.com> ---
(In reply to Darren Tucker from comment #3)
> (In reply to Shreenidhi Shedi from comment #2)
> > > Your server is lying about what ciphers it supports
> >
> > This is the concern I have here. We are not explicitly setting these
> > in sshd_config and using defaults. Why does default cipher list show
> > chacha20 when it is not supporting it?
>
> Because your modifications to the server are insufficient. They
> should remove it but don't.

Can you please point out what we are missing? here is our spec file.

https://github.com/vmware/photon/blob/5.0/SPECS/openssh/openssh.spec

Thanks a lot for your quick response and inputs. Like I mentioned
earlier, we are not modifying things much; do we need to enable any
config during configure stage?

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #5 from Darren Tucker <dtucker@dtucker.net> ---
(In reply to Shreenidhi Shedi from comment #4)
> Can you please point out what we are missing? here is our spec file.
>
> https://github.com/vmware/photon/blob/5.0/SPECS/openssh/openssh.spec
>
> Thanks a lot for your quick response and inputs. Like I mentioned
> earlier, we are not modifying things much; do we need to enable any
> config during configure stage?

In comment#1 you said:
"We did something like in PhotonOS 3.0:
https://github.com/vmware/photon/blob/3.0/SPECS/openssh/openssh-7.8p1-fips.patch"

but now you're pointing to a repo whos SPEC file doesn't contain
anything like that patch. It also specifies an OpenSSH version that
corresponds to *neither* of the versions you mentioned in comment#1.

You are applying a FIPS patch to OpenSSH or are you not? Are you
building a stock(ish) OpenSSH against a FIPS OpenSSL? Exactly which
versions are you talking about and exactly which modifications are you
making to it?

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #6 from Shreenidhi Shedi <sshedi@vmware.com> ---
Sorry for the confusion.

The patch I mentioned #1 is in Photon 3.0. Ignore the patch related
info I provided, that's something we are doing in 3.0 version only.

Now I'm hitting this issue in Photon 5.0 where we are using 9.3p2.

In this we just use what's given by upstream with mostly default
settings.

In this our cipher list is, exactly this
https://github.com/openssh/openssh-portable/blob/master/myproposal.h#L59

And we are hitting this issue.

You can try this in your setup as well, enable openssl fips in server,
use latest openssh server and try connecting from a client with no
ciphers mentioned.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Damien Miller <djm@mindrot.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Resolution|--- |INVALID
Status|NEW |RESOLVED

--- Comment #7 from Damien Miller <djm@mindrot.org> ---
Darren has stated several times what the problem is: your FIPS
modifications are simply broken. Refer to his answer in comment 1.

We're not in a position to fix this because we didn't write the FIPS
patch that you're using. You need to contact its authors if you want
support - we cannot help beyond pointing out how it is broken, which
Darren has already done.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #8 from Shreenidhi Shedi <sshedi@vmware.com> ---
Created attachment 3718
--> https://bugzilla.mindrot.org/attachment.cgi?id=3718&action=edit
sshd-config and sshd_config

Adding sshd config & client log both.

Doing ssh to localhost after enabling fips.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #9 from Shreenidhi Shedi <sshedi@vmware.com> ---
Hi Damien, thanks for your response.

Please ignore the details on the patch I provided.

I'm currently talking about newer openssh with no patch included, check
this spec.

https://github.com/vmware/photon/blob/5.0/SPECS/openssh/openssh.spec

I'm using 9.3p2 here and no patch and don't intend to add it as well.

Now question is, if I take latest openssh and deploy it on both client
and server and enable fips in server, will client be able to
communicate with the server?

I am attaching the log for this scenario but with a slightly lower
version of server and client. PTAL.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Shreenidhi Shedi <sshedi@vmware.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
Resolution|INVALID |---
Status|RESOLVED |REOPENED

--- Comment #10 from Shreenidhi Shedi <sshedi@vmware.com> ---
I have to reopen this. I think the ticket is closed with
misunderstanding.

Please check the logs I attached and try it once in your machine.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #11 from Shreenidhi Shedi <sshedi@vmware.com> ---
Created attachment 3719
--> https://bugzilla.mindrot.org/attachment.cgi?id=3719&action=edit
client connection log

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #12 from Damien Miller <djm@mindrot.org> ---
Created attachment 3720
--> https://bugzilla.mindrot.org/attachment.cgi?id=3720&action=edit
runtime probing of ciphers

> You can try this in your setup as well, enable openssl fips in server, use
> latest openssh server and try connecting from a client with no ciphers
> mentioned.

ok, you're putting OpenSSL in FIPS mode and not patching OpenSSH.
You've then created a situation where the OpenSSL you're using is
behaving differently to the OpenSSL that OpenSSH was compiled with, and
currently OpenSSH is not in a position to detect this.

Changing this basically requires that OpenSSH do runtime probing of all
cryptography to see whether something has changed underneath it.

This patch is an example of how we might approach this. Maybe it helps
your case? It certainly isn't complete - we'd need to do effectively
the same thing for MACs, public key algorithms and key agreement
algorithms too since I bet some of those could be disabled by OpenSSL's
FIPS support too. The patch could probably be simplified if there's a
simpler way to query whether OpenSSL supports a particular algorithm.

In the meantime, if you're changing your crypto library to disable
particular algorithms, then you need to *manually* change your
ssh_config and sshd_config to match.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #13 from Shreenidhi Shedi <sshedi@vmware.com> ---
Hi Damien,

Thanks for being patient and understanding the problem.

So if I have understood it right, upstream is intending to fix this
issue and we can expect a fix in the future releases of openssh.

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #14 from Damien Miller <djm@mindrot.org> ---
No, I don't think we've decided that yet.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Dmitry Belyavskiy <dbelyavs@redhat.com> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |dbelyavs@redhat.com

--- Comment #15 from Dmitry Belyavskiy <dbelyavs@redhat.com> ---
In case of OpenSSL 3.0+ EVP_fetch* methods can be used for this
purpose.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #16 from Damien Miller <djm@mindrot.org> ---
I don't think that does what we want - that function seems to allow
going from a string name to an EVP but we already have the EVP and just
want to check that it is enabled by OpenSSL without having to arrange
everything EVP_*_init() wants.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Damien Miller <djm@mindrot.org> changed:

What |Removed |Added
----------------------------------------------------------------------------
Attachment #3720|0 |1
is obsolete| |

--- Comment #17 from Damien Miller <djm@mindrot.org> ---
Created attachment 3721
--> https://bugzilla.mindrot.org/attachment.cgi?id=3721&action=edit
probe ciphers, MACs and KEX

--
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

Roumen Petrov <bugtrack@roumenpetrov.info> changed:

What |Removed |Added
----------------------------------------------------------------------------
CC| |bugtrack@roumenpetrov.info

--- Comment #18 from Roumen Petrov <bugtrack@roumenpetrov.info> ---
So, world knows hot to detect what is available in FIPS mode.
More then 15 years OpenBSD ream refuse to do simple detection. It is
expected to continue in next decades.

Shreenidhi Shedi,
Just use PKIX-SSH instead - could be used with OpenSSL, RedHad and
Solaris FIPS validated OpenSSL libraries.

Regards,
Roumen

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #19 from Shreenidhi Shedi <sshedi@vmware.com> ---
>Shreenidhi Shedi,
> Just use PKIX-SSH instead - could be used with OpenSSL, RedHad and Solaris FIPS validated OpenSSL libraries.

Hi Roumen,

Thanks for the input.
I'm not sure what pkix-ssh is, can you share some more info please?

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs
[Bug 3603] ssh clients can't communicate with server with default cipher when fips is enabled at server end [ In reply to ]
https://bugzilla.mindrot.org/show_bug.cgi?id=3603

--- Comment #20 from Dmitry Belyavskiy <dbelyavs@redhat.com> ---
I see several problems with the proposed patch. It resolves the case
when the run-time and build-time OpenSSL version differs in
capabilities. The problem is it relies on legacy OpenSSL API that
contradicts the initial request (FIPS compatibility). Also EC curve
detection uses the API OpenSSL considers legacy (and so not
FIPS-compliant). And from the FIPS perspective, all NIST curves
supported by OpenSSH are allowed.

--
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.
_______________________________________________
openssh-bugs mailing list
openssh-bugs@mindrot.org
https://lists.mindrot.org/mailman/listinfo/openssh-bugs