Skip to content

Conversation

jhessane
Copy link

Dropbear didn't build on Illumos and Solaris for a few reasons:

  • mv is not the coreutils one, so the -v option doesn't exist. However, the gnu coreutils are shipped under /usr/gnu/bin so we can just change which mv to use on Illumos or Solaris to /usr/gnu/bin/mv, if not on Solaris or Illumos it will default to mv;

  • we need to set STDC_WANT_LIB_EXT1 to 1 to use memory safe functions, so I added a definition for Solaris, defined to 1 if we are on Solaris/Illumos and a check in dbhelper.h to define STDC_WANT_LIB_EXT1 to 1 if we are on Solaris and then we can use memset_s;

  • there is no lastlog file on Solaris/Illumos;

  • linking fails only on Solaris because we need to link with libssp (-lssp) to use __stack_chk_guard and __stack_chk_fail, linking with libssp on Illumos doesn't seem to add problems.

With all those little fixes, Dropbear now works on Illumos and Solaris. I tested those fixes on OmniOS r151054 (latest LTS), SmartOS 20250710T000436Z (latest release), on Solaris 11.4 and also on Void Linux with glibc to see if didn't obliterate something.
However while working on that, I did not pay attention to the version of autoconf installed of my system, it was 2.72, so I rolled back to 2.71 to generate a new configure script, as it was before.

jhessane added 5 commits July 20, 2025 16:17
On Solaris and illumos, mv does not have the -v option (verbose), but on Solaris
and most Illumos distros, the gnu coreutils are shipped so we can just change
which mv to use (in this case /usr/gnu/bin/mv). On all other OSes it default to
just mv.
On Solaris and Illumos, safe functions usage is not set to 1 by default, so we
should have a way to define __STDC_WANT_LIB_EXT1__ to 1 on Solaris and Illumos.
On Solaris and Illumos, CONF_LASTLOG_FILE is set, however there is no support
of lastlog.
This is a problem on Solaris only, but linking fails without -lssp. Having this
on Illumos does not seem to had problems.
I did not pay attention, but my system had 2.72. Using 2.71 as before.
@jhessane
Copy link
Author

I'll investigate checks later today, at first glance it doesn't seem too weird. Checks also don't work on solaris and illumos because calling make in the Makefile calls dmake (sun make). Should add something to specify gmake if we are on solaris.

Copy link
Owner

@mkj mkj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good to have fixes for platforms!

I think it would be fine to just change mv -v to mv - I guess it was only added for debugging. A couple of other thoughts below.

CFLAGS="$CFLAGS -I/usr/local/include"
LDFLAGS="$LDFLAGS -L/usr/local/lib -R/usr/local/lib"
conf_lastlog_location="/var/adm/lastlog"
LDFLAGS="$LDFLAGS -L/usr/local/lib -R/usr/local/lib -lssp"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably better to do

AC_CHECK_LIB(ssp, some_function_in_libssp, LIBS="$LIBS -lssp")

since I suspect there are old versions of solaris without libssp that would break. (Not sure what function you would use there).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can just check for __stack_chk_fail,it's the function used:

AC_CHECK_LIB(ssp, __stack_chk_fail, LIBS="$LIBS -lssp")

When running ./configure:

checking for __stack_chk_fail in -lssp... yes

In Makefile:

LIBS+=-lz  -lssp -lsocket -lnsl

However when building:

gcc -mindirect-branch=thunk -mfunction-return=thunk -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fPIE -Wundef -fno-strict-overflow -Wno-pointer-sign -Os -W -Wall -I/usr/local/include  -I./src/../libtomcrypt/src/headers/ -I. -I./src -DDROPBEAR_SERVER -DDROPBEAR_CLIENT src/scpmisc.c -o obj/scpmisc.o -c
gcc -Wl,-pie  -L/usr/local/lib -R/usr/local/lib -o scp ./obj/scp.o ./obj/progressmeter.o ./obj/atomicio.o ./obj/scpmisc.o ./obj/compat.o
Undefined                       first referenced
 symbol                             in file
__stack_chk_fail                    ./obj/scp.o
__stack_chk_guard                   ./obj/scp.o
ld: fatal: symbol referencing errors
collect2: error: ld returned 1 exit status
gmake: *** [Makefile:237: scp] Error 1

Same error as before. I'm testing on Solaris, I know that their linker is a bit strict on libraries order so I tried editing the makefile, putting -lssp in first position, doesn't change anything. I also tried to use an other linker, but weirdly enough, same thing. I'll try on OmniOS.

Copy link
Author

@jhessane jhessane Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This way works on OmniOS. From what I read in their gcc patches; they added __stack_chk_fail() and __stack_chk_guard in the illumos libc, so they added solaris2 to the ssp compatibles OSes in gcc. I don't really know what to do from here, should we keep it the way it was (having -lssp defined in LDFLAGS) or hope that the great Oracle will patch his gcc?
Using gcc14 on both Solaris and OmniOS, there is gcc15 on Solaris but it seems like it's going to be the same problem.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I guess if it's fiddly linker order leave it as-is.

#endif

#if defined(SOLARIS)
#define __STDC_WANT_LIB_EXT1__ 1
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this can be enabled unconditionally, no need to check for Solaris. AFAIK the only reason it's hidden behind a #define is in case of clashes with program symbols, but that won't be the case here.

Copy link
Author

@jhessane jhessane Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really know how the support of older OSes is, so I did it this way in case a compiler is not C11 compliant (I think it's not even on some recent C11 compilers because it's optional).

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm pretty sure it'll be fine without the #if defined(SOLARIS)

@mkj
Copy link
Owner

mkj commented Jul 21, 2025

I'll investigate checks later today, at first glance it doesn't seem too weird.

Don't bother about it - it's a spurious failure I've been meaning to track down.

Checks also don't work on solaris and illumos because calling make in the Makefile calls dmake (sun make). Should add something to specify gmake if we are on solaris.

Ah, I'll see how easy it is to make more portable.

@mkj
Copy link
Owner

mkj commented Sep 2, 2025

Checks also don't work on solaris and illumos because calling make in the Makefile calls dmake (sun make). Should add something to specify gmake if we are on solaris.

Hopefully it's a bit better in
beb0616

@jhessane
Copy link
Author

jhessane commented Sep 2, 2025

There are still a few problems related to tests with some python libraries, I'll see what's possible to be done there. In the meantime, it looks good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants