Skip to content

Conversation

@stevew817
Copy link
Contributor

@stevew817 stevew817 commented Jan 26, 2025

See ARM-software/LLVM-embedded-toolchain-for-Arm#628 for context.

This enables newlib-nano as a multilib. I didn't find any release scripts, so I don't know how to add it to the CI such that it also generates an overlay package for newlib-nano on release.

Note: this also fixes an issue with the current newlib builds where they don't get built correctly for anything above ARMv6 as the -march, -mfloat-abi and -mfpu arguments weren't being passed on to the newlib compilation configuration. A little unsure why this hasn't popped up before...

This is meant to reflect what ARM GCC provides when compiling with
--specs=nano.specs
newlib was not building correctly for ARMv7 and ARMv8 targets due to not
being passed the -march, -mfpu and -mfloat-abi flags which are required for
the compiler to generate correct assembly (it was outputing incorrect
assembly without triggering any warnings)
@stevew817 stevew817 force-pushed the feature/newlib-nano branch from 4435bff to c66cd73 Compare January 26, 2025 17:56
@voltur01
Copy link
Contributor

Hi Stephen,

Thank you for working on this and posting the patch!

I was playing with the changes, I can confirm that with this patch I can build the newlib-nano overlay package which works with the C sample https://github.com/arm/arm-toolchain/tree/arm-software/arm-software/embedded/newlib-samples/src/baremetal-semihosting-newlib (after adjusting the config file name in the Makefile), however the C++ sample https://github.com/arm/arm-toolchain/tree/arm-software/arm-software/embedded/newlib-samples/src/cpp-baremetal-semihosting-newlib reports missing symbols: ungetwc, fgetwc, swprintf. Is this expected for the nano variant?

Surprisingly, I am not able to build the normal newlib package anymore: newlib/libm/common/math_config.h gives multiple errors: couldn't allocate output register for constraint 'w' in __asm__ __volatile__ ("", : "+w" (x))

@stevew817
Copy link
Contributor Author

stevew817 commented Jan 28, 2025

Thanks for the swift feedback!

Is this expected for the nano variant?

I don't think so - according to the readme for newlib, enabling newlib-nano-formatted-io shouldn't impact any of the wchar-related functionality. But, when looking at what the newlib makefile is doing, it is actually dropping all wide-char-related functions from the library build when compiling with nano-formatted IO set. See here, line 298 - 335.
I guess that means the C++ libs accompanying nanolib need to be built without wchar too in order for that to not show? Does libstdc++ somehow account for that without a specific flag, and this is popping up here since we're using libc++ here? Or am I just not finding it in the ARMGCC build script over here? I see that picolibc is also compiling the C++ libs without wchar support, so that's probably the 'fix'.

Surprisingly, I am not able to build the normal newlib package anymore: newlib/libm/common/math_config.h gives multiple errors: couldn't allocate output register for constraint 'w' in asm volatile ("", : "+w" (x))

I'll admit I didn't retry building the regular newlib locally (as each build run takes about four hours for me... I do need a new machine). Since I didn't change any of the newlib config flags, it's probably due to now passing the actual architecture and architectural option flags to the compiler? Before I spend hours rebuilding newlib from scratch, so you have any more information on the failure (such as line number etc)?

@voltur01
Copy link
Contributor

Please see the newlib build log for one of the variants - looks like there is the same pattern of errors in multiple variants.

2025-01-27_newlib_buildlog.txt

@voltur01
Copy link
Contributor

LLVM libc++ seems to have this option to turn wide characters support on/off: https://github.com/llvm/llvm-project/blob/83433d936195c612a51b54397f82ab0d97369d86/libcxx/CMakeLists.txt#L98

@stevew817
Copy link
Contributor Author

Please see the newlib build log for one of the variants - looks like there is the same pattern of errors in multiple variants.

Thanks! Is it only on AArch64 or does it also appear on ARM?

@voltur01
Copy link
Contributor

I rerun the build, apparently it tries to build the first configuration which is aarch64a_soft_nofp_exn_rtti, fails and stops.

When I try to build armv6m and armv8.m-main separately, the build succeeds, so likely the above is an AArch64 specific issue.

Building newlibe with --enable-newlib-nano-formatted-io removes all support
for wchar_t functions from the C libary, meaning that the CPP library on
top also needs to not rely on the wchar variants.
It's incompatible, see
https://gitlab.arm.com/tooling/gnu-devtools-for-arm/-/blob/main/build-baremetal-toolchain.sh#L624
Instead, ship a size-optimized build of regular newlib as the 'newlib-nano'
multilib for AArch64.
Forces the build system to keep them separate and keeps us free from
having to think about reconfiguration issues when switching between library
builds.
@stevew817
Copy link
Contributor Author

Sorry for the delayed response, I was out on vacation.

After digging back into this, I came to the following:

  • AArch64 targets with forcibly-disabled FPU ("nofp") won't be able to compile newlib with LLVM. This is due to the fact that newlib ships a whole lot of assembly optimisations for AArch64, and a lot of them use the floating-point registers. GCC happily converts that under the hood it seems, but LLVM balks at it. Since AArch64 without FPU seems to be a bit of an academic case (are there any such real-life targets?), I suggest we don't build newlib/nanolib for nofp AArch64.
  • Nano specs as a whole aren't supported on AArch64 at all in ARMGCC, so I suggest we don't try to support it here either.
  • Newlib built with nano formatted I/O apparently doesn't provide any of the wchar stdlib functions, even though it should according to that flag's doc. To work around this (and in the spirit of "nano"), I am suggesting to build libc++ without wide character support when it is used in conjunction with newlib-nano. However, that uncovered an issue in libc++ where it needs mbstate_t to be declared. According to the C standard, that type should be declared in wchar.h, but if libc++ is compiled without wide character support it won't even try to include wchar.h (even though even newlib-nano provides it). I'm providing a patch for libc++ here to fix that. If this is the way forward, that patch should probably be upstreamed to LLVM.

@voltur01 can you check the changes I just pushed?

@voltur01
Copy link
Contributor

Hi Steven,

Thank you for the updated patch!

I can confirm that both newlib and newllib-nano packages build successfully and I can run newlib-samples with both.

Code change look good as well. As you rightly pointed out, the libcxx changes would be best done in upstream LLVM. Would you mind creating the pull request in LLVM, please?

If you take the libcxx change form this PR I can approve and merge it: we only build newlib package in our CI so it will keep working, we can add newlib-nano when the upstream libcxx change lands.

And newlib insists on using them for any AArch64 target. So for now don't
build newlib for AArch64 with software-disabled FPU.
@stevew817
Copy link
Contributor Author

Dropped the libc++ change from the PR as requested, and hopefully llvm/llvm-project#126924 will land instead. If it doesn't, we'll have to do a workaround similar to picolibc where we patch newlib with an extra header in <bits/types/mbstate_t.h>

@voltur01
Copy link
Contributor

Thank you! I will approve and merge now.

@voltur01 voltur01 merged commit 23f1d75 into arm:arm-software Feb 17, 2025
pratlucas pushed a commit to pratlucas/arm-toolchain that referenced this pull request Feb 28, 2025
See
ARM-software/LLVM-embedded-toolchain-for-Arm#628
for context.

This enables newlib-nano as a multilib. I didn't find any release
scripts, so I don't know how to add it to the CI such that it also
generates an overlay package for newlib-nano on release.

Note: this also fixes an issue with the current newlib builds where they
don't get built correctly for anything above ARMv6 as the `-march`,
`-mfloat-abi` and `-mfpu` arguments weren't being passed on to the
newlib compilation configuration. A little unsure why this hasn't popped
up before...
pratlucas pushed a commit that referenced this pull request Feb 28, 2025
See
ARM-software/LLVM-embedded-toolchain-for-Arm#628
for context.

This enables newlib-nano as a multilib. I didn't find any release
scripts, so I don't know how to add it to the CI such that it also
generates an overlay package for newlib-nano on release.

Note: this also fixes an issue with the current newlib builds where they
don't get built correctly for anything above ARMv6 as the `-march`,
`-mfloat-abi` and `-mfpu` arguments weren't being passed on to the
newlib compilation configuration. A little unsure why this hasn't popped
up before...
dcandler added a commit that referenced this pull request May 15, 2025
#314 should have only enabled
aarch64 nofp variants for LLVM-libc builds. However it also enabled the
variants for newlib builds, which were disabled since newlib is
similarly unable to build those variants (see
#60 (comment)).
While there is now a fix for the LLVM-libc build, there is yet no fix
for newlib so they need to remain disabled for that selection.

This patch sets the variants to only build for picolibc and llvmlibc,
but not newlib.
statham-arm added a commit that referenced this pull request May 28, 2025
PR #60, which introduced it, set the build flags `-g -Oz` for
newlib-nano (both libc and libcxx), and `-g -O2` for full newlib libcxx.
`-g` was probably left over from development: in our production builds
it's taking up a great deal of space, making the newlib-nano overlay
archive more like ¾Gb than ¼Gb.

I've reverted the entire flags change for full newlib libcxx, putting it
back to how it was before that PR. For newlib-nano, I've removed the
`-g`, but kept `-Oz`, on the basis that optimizing for small size still
seems sensible if someone's deliberately selecting the smallest version
of newlib.
llvm-sync bot pushed a commit that referenced this pull request Aug 6, 2025
… >= 4G (#151460)"

This reverts commit 600976f.

The test was crashing trying to access any element of the GPR struct.

(gdb) disas
Dump of assembler code for function _ZN7testing8internal11CmpHelperEQIyyEENS_15AssertionResultEPKcS4_RKT_RKT0_:
   0x00450afc <+0>:	push	{r4, r5, r6, r7, r8, r9, r10, r11, lr}
   0x00450b00 <+4>:	sub	sp, sp, #60	; 0x3c
   0x00450b04 <+8>:	ldr	r5, [sp, #96]	; 0x60
=> 0x00450b08 <+12>:	ldm	r3, {r4, r7}
   0x00450b0c <+16>:	ldm	r5, {r6, r9}
   0x00450b10 <+20>:	eor	r7, r7, r9
   0x00450b14 <+24>:	eor	r6, r4, r6
   0x00450b18 <+28>:	orrs	r7, r6, r7

(gdb) p/x r3
$3 = 0x3e300f6e

"However, load and store multiple instructions (LDM and STM) and load and store double-word (LDRD or STRD) must be aligned to at least a word boundary."

https://developer.arm.com/documentation/den0013/d/Porting/Alignment

>>> 0x3e300f6e % 4
2

Program received signal SIGBUS, Bus error.
0x00450b08 in testing::AssertionResult testing::internal::CmpHelperEQ<unsigned long long, unsigned long long>(char const*, char const*, unsigned long long const&, unsigned long long const&) ()

The struct is packed with 1 byte alignment, but it needs to start at an aligned address for us
to ldm from it. So I've done that with alignas.

Also fixed some compiler warnings in the test itself.
llvm-sync bot pushed a commit that referenced this pull request Aug 6, 2025
…RM64` if PC >= 4G (#151460)"

This reverts commit 600976f.

The test was crashing trying to access any element of the GPR struct.

(gdb) disas
Dump of assembler code for function _ZN7testing8internal11CmpHelperEQIyyEENS_15AssertionResultEPKcS4_RKT_RKT0_:
   0x00450afc <+0>:	push	{r4, r5, r6, r7, r8, r9, r10, r11, lr}
   0x00450b00 <+4>:	sub	sp, sp, #60	; 0x3c
   0x00450b04 <+8>:	ldr	r5, [sp, #96]	; 0x60
=> 0x00450b08 <+12>:	ldm	r3, {r4, r7}
   0x00450b0c <+16>:	ldm	r5, {r6, r9}
   0x00450b10 <+20>:	eor	r7, r7, r9
   0x00450b14 <+24>:	eor	r6, r4, r6
   0x00450b18 <+28>:	orrs	r7, r6, r7

(gdb) p/x r3
$3 = 0x3e300f6e

"However, load and store multiple instructions (LDM and STM) and load and store double-word (LDRD or STRD) must be aligned to at least a word boundary."

https://developer.arm.com/documentation/den0013/d/Porting/Alignment

>>> 0x3e300f6e % 4
2

Program received signal SIGBUS, Bus error.
0x00450b08 in testing::AssertionResult testing::internal::CmpHelperEQ<unsigned long long, unsigned long long>(char const*, char const*, unsigned long long const&, unsigned long long const&) ()

The struct is packed with 1 byte alignment, but it needs to start at an aligned address for us
to ldm from it. So I've done that with alignas.

Also fixed some compiler warnings in the test itself.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants