[libc++] Cleanly implement the base locale API for BSD-like platforms #115176

ldionne · 2024-11-06T15:39:26Z

Instead of going through the old locale entry points, define the base localization API for BSD-like platforms (Apple and FreeBSD) from scratch, using <xlocale.h> as a basis. This doesn't actually change how that functionality is implemented, it only avoids going through a maze to do so.

This clean new support is implemented in a separate __locale_dir/support directory, which mirrors what we do for the threading support API. Eventually, everything under __locale_dir/locale_base_api will go away.

rdar://131476632

Instead of going through the old locale entry points, define the base localization API for BSD-like platforms (Apple and FreeBSD) from scratch, using <xlocale.h> as a basis. This doesn't actually change how that functionality is implemented, it only avoids going through a maze to do so. This patch also introduces generic headers that can be mixed-and-matched to implement platform support for localization, which is going to be useful because e.g. Windows does provide a few BSD-like functions that it can reuse. This clean new support is implemented in a separate __locale_dir/support directory, which mirrors what we do for the threading support API. Eventually, everything under __locale_dir/locale_base_api will go away. rdar://131476632

llvmbot · 2024-11-06T15:40:01Z

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

Instead of going through the old locale entry points, define the base localization API for BSD-like platforms (Apple and FreeBSD) from scratch, using <xlocale.h> as a basis. This doesn't actually change how that functionality is implemented, it only avoids going through a maze to do so.

This patch also introduces generic headers that can be mixed-and-matched to implement platform support for localization, which is going to be useful because e.g. Windows does provide a few BSD-like functions that it can reuse.

This clean new support is implemented in a separate __locale_dir/support directory, which mirrors what we do for the threading support API. Eventually, everything under __locale_dir/locale_base_api will go away.

rdar://131476632

Patch is 25.19 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/115176.diff

12 Files Affected:

(modified) libcxx/include/CMakeLists.txt (+6-2)
(modified) libcxx/include/__locale_dir/locale_base_api.h (+46-42)
(removed) libcxx/include/__locale_dir/locale_base_api/apple.h (-23)
(removed) libcxx/include/__locale_dir/locale_base_api/freebsd.h (-23)
(added) libcxx/include/__locale_dir/support/apple.h (+24)
(added) libcxx/include/__locale_dir/support/characters/bsd_like.h (+93)
(added) libcxx/include/__locale_dir/support/freebsd.h (+24)
(added) libcxx/include/__locale_dir/support/management/bsd_like.h (+39)
(added) libcxx/include/__locale_dir/support/other/bsd_like.h (+101)
(added) libcxx/include/__locale_dir/support/strtonum/bsd_like.h (+46)
(modified) libcxx/include/iomanip (+1)
(modified) libcxx/include/module.modulemap (+18-3)

diff --git a/libcxx/include/CMakeLists.txt b/libcxx/include/CMakeLists.txt
index 87eaf64b245017..3d9fad72f1ff3e 100644
--- a/libcxx/include/CMakeLists.txt
+++ b/libcxx/include/CMakeLists.txt
@@ -494,16 +494,20 @@ set(files
   __locale
   __locale_dir/locale_base_api.h
   __locale_dir/locale_base_api/android.h
-  __locale_dir/locale_base_api/apple.h
   __locale_dir/locale_base_api/bsd_locale_defaults.h
   __locale_dir/locale_base_api/bsd_locale_fallbacks.h
-  __locale_dir/locale_base_api/freebsd.h
   __locale_dir/locale_base_api/fuchsia.h
   __locale_dir/locale_base_api/ibm.h
   __locale_dir/locale_base_api/musl.h
   __locale_dir/locale_base_api/openbsd.h
   __locale_dir/locale_base_api/win32.h
   __locale_dir/locale_guard.h
+  __locale_dir/support/apple.h
+  __locale_dir/support/characters/bsd_like.h
+  __locale_dir/support/freebsd.h
+  __locale_dir/support/management/bsd_like.h
+  __locale_dir/support/other/bsd_like.h
+  __locale_dir/support/strtonum/bsd_like.h
   __math/abs.h
   __math/copysign.h
   __math/error_functions.h
diff --git a/libcxx/include/__locale_dir/locale_base_api.h b/libcxx/include/__locale_dir/locale_base_api.h
index e798a006625b59..5cbe91207ca74e 100644
--- a/libcxx/include/__locale_dir/locale_base_api.h
+++ b/libcxx/include/__locale_dir/locale_base_api.h
@@ -94,42 +94,44 @@
 //  int     __sscanf(const char*, __locale_t, const char*, ...);
 // }
 
+#if defined(__APPLE__)
+#  include <__locale_dir/support/apple.h>
+#elif defined(__FreeBSD__)
+#  include <__locale_dir/support/freebsd.h>
+#else
+
 // TODO: This is a temporary definition to bridge between the old way we defined the locale base API
 //       (by providing global non-reserved names) and the new API. As we move individual platforms
 //       towards the new way of defining the locale base API, this should disappear since each platform
 //       will define those directly.
-#if defined(_LIBCPP_MSVCRT_LIKE)
-#  include <__locale_dir/locale_base_api/win32.h>
-#elif defined(_AIX) || defined(__MVS__)
-#  include <__locale_dir/locale_base_api/ibm.h>
-#elif defined(__ANDROID__)
-#  include <__locale_dir/locale_base_api/android.h>
-#elif defined(__OpenBSD__)
-#  include <__locale_dir/locale_base_api/openbsd.h>
-#elif defined(__Fuchsia__)
-#  include <__locale_dir/locale_base_api/fuchsia.h>
-#elif defined(__wasi__) || _LIBCPP_HAS_MUSL_LIBC
-#  include <__locale_dir/locale_base_api/musl.h>
-#elif defined(__APPLE__)
-#  include <__locale_dir/locale_base_api/apple.h>
-#elif defined(__FreeBSD__)
-#  include <__locale_dir/locale_base_api/freebsd.h>
-#endif
+#  if defined(_LIBCPP_MSVCRT_LIKE)
+#    include <__locale_dir/locale_base_api/win32.h>
+#  elif defined(_AIX) || defined(__MVS__)
+#    include <__locale_dir/locale_base_api/ibm.h>
+#  elif defined(__ANDROID__)
+#    include <__locale_dir/locale_base_api/android.h>
+#  elif defined(__OpenBSD__)
+#    include <__locale_dir/locale_base_api/openbsd.h>
+#  elif defined(__Fuchsia__)
+#    include <__locale_dir/locale_base_api/fuchsia.h>
+#  elif defined(__wasi__) || _LIBCPP_HAS_MUSL_LIBC
+#    include <__locale_dir/locale_base_api/musl.h>
+#  endif
 
-#ifdef _LIBCPP_LOCALE__L_EXTENSIONS
-#  include <__locale_dir/locale_base_api/bsd_locale_defaults.h>
-#else
-#  include <__locale_dir/locale_base_api/bsd_locale_fallbacks.h>
-#endif
+#  ifdef _LIBCPP_LOCALE__L_EXTENSIONS
+#    include <__locale_dir/locale_base_api/bsd_locale_defaults.h>
+#  else
+#    include <__locale_dir/locale_base_api/bsd_locale_fallbacks.h>
+#  endif
 
-#include <__cstddef/size_t.h>
-#include <__utility/forward.h>
-#include <ctype.h>
-#include <string.h>
-#include <time.h>
-#if _LIBCPP_HAS_WIDE_CHARACTERS
-#  include <wctype.h>
-#endif
+#  include <__cstddef/size_t.h>
+#  include <__utility/forward.h>
+#  include <ctype.h>
+#  include <string.h>
+#  include <time.h>
+#  if _LIBCPP_HAS_WIDE_CHARACTERS
+#    include <wctype.h>
+#  endif
 _LIBCPP_BEGIN_NAMESPACE_STD
 namespace __locale {
 //
@@ -137,9 +139,9 @@ namespace __locale {
 //
 using __locale_t = locale_t;
 
-#ifndef _LIBCPP_MSVCRT_LIKE
+#  ifndef _LIBCPP_MSVCRT_LIKE
 inline _LIBCPP_HIDE_FROM_ABI __locale_t __uselocale(__locale_t __loc) { return uselocale(__loc); }
-#endif
+#  endif
 
 inline _LIBCPP_HIDE_FROM_ABI __locale_t __newlocale(int __category_mask, const char* __name, __locale_t __loc) {
   return newlocale(__category_mask, __name, __loc);
@@ -189,7 +191,7 @@ inline _LIBCPP_HIDE_FROM_ABI size_t __strxfrm(char* __dest, const char* __src, s
 inline _LIBCPP_HIDE_FROM_ABI int __toupper(int __ch, __locale_t __loc) { return toupper_l(__ch, __loc); }
 inline _LIBCPP_HIDE_FROM_ABI int __tolower(int __ch, __locale_t __loc) { return tolower_l(__ch, __loc); }
 
-#if _LIBCPP_HAS_WIDE_CHARACTERS
+#  if _LIBCPP_HAS_WIDE_CHARACTERS
 inline _LIBCPP_HIDE_FROM_ABI int __wcscoll(const wchar_t* __s1, const wchar_t* __s2, __locale_t __loc) {
   return wcscoll_l(__s1, __s2, __loc);
 }
@@ -208,7 +210,7 @@ inline _LIBCPP_HIDE_FROM_ABI int __iswpunct(wint_t __ch, __locale_t __loc) { ret
 inline _LIBCPP_HIDE_FROM_ABI int __iswxdigit(wint_t __ch, __locale_t __loc) { return iswxdigit_l(__ch, __loc); }
 inline _LIBCPP_HIDE_FROM_ABI wint_t __towupper(wint_t __ch, __locale_t __loc) { return towupper_l(__ch, __loc); }
 inline _LIBCPP_HIDE_FROM_ABI wint_t __towlower(wint_t __ch, __locale_t __loc) { return towlower_l(__ch, __loc); }
-#endif
+#  endif
 
 inline _LIBCPP_HIDE_FROM_ABI size_t
 __strftime(char* __s, size_t __max, const char* __format, const tm* __tm, __locale_t __loc) {
@@ -221,7 +223,7 @@ __strftime(char* __s, size_t __max, const char* __format, const tm* __tm, __loca
 inline _LIBCPP_HIDE_FROM_ABI decltype(__libcpp_mb_cur_max_l(__locale_t())) __mb_len_max(__locale_t __loc) {
   return __libcpp_mb_cur_max_l(__loc);
 }
-#if _LIBCPP_HAS_WIDE_CHARACTERS
+#  if _LIBCPP_HAS_WIDE_CHARACTERS
 inline _LIBCPP_HIDE_FROM_ABI wint_t __btowc(int __ch, __locale_t __loc) { return __libcpp_btowc_l(__ch, __loc); }
 inline _LIBCPP_HIDE_FROM_ABI int __wctob(wint_t __ch, __locale_t __loc) { return __libcpp_wctob_l(__ch, __loc); }
 inline _LIBCPP_HIDE_FROM_ABI size_t
@@ -249,16 +251,16 @@ inline _LIBCPP_HIDE_FROM_ABI size_t
 __mbsrtowcs(wchar_t* __dest, const char** __src, size_t __len, mbstate_t* __ps, __locale_t __loc) {
   return __libcpp_mbsrtowcs_l(__dest, __src, __len, __ps, __loc);
 }
-#endif
+#  endif
 
 _LIBCPP_DIAGNOSTIC_PUSH
 _LIBCPP_CLANG_DIAGNOSTIC_IGNORED("-Wgcc-compat")
 _LIBCPP_GCC_DIAGNOSTIC_IGNORED("-Wformat-nonliteral") // GCC doesn't support [[gnu::format]] on variadic templates
-#ifdef _LIBCPP_COMPILER_CLANG_BASED
-#  define _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(...) _LIBCPP_ATTRIBUTE_FORMAT(__VA_ARGS__)
-#else
-#  define _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(...) /* nothing */
-#endif
+#  ifdef _LIBCPP_COMPILER_CLANG_BASED
+#    define _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(...) _LIBCPP_ATTRIBUTE_FORMAT(__VA_ARGS__)
+#  else
+#    define _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(...) /* nothing */
+#  endif
 
 template <class... _Args>
 _LIBCPP_HIDE_FROM_ABI _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(__printf__, 4, 5) int __snprintf(
@@ -276,9 +278,11 @@ _LIBCPP_HIDE_FROM_ABI _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT(__scanf__, 3, 4) int __s
   return __libcpp_sscanf_l(__s, __loc, __format, std::forward<_Args>(__args)...);
 }
 _LIBCPP_DIAGNOSTIC_POP
-#undef _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT
+#  undef _LIBCPP_VARIADIC_ATTRIBUTE_FORMAT
 
 } // namespace __locale
 _LIBCPP_END_NAMESPACE_STD
 
+#endif // Compatibility definition of locale base APIs
+
 #endif // _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_H
diff --git a/libcxx/include/__locale_dir/locale_base_api/apple.h b/libcxx/include/__locale_dir/locale_base_api/apple.h
deleted file mode 100644
index 69faa260be2190..00000000000000
--- a/libcxx/include/__locale_dir/locale_base_api/apple.h
+++ /dev/null
@@ -1,23 +0,0 @@
-// -*- C++ -*-
-//===-----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_APPLE_H
-#define _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_APPLE_H
-
-#include <__config>
-#include <ctype.h>
-#include <string.h>
-#include <time.h>
-#if _LIBCPP_HAS_WIDE_CHARACTERS
-#  include <wctype.h>
-#endif
-
-#include <xlocale.h>
-
-#endif // _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_APPLE_H
diff --git a/libcxx/include/__locale_dir/locale_base_api/freebsd.h b/libcxx/include/__locale_dir/locale_base_api/freebsd.h
deleted file mode 100644
index cf683d14d16880..00000000000000
--- a/libcxx/include/__locale_dir/locale_base_api/freebsd.h
+++ /dev/null
@@ -1,23 +0,0 @@
-// -*- C++ -*-
-//===-----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_FREEBSD_H
-#define _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_FREEBSD_H
-
-#include <__config>
-#include <ctype.h>
-#include <string.h>
-#include <time.h>
-#if _LIBCPP_HAS_WIDE_CHARACTERS
-#  include <wctype.h>
-#endif
-
-#include <xlocale.h>
-
-#endif // _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_FREEBSD_H
diff --git a/libcxx/include/__locale_dir/support/apple.h b/libcxx/include/__locale_dir/support/apple.h
new file mode 100644
index 00000000000000..42307aa9e8cb91
--- /dev/null
+++ b/libcxx/include/__locale_dir/support/apple.h
@@ -0,0 +1,24 @@
+//===-----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_APPLE_H
+#define _LIBCPP___LOCALE_DIR_SUPPORT_APPLE_H
+
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+#include <__locale_dir/support/management/bsd_like.h> // must come first since it defines __locale_t
+
+#include <__locale_dir/support/characters/bsd_like.h>
+#include <__locale_dir/support/other/bsd_like.h>
+#include <__locale_dir/support/strtonum/bsd_like.h>
+
+#endif // _LIBCPP___LOCALE_DIR_SUPPORT_APPLE_H
diff --git a/libcxx/include/__locale_dir/support/characters/bsd_like.h b/libcxx/include/__locale_dir/support/characters/bsd_like.h
new file mode 100644
index 00000000000000..3795c65457ae1e
--- /dev/null
+++ b/libcxx/include/__locale_dir/support/characters/bsd_like.h
@@ -0,0 +1,93 @@
+//===-----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_CHARACTERS_BSD_LIKE_H
+#define _LIBCPP___LOCALE_DIR_SUPPORT_CHARACTERS_BSD_LIKE_H
+
+#include <__config>
+#include <ctype.h>
+#include <stdlib.h>
+#include <string.h>
+#include <time.h>
+#if _LIBCPP_HAS_WIDE_CHARACTERS
+#  include <wchar.h>
+#  include <wctype.h>
+#endif
+
+#include <xlocale.h>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+namespace __locale {
+
+inline _LIBCPP_HIDE_FROM_ABI int __islower(int __c, __locale_t __loc) { return ::islower_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __isupper(int __c, __locale_t __loc) { return ::isupper_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __isdigit(int __c, __locale_t __loc) { return ::isdigit_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __isxdigit(int __c, __locale_t __loc) { return ::isxdigit_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __toupper(int __c, __locale_t __loc) { return ::toupper_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __tolower(int __c, __locale_t __loc) { return ::tolower_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __strcoll(const char* __s1, const char* __s2, __locale_t __loc) {
+  return ::strcoll_l(__s1, __s2, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t __strxfrm(char* __dest, const char* __src, size_t __n, __locale_t __loc) {
+  return ::strxfrm_l(__dest, __src, __n, __loc);
+}
+
+#if _LIBCPP_HAS_WIDE_CHARACTERS
+inline _LIBCPP_HIDE_FROM_ABI int __iswspace(wint_t __c, __locale_t __loc) { return ::iswspace_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswprint(wint_t __c, __locale_t __loc) { return ::iswprint_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswcntrl(wint_t __c, __locale_t __loc) { return ::iswcntrl_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswupper(wint_t __c, __locale_t __loc) { return ::iswupper_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswlower(wint_t __c, __locale_t __loc) { return ::iswlower_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswalpha(wint_t __c, __locale_t __loc) { return ::iswalpha_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswblank(wint_t __c, __locale_t __loc) { return ::iswblank_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswdigit(wint_t __c, __locale_t __loc) { return ::iswdigit_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswpunct(wint_t __c, __locale_t __loc) { return ::iswpunct_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __iswxdigit(wint_t __c, __locale_t __loc) { return ::iswxdigit_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI wint_t __towupper(wint_t __c, __locale_t __loc) { return ::towupper_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI wint_t __towlower(wint_t __c, __locale_t __loc) { return ::towlower_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __wcscoll(const wchar_t* __ws1, const wchar_t* __ws2, __locale_t __loc) {
+  return ::wcscoll_l(__ws1, __ws2, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t __wcsxfrm(wchar_t* __dest, const wchar_t* __src, size_t __n, __locale_t __loc) {
+  return ::wcsxfrm_l(__dest, __src, __n, __loc);
+}
+#endif // _LIBCPP_HAS_WIDE_CHARACTERS
+
+inline _LIBCPP_HIDE_FROM_ABI size_t
+__strftime(char* __s, size_t __max, const char* __format, const struct tm* __tm, __locale_t __loc) {
+  return ::strftime_l(__s, __max, __format, __tm, __loc);
+}
+
+} // namespace __locale
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___LOCALE_DIR_SUPPORT_CHARACTERS_BSD_LIKE_H
diff --git a/libcxx/include/__locale_dir/support/freebsd.h b/libcxx/include/__locale_dir/support/freebsd.h
new file mode 100644
index 00000000000000..c91b7e8c8e082d
--- /dev/null
+++ b/libcxx/include/__locale_dir/support/freebsd.h
@@ -0,0 +1,24 @@
+//===-----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_FREEBSD_H
+#define _LIBCPP___LOCALE_DIR_SUPPORT_FREEBSD_H
+
+#include <__config>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+#include <__locale_dir/support/management/bsd_like.h> // must come first since it defines __locale_t
+
+#include <__locale_dir/support/characters/bsd_like.h>
+#include <__locale_dir/support/other/bsd_like.h>
+#include <__locale_dir/support/strtonum/bsd_like.h>
+
+#endif // _LIBCPP___LOCALE_DIR_SUPPORT_FREEBSD_H
diff --git a/libcxx/include/__locale_dir/support/management/bsd_like.h b/libcxx/include/__locale_dir/support/management/bsd_like.h
new file mode 100644
index 00000000000000..d4a0dafeeca05f
--- /dev/null
+++ b/libcxx/include/__locale_dir/support/management/bsd_like.h
@@ -0,0 +1,39 @@
+//===-----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_MANAGEMENT_BSD_LIKE_H
+#define _LIBCPP___LOCALE_DIR_SUPPORT_MANAGEMENT_BSD_LIKE_H
+
+#include <__config>
+#include <clocale> // std::lconv
+
+#include <xlocale.h> // uselocale() and others
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+namespace __locale {
+
+using __locale_t = ::locale_t;
+
+inline _LIBCPP_HIDE_FROM_ABI __locale_t __uselocale(__locale_t __loc) { return ::uselocale(__loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI __locale_t __newlocale(int __category_mask, const char* __locale, __locale_t __base) {
+  return ::newlocale(__category_mask, __locale, __base);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI void __freelocale(__locale_t __loc) { ::freelocale(__loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI lconv* __localeconv(__locale_t& __loc) { return ::localeconv_l(__loc); }
+
+} // namespace __locale
+_LIBCPP_END_NAMESPACE_STD
+
+#endif // _LIBCPP___LOCALE_DIR_SUPPORT_MANAGEMENT_BSD_LIKE_H
diff --git a/libcxx/include/__locale_dir/support/other/bsd_like.h b/libcxx/include/__locale_dir/support/other/bsd_like.h
new file mode 100644
index 00000000000000..a4cb5417553685
--- /dev/null
+++ b/libcxx/include/__locale_dir/support/other/bsd_like.h
@@ -0,0 +1,101 @@
+//===-----------------------------------------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_OTHER_BSD_LIKE_H
+#define _LIBCPP___LOCALE_DIR_SUPPORT_OTHER_BSD_LIKE_H
+
+#include <__config>
+#include <__cstddef/size_t.h>
+#include <__std_mbstate_t.h>
+#include <__utility/forward.h>
+#if _LIBCPP_HAS_WIDE_CHARACTERS
+#  include <wchar.h>
+#endif
+
+#include <xlocale.h>
+
+#if !defined(_LIBCPP_HAS_NO_PRAGMA_SYSTEM_HEADER)
+#  pragma GCC system_header
+#endif
+
+_LIBCPP_BEGIN_NAMESPACE_STD
+namespace __locale {
+
+inline _LIBCPP_HIDE_FROM_ABI decltype(MB_CUR_MAX) __mb_len_max(__locale_t __loc) { return MB_CUR_MAX_L(__loc); }
+
+#if _LIBCPP_HAS_WIDE_CHARACTERS
+inline _LIBCPP_HIDE_FROM_ABI wint_t __btowc(int __c, __locale_t __loc) { return ::btowc_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI int __wctob(wint_t __c, __locale_t __loc) { return ::wctob_l(__c, __loc); }
+
+inline _LIBCPP_HIDE_FROM_ABI size_t
+__wcsnrtombs(char* __dest, const wchar_t** __src, size_t __nwc, size_t __len, mbstate_t* __ps, __locale_t __loc) {
+  return ::wcsnrtombs_l(__dest, __src, __nwc, __len, __ps, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t __wcrtomb(char* __s, wchar_t __wc, mbstate_t* __ps, __locale_t __loc) {
+  return ::wcrtomb_l(__s, __wc, __ps, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t
+__mbsnrtowcs(wchar_t* __dest, const char** __src, size_t __nms, size_t __len, mbstate_t* __ps, __locale_t __loc) {
+  return ::mbsnrtowcs_l(__dest, __src, __nms, __len, __ps, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t
+__mbrtowc(wchar_t* __pwc, const char* __s, size_t __n, mbstate_t* __ps, __locale_t __loc) {
+  return ::mbrtowc_l(__pwc, __s, __n, __ps, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI int __mbtowc(wchar_t* __pwc, const char* __pmb, size_t __max, __locale_t __loc) {
+  return ::mbtowc_l(__pwc, __pmb, __max, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t __mbrlen(const char* __s, size_t __n, mbstate_t* __ps, __locale_t __loc) {
+  return ::mbrlen_l(__s, __n, __ps, __loc);
+}
+
+inline _LIBCPP_HIDE_FROM_ABI size_t
+__mbsrtowcs(wchar_t* __dest, const char** __src, size_t __len, mbstate_t* __ps, __locale_t __loc) {
+  return ::mbsrtowcs_l(__d...
[truncated]

ldionne · 2024-11-06T15:54:51Z

It turns out that I had gotten confused, and Windows doesn't provide any of the BSD-like functions. This was such a maze. I think it makes sense to merge the 4 bsd_like.h headers into one in that case -- I'll update the PR soon.

philnik777

Basically LGTM. Just two questions.

philnik777 · 2024-11-11T17:45:56Z

libcxx/include/__locale_dir/support/apple.h

@@ -7,17 +6,15 @@
 //
 //===----------------------------------------------------------------------===//

-#ifndef _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_APPLE_H
-#define _LIBCPP___LOCALE_DIR_LOCALE_BASE_API_APPLE_H
+#ifndef _LIBCPP___LOCALE_DIR_SUPPORT_APPLE_H


Why are you introducing a new directory here?

I want to move towards the following organization, which is what we do for the threading support:

__locale/ support.h support/ apple.h freebsd.h windows.h etc..

So basically, I'm writing the "new" code under that organization and eventually the old stuff can be removed since it will be unused.

philnik777 · 2024-11-11T17:46:40Z

libcxx/include/iomanip

@@ -49,6 +49,7 @@ template <class charT, class traits, class Allocator>
 #  include <__ostream/basic_ostream.h>
 #  include <ios>
 #  include <istream>
+#  include <locale>


What does <iomanip> require from <locale>?

Things like money_get, time_get and friends.

…llvm#115176) Instead of going through the old locale entry points, define the base localization API for BSD-like platforms (Apple and FreeBSD) from scratch, using <xlocale.h> as a basis. This doesn't actually change how that functionality is implemented, it only avoids going through a maze to do so. This clean new support is implemented in a separate __locale_dir/support directory, which mirrors what we do for the threading support API. Eventually, everything under __locale_dir/locale_base_api will go away. rdar://131476632

This updates libcxx and libcxxabi to LLVM 20.1.4: https://github.com/llvm/llvm-project/releases/tag/llvmorg-20.1.4 Before going into each additional change of the PR, these are some noteworthy changes from this LLVM release: - Freezing C++03 headers - RFC: https://discourse.llvm.org/t/rfc-freezing-c-03-headers-in-libc/77319 - PRs: - llvm/llvm-project#108999 - llvm/llvm-project#109000 - llvm/llvm-project#109001 - llvm/llvm-project#109002 - This copies libc++ headers of the last LLVM 19 release into a separate directory (https://github.com/llvm/llvm-project/tree/main/libcxx/include/__cxx03) and redirects all C++03 workflow there. The motivation is not to fix C++03 related changes unless they are critical bugs, and simplifies the main headers. So the main headers are like ``` #if __cplusplus < 201103L && defined(_LIBCPP_USE_FROZEN_CXX03_HEADERS) # include <__cxx03/algorithm> #else ... main header content ... ``` As you can see it looks like we can avoid opting into this by not defining `_LIBCPP_USE_FROZEN_CXX03_HEADERS` at the moment. I think their eventual goal is to remove C++03 support from the main headers but it hasn't seem to have happened yet and I don't know what their timeline for that is. Adding that `__cxx03` directory increases the header size by 10MB. We can get away not using them by not defining `_LIBCPP_USE_FROZEN_CXX03_HEADERS` in this release, so this update does not include that directory. - Sharing of headers between libc++ and libc (Project Hand in Hand) - RFC: https://discourse.llvm.org/t/rfc-project-hand-in-hand-llvm-libc-libc-code-sharing/77701 - PR: llvm/llvm-project#91651 (More are likely to come) This tries to share some libc headers from libcxx. libcxx's source files can depend on headers from `libc/shared`, `libc/src/__support`, `libc/include/llvm-libc-macros`, and `libc/include/llvm-libc-types`. And it turns out libcxx can also depend on `libc/hdr`, even though the directory is not in the diagram in the RFC. These headers can be only included from `libcxx/src` and not `libcxx/include`, to prevent libcxx's API from changing. So these headers don't need to be copied into `cache/`. - Locale API reimplementation It doesn't seem to have an RFC, but the PRs are: - llvm/llvm-project#113737 - llvm/llvm-project#115176 - llvm/llvm-project#115752 - llvm/llvm-project#122489 It looks this aims provide a new way to define locale API for each platform. Currently Apple, FreeBSD, MSVCRT, and Fuschia are using the new API and the others are still using the old one: https://github.com/llvm/llvm-project/blob/ec28b8f9cc7f2ac187d8a617a6d08d5e56f9120e/libcxx/include/__locale_dir/locale_base_api.h#L116-L138 For our purpose, adding `if defined(__EMSCRIPTEN__)` entry to the list of "old" list seems to work for the moment, but we may need to move to the new way eventually later. --- Additional changes: - Copy `vendor/llvm/default_assertion_handler.in` to `__assertion_handler`: aa53648 Our previous `__assertion_handler` was copied from `libcxx/vendor/llvm/default_assertion_handler.in`. This updates our `__assertion_handler` to the new `libcxx/vendor/llvm/default_assertion_handler.in`. For what this file is, see the PR description of #22994, with which the file was added. - Remove `libcxx/include/__cxx03` directory: d646f6b As I described in "Freezing C++ headers" above, C++03 headers were copied to `include/cxx03` to be "frozen". But by not defining `_LIBCPP_USE_FROZEN_CXX03_HEADERS` we can still use the main headers. This new `__cxx03` header directory is almost 10M and we are not using it in this update, this deletes it. - Define more variables in `__config_site`: 9912236 libcxx decides to almost all configuration macros to be defined. So before it tested whether a macro was defined/undefined, and now it assumes it is defined and tests whether its value is 1/0. Before llvm/llvm-project#112094 `_config_site.in` used to use `#cmakedefine`, which only defined variables when they were enabled, but now it uses `#cmakedefine01`, which always defines variables and assigns 1 or 0 depending on whether the feature is enabled. So this change adds `#define` to each variable that `_config_site.in` has an entry of. The value assigned is 1 if it was defined in our previous Emscripten environment and 0 if not. - Fix Emscripten's locale: 2ae01b0 Before, the code was like https://github.com/emscripten-core/emscripten/blob/dc1abd514b1bade135a01a4453a9ff6def0793b6/system/lib/libcxx/include/__locale_dir/locale_base_api.h#L12-L30 But now they are divided into two parts, one using the new API and the other using old. See "Locale API reimplementation" above. https://github.com/llvm/llvm-project/blob/ec28b8f9cc7f2ac187d8a617a6d08d5e56f9120e/libcxx/include/__locale_dir/locale_base_api.h#L116-L138 This adds Emscripten in the beginning of the "old" API list. (This has to be the beginning; see #23414) - Import libc headers used by libcxx: 12a4ee4, 12a4ee4 and 43c8ce4 This imports a part of libc headers into `system/lib/llvm-libc`. The imported directories are: - `libc/shared` - `libc/src/__support` - `libc/include/llvm-libc-macros` - `libc/include/llvm-libc-types` - `libc/hdr` See "sharing of headers between libc+ and libc" above for details. This also applies llvm/llvm-project#133999, which is a bugfix that has not be backported, which fixes the bug of including from a wrong directory. - `std::uncaught_exception` -> `std::uncaught_exceptions`: 1bf4e78 `std::uncaught_exception` has been deprecated in C++17, so it generates a warning (which we treat as an error). Replaced it with `std::uncaught_exceptions`, which returns the number of uncaught exceptions (but can still be used as a boolean in the test). - Remove `ryu_constants.h` and `ryu_long_double_constants.h` from libc: 5767ac4 These are not used from libcxx and `ryu_long_double_constants.h` is huge (12M).

ldionne requested a review from a team as a code owner November 6, 2024 15:39

llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Nov 6, 2024

ldionne added 2 commits November 6, 2024 12:19

Merge all the BSD-like headers since they can't be reused on Windows

0791507

Add missing includes

9a83bb6

philnik777 reviewed Nov 11, 2024

View reviewed changes

philnik777 approved these changes Nov 11, 2024

View reviewed changes

ldionne merged commit 9c3a7ad into llvm:main Nov 11, 2024
64 checks passed

ldionne deleted the review/locale-7-extract-apple-and-freebsd branch November 11, 2024 18:11

aheejin mentioned this pull request May 16, 2025

Update libcxx and libcxxabi to LLVM 20.1.4 emscripten-core/emscripten#24346

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[libc++] Cleanly implement the base locale API for BSD-like platforms #115176

[libc++] Cleanly implement the base locale API for BSD-like platforms #115176

Uh oh!

ldionne commented Nov 6, 2024 •

edited

Loading

Uh oh!

llvmbot commented Nov 6, 2024

Uh oh!

ldionne commented Nov 6, 2024

Uh oh!

philnik777 left a comment

Uh oh!

philnik777 Nov 11, 2024

Uh oh!

ldionne Nov 11, 2024

Uh oh!

ldionne Nov 11, 2024

Uh oh!

philnik777 Nov 11, 2024

Uh oh!

ldionne Nov 11, 2024 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

[libc++] Cleanly implement the base locale API for BSD-like platforms #115176

[libc++] Cleanly implement the base locale API for BSD-like platforms #115176

Uh oh!

Conversation

ldionne commented Nov 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

llvmbot commented Nov 6, 2024

Uh oh!

ldionne commented Nov 6, 2024

Uh oh!

philnik777 left a comment

Choose a reason for hiding this comment

Uh oh!

philnik777 Nov 11, 2024

Choose a reason for hiding this comment

Uh oh!

ldionne Nov 11, 2024

Choose a reason for hiding this comment

Uh oh!

ldionne Nov 11, 2024

Choose a reason for hiding this comment

Uh oh!

philnik777 Nov 11, 2024

Choose a reason for hiding this comment

Uh oh!

ldionne Nov 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ldionne commented Nov 6, 2024 •

edited

Loading

ldionne Nov 11, 2024 •

edited

Loading