Q: runtime pal.h declarations

[tmds]

When updating versions of clang we had some compiler errors due to mismatch in declarations between pal.h and system headers (mainly: missing throw specifiers). We can fix these issues by making pal.h match with the system headers.

What is the rationale for having these declaration in pal.h, instead of including system headers?

cc @jkotas @janvorli @omajid

[omajid]

For example, this is the sort of error we ran into with clang 10: #32837

[jkotas]

The PAL is trying to emulate Windows for historic reasons. It made the initial port to Unix easy. Definitions of number of functions on Windows are different from equivalent definitions on Unix. It means that we cannot just include system headers because of the definitions would collide.

Ideally, we would make this Windows emulator go away. A step in this direction would be to define PAL_STDCPP_COMPAT always and make sure that everything still builds and works. It would enable us to include system headers instead of duplicating selected parts of them.

[janvorli]

The runtime code on Unix never includes anything from the actual platform, just the pal.h that provides all the functions the runtime depends on. That includes emulated Windows APIs, Windows data types, some extra PAL APIs that don't match windows ones (names starting with PAL_) and some standard C functions like various printf / scanf etc.

The code in PAL implementation also includes the pal.h since it needs to use the Windows data types or define the functions declared in the pal.h. However, the pal.h is not included directly, but rather through palinternal.h that besides other things also handles conflicting function and type names between the Windows and Unix worlds. There are details on that in comments at the beginning of the palinternal.h.

The throw specifier case is kind of special as it needs to be on the function declarations in the pal.h and since these are included from both PAL and runtime, they need to match what the platform uses.

[tmds]

A step in this direction would be to define PAL_STDCPP_COMPAT always and make sure that everything still builds and works.

It would be interesting to give this a try to avoid these compiler issues in the future.

[tmds]

I experimented with this a bit, and came to a better understanding of what type things PAL deals with. 2 examples:

• wchar_t is 2-byte utf-16 on Windows, and 4-byte 'unspecified encoding' on Linux. Since strings in managed .NET are UTF-16 at some point you need native functions that work with UTF-16, and those will be distinct, on non-Windows, from system functions that accept wchar_t.

• There are some Windows-specific sal annotations on buffers using __in and __out:
  

  runtime/src/coreclr/src/pal/inc/rt/sal.h

  Lines 2658 to 2659 in db70edd

  	#define __in _SAL1_Source_(__in, (), _In_)
  	#define __out _SAL1_Source_(__out, (), _Out_)

  
  Some Linux headers use __in and __out as identifiers, so they can't be defined to disappear.

All such cases can be death with, though the effort is higher than what I had imagined.

[yowl]

I'm confused with the wchar_t problem on Linux. If you have PAL_STDCPP_COMPAT then you dont get the, for example, PAL_fwprintf implementation as its within a #ifndef PAL_STDCPP_COMPAT in pal.h What's the solution here, always substitute fwprintf and other wide char functions on Linux?

[jkotas]

#define fwprintf PAL_fwprintf is within #ifndef PAL_STDCPP_COMPAT. PAL_fwprintf is outside #ifndef PAL_STDCPP_COMPAT. You should be able to replace fwprintf with PAL_fwprintf to make things compile.

[yowl]

Thanks,

runtime/src/coreclr/pal/src/include/pal/palinternal.h

Line 335 in 07ed923

#include "mbusafecrt.h"

Uses size_t but at its include point that's been defined to DUMMY_size_t, the undef comes after, and hence:

In file included from /home/scott/github/runtimelab/src/coreclr/nativeresources/../pal/src/include/pal/palinternal.h:335:
  /home/scott/github/runtimelab/src/coreclr/pal/inc/mbusafecrt.h:48:40: error: unknown type name 'DUMMY_size_t'
  extern errno_t strcat_s( char* ioDest, size_t inDestBufferSize, const char* inSrc );
                                         ^
  /home/scott/github/runtimelab/src/coreclr/nativeresources/../pal/src/include/pal/palinternal.h:207:16: note: expanded from macro 'size_t'
  #define size_t DUMMY_size_t
                 ^

I tried adding to mbusafecrt.h:

#ifdef PAL_STDCPP_COMPAT
#undef size_t
#include <stddef.h>
#endif

which clears this one, but moves to the next:

  In file included from /home/scott/github/runtimelab/src/coreclr/nativeresources/resourcestring.cpp:6:
  In file included from /home/scott/github/runtimelab/src/coreclr/inc/utilcode.h:14:
  In file included from /home/scott/github/runtimelab/src/coreclr/inc/winwrap.h:41:
  In file included from /home/scott/github/runtimelab/src/coreclr/nativeresources/../pal/src/include/pal/malloc.hpp:22:
  In file included from /home/scott/github/runtimelab/src/coreclr/nativeresources/../pal/src/include/pal/corunix.hpp:23:
  In file included from /home/scott/github/runtimelab/src/coreclr/nativeresources/../pal/src/include/pal/palinternal.h:335:
  /home/scott/github/runtimelab/src/coreclr/pal/inc/mbusafecrt.h:53:16: error: declaration of 'strcat_s' has a different language linkage
  extern errno_t strcat_s( char* ioDest, size_t inDestBufferSize, const char* inSrc );
                 ^
  /home/scott/github/runtimelab/src/coreclr/pal/inc/rt/safecrt.h:666:17: note: previous definition is here
  errno_t __cdecl strcat_s(char *_Dst, size_t _SizeInBytes, const char *_Src)
                  ^

Comments in mbusafecrt.h indicates its for MacOS but its included unconditionally by palinternal.h. Maybe the right thing is not to include it for Linux ?