Skip to content

Commit

Permalink
Support building from a MinGW-w64-based compiler (#15077)
Browse files Browse the repository at this point in the history
This is a continuation of #15070 that allows a compiler built with MinGW-w64 to itself build programs correctly. Resolves part of #6170.

* Because linker flags for GCC may now be executed on a Windows environment, we use the correct form of argument quoting. We also drop `-rdynamic` since that only makes sense for ELF executables.
* Targetting `x86_64-windows-gnu`, including normal compilations from such a Crystal compiler, will not copy dependent DLLs to the output directory. Crystal itself and programs built under MSYS2 will just work as long as the proper environment is used. You are on your own here, although `ldd` exists on MSYS2 so that you don't need the MSVC build tools for this.
* The correct GCC compiler flag to select `wmain` over `main` as the C entry point is `-municode`. (The system entry point is presumably `_start` now.)
* `legacy_stdio_definitions.obj` doesn't exist on MinGW-w64, so we disable it outside MSVC.
* For build command lines that are too long on Windows, we use GCC's response file support.

To build a MinGW-w64 compiler:

```cmd
@Rem on the MSVC developer prompt
make -fMakefile.win crystal
bin\crystal build --cross-compile --target=x86_64-windows-gnu src\compiler\crystal.cr -Dwithout_interpreter
```

```sh
# on MSYS2's UCRT64 environment
pacman -Sy \
  mingw-w64-ucrt-x86_64-gc mingw-w64-ucrt-x86_64-pcre2 mingw-w64-ucrt-x86_64-libiconv \
  mingw-w64-ucrt-x86_64-zlib mingw-w64-ucrt-x86_64-openssl mingw-w64-ucrt-x86_64-llvm
cc crystal.obj -o crystal \
  $(pkg-config bdw-gc libpcre2-8 iconv zlib openssl --libs) \
  $(llvm-config --libs --system-libs --ldflags) \
  -lDbgHelp -lole32 -lWS2_32
export CRYSTAL_PATH='lib;$ORIGIN\src'
export CRYSTAL_LIBRARY_PATH=''
```

Now you can run or build a considerable number of files from here, such as `./crystal.exe samples/2048.cr` and `./crystal.exe spec spec/std/regex_spec.cr`. Notable omissions are OpenSSL and LLVM, as fixing their version detection macros is a bit complicated.

The interpreter is not supported. Most likely, `Crystal::Loader` would have a GCC-style `.parse`, but the rest of the functionality would be identical to the MSVC `LoadLibraryExW`-based loader.

~~Also, some invocations like `./crystal.exe spec spec/std/json` will fail since the whole command line string is too long. Similar to MSVC, [GCC also handles response files starting with `@`](https://gcc.gnu.org/onlinedocs/gcc/Overall-Options.html), so this can be implemented later; a workaround is to use `--single-module`.~~

For reference, here are all the useful MSYS2 packages and their corresponding pkg-config names:

| MSYS2 package name             | pkg-config name |
|-|-|
| mingw-w64-ucrt-x86_64-gc       | bdw-gc     |
| mingw-w64-ucrt-x86_64-pcre2    | libpcre2-8 |
| mingw-w64-ucrt-x86_64-libiconv | iconv      |
| mingw-w64-ucrt-x86_64-gmp      | gmp        |
| mingw-w64-ucrt-x86_64-zlib     | zlib       |
| mingw-w64-ucrt-x86_64-libxml2  | libxml-2.0 |
| mingw-w64-ucrt-x86_64-libyaml  | yaml-0.1   |
| mingw-w64-ucrt-x86_64-openssl  | openssl    |
| mingw-w64-ucrt-x86_64-libffi   | libffi     |
| mingw-w64-ucrt-x86_64-llvm     | _(use llvm-config instead)_ |
  • Loading branch information
HertzDevil authored Oct 19, 2024
1 parent fa25838 commit 3a78e8a
Show file tree
Hide file tree
Showing 4 changed files with 80 additions and 35 deletions.
26 changes: 17 additions & 9 deletions src/compiler/crystal/codegen/link.cr
Original file line number Diff line number Diff line change
Expand Up @@ -120,18 +120,18 @@ module Crystal
end

class Program
def lib_flags
has_flag?("msvc") ? lib_flags_windows : lib_flags_posix
def lib_flags(cross_compiling : Bool = false)
has_flag?("msvc") ? lib_flags_windows(cross_compiling) : lib_flags_posix(cross_compiling)
end

private def lib_flags_windows
private def lib_flags_windows(cross_compiling)
flags = [] of String

# Add CRYSTAL_LIBRARY_PATH locations, so the linker preferentially
# searches user-given library paths.
if has_flag?("msvc")
CrystalLibraryPath.paths.each do |path|
flags << Process.quote_windows("/LIBPATH:#{path}")
flags << quote_flag("/LIBPATH:#{path}", cross_compiling)
end
end

Expand All @@ -141,14 +141,14 @@ module Crystal
end

if libname = ann.lib
flags << Process.quote_windows("#{libname}.lib")
flags << quote_flag("#{libname}.lib", cross_compiling)
end
end

flags.join(" ")
end

private def lib_flags_posix
private def lib_flags_posix(cross_compiling)
flags = [] of String
static_build = has_flag?("static")

Expand All @@ -158,7 +158,7 @@ module Crystal
# Add CRYSTAL_LIBRARY_PATH locations, so the linker preferentially
# searches user-given library paths.
CrystalLibraryPath.paths.each do |path|
flags << Process.quote_posix("-L#{path}")
flags << quote_flag("-L#{path}", cross_compiling)
end

link_annotations.reverse_each do |ann|
Expand All @@ -173,17 +173,25 @@ module Crystal
elsif (lib_name = ann.lib) && (flag = pkg_config(lib_name, static_build))
flags << flag
elsif (lib_name = ann.lib)
flags << Process.quote_posix("-l#{lib_name}")
flags << quote_flag("-l#{lib_name}", cross_compiling)
end

if framework = ann.framework
flags << "-framework" << Process.quote_posix(framework)
flags << "-framework" << quote_flag(framework, cross_compiling)
end
end

flags.join(" ")
end

private def quote_flag(flag, cross_compiling)
if cross_compiling
has_flag?("windows") ? Process.quote_windows(flag) : Process.quote_posix(flag)
else
Process.quote(flag)
end
end

# Searches among CRYSTAL_LIBRARY_PATH, the compiler's directory, and PATH
# for every DLL specified in the used `@[Link]` annotations. Yields the
# absolute path and `true` if found, the base name and `false` if not found.
Expand Down
78 changes: 54 additions & 24 deletions src/compiler/crystal/compiler.cr
Original file line number Diff line number Diff line change
Expand Up @@ -354,7 +354,7 @@ module Crystal
run_dsymutil(output_filename) unless debug.none?
{% end %}

{% if flag?(:windows) %}
{% if flag?(:msvc) %}
copy_dlls(program, output_filename) unless static?
{% end %}
end
Expand Down Expand Up @@ -424,26 +424,8 @@ module Crystal

private def linker_command(program : Program, object_names, output_filename, output_dir, expand = false)
if program.has_flag? "msvc"
lib_flags = program.lib_flags
# Execute and expand `subcommands`.
if expand
lib_flags = lib_flags.gsub(/`(.*?)`/) do
command = $1
begin
error_io = IO::Memory.new
output = Process.run(command, shell: true, output: :pipe, error: error_io) do |process|
process.output.gets_to_end
end
unless $?.success?
error_io.rewind
error "Error executing subcommand for linker flags: #{command.inspect}: #{error_io}"
end
output
rescue exc
error "Error executing subcommand for linker flags: #{command.inspect}: #{exc}"
end
end
end
lib_flags = program.lib_flags(@cross_compile)
lib_flags = expand_lib_flags(lib_flags) if expand

object_arg = Process.quote_windows(object_names)
output_arg = Process.quote_windows("/Fe#{output_filename}")
Expand Down Expand Up @@ -487,15 +469,63 @@ module Crystal
{linker, cmd, nil}
elsif program.has_flag? "wasm32"
link_flags = @link_flags || ""
{"wasm-ld", %(wasm-ld "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} -lc #{program.lib_flags}), object_names}
{"wasm-ld", %(wasm-ld "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} -lc #{program.lib_flags(@cross_compile)}), object_names}
elsif program.has_flag? "avr"
link_flags = @link_flags || ""
link_flags += " --target=avr-unknown-unknown -mmcu=#{@mcpu} -Wl,--gc-sections"
{DEFAULT_LINKER, %(#{DEFAULT_LINKER} "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} #{program.lib_flags}), object_names}
{DEFAULT_LINKER, %(#{DEFAULT_LINKER} "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} #{program.lib_flags(@cross_compile)}), object_names}
elsif program.has_flag?("win32") && program.has_flag?("gnu")
link_flags = @link_flags || ""
lib_flags = program.lib_flags(@cross_compile)
lib_flags = expand_lib_flags(lib_flags) if expand
cmd = %(#{DEFAULT_LINKER} #{Process.quote_windows(object_names)} -o #{Process.quote_windows(output_filename)} #{link_flags} #{lib_flags})

if cmd.size > 32000
# The command line would be too big, pass the args through a file instead.
# GCC response file does not interpret those args as shell-escaped
# arguments, we must rebuild the whole command line
args_filename = "#{output_dir}/linker_args.txt"
File.open(args_filename, "w") do |f|
object_names.each do |object_name|
f << object_name.gsub(GCC_RESPONSE_FILE_TR) << ' '
end
f << "-o " << output_filename.gsub(GCC_RESPONSE_FILE_TR) << ' '
f << link_flags << ' ' << lib_flags
end
cmd = "#{DEFAULT_LINKER} #{Process.quote_windows("@" + args_filename)}"
end

{DEFAULT_LINKER, cmd, nil}
else
link_flags = @link_flags || ""
link_flags += " -rdynamic"
{DEFAULT_LINKER, %(#{DEFAULT_LINKER} "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} #{program.lib_flags}), object_names}
{DEFAULT_LINKER, %(#{DEFAULT_LINKER} "${@}" -o #{Process.quote_posix(output_filename)} #{link_flags} #{program.lib_flags(@cross_compile)}), object_names}
end
end

private GCC_RESPONSE_FILE_TR = {
" ": %q(\ ),
"'": %q(\'),
"\"": %q(\"),
"\\": "\\\\",
}

private def expand_lib_flags(lib_flags)
lib_flags.gsub(/`(.*?)`/) do
command = $1
begin
error_io = IO::Memory.new
output = Process.run(command, shell: true, output: :pipe, error: error_io) do |process|
process.output.gets_to_end
end
unless $?.success?
error_io.rewind
error "Error executing subcommand for linker flags: #{command.inspect}: #{error_io}"
end
output.chomp
rescue exc
error "Error executing subcommand for linker flags: #{command.inspect}: #{exc}"
end
end
end

Expand Down
7 changes: 6 additions & 1 deletion src/crystal/system/win32/wmain.cr
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,12 @@ require "c/stdlib"

{% begin %}
# we have both `main` and `wmain`, so we must choose an unambiguous entry point
@[Link({{ flag?(:static) ? "libcmt" : "msvcrt" }}, ldflags: "/ENTRY:wmainCRTStartup")]
@[Link({{ flag?(:static) ? "libcmt" : "msvcrt" }})]
{% if flag?(:msvc) %}
@[Link(ldflags: "/ENTRY:wmainCRTStartup")]
{% elsif flag?(:gnu) %}
@[Link(ldflags: "-municode")]
{% end %}
{% end %}
lib LibCrystalMain
end
Expand Down
4 changes: 3 additions & 1 deletion src/lib_c/x86_64-windows-msvc/c/stdio.cr
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
require "./stddef"

@[Link("legacy_stdio_definitions")]
{% if flag?(:msvc) %}
@[Link("legacy_stdio_definitions")]
{% end %}
lib LibC
# unused
fun printf(format : Char*, ...) : Int
Expand Down

0 comments on commit 3a78e8a

Please sign in to comment.