Use custom build mode for mdx to work with cstubs. #8704

frejsoya · 2023-09-19T12:20:55Z

Change the hardcoded link flags to use custom mode.

frejsoya · 2023-09-19T12:32:50Z

This is one possible fix, with potentially slower link/build time. But this way, ctypes and mdx can be used together.

Note that given PR #8631, this may be the wrong approach. But the final exe there also looks to be in done in custom mode.

Best
Frej Soya

rgrinberg · 2023-09-19T12:57:53Z

Yeah, I think this will make the mdx tests needlessly slow for everyone. Could you copy @nojb's approach instead?

frejsoya · 2023-09-20T12:25:12Z

@rgrinberg

I set up a test case but noticed dune utop now fails after #8631 is merged. Pushed the testcase to this branch/pr

Differences from existing test cases for utop.

ctypes
pkg-config for compiler flags.
This is on osx.

The failure is:

 Fatal error: cannot load shared library dllexamplelib_stubs
  Reason: dlopen(dllexamplelib_stubs.so, 0x000A): symbol not found in flat namespace '_example_add2'

Possible cases

dlopen works differently on osx and dependencies are not loaded transitively (man page for linux dlopen is explicitly about this)
dllexamplelib_stubs.so is not build correctly. otool -L dllexamplelib_stubs.so does not mention libexample.so

Alizter · 2023-09-20T13:06:23Z

cc @nojb

nojb · 2023-09-20T15:56:19Z

Sorry for the naïve question, but where is example_add2 defined?

frejsoya · 2023-09-20T16:19:54Z

@nojb test/blackbox-tests/test-cases/ctypes/libexample/example.{h,c}.

nojb · 2023-09-20T18:32:26Z

@nojb test/blackbox-tests/test-cases/ctypes/libexample/example.{h,c}.

Thanks. I am not familiar with the ctypes build rules so it is a bit hard to know what is going on off the top of my head, but at least in
https://github.com/frejsoya/dune/blob/24737d5d4debf9f1b6c2d383391e4a4f7efc1d73/test/blackbox-tests/test-cases/ctypes/libexample/dune#L17
the shared library name seems wrong: it should be dllexample.so (but this may not suffice to fix the issue).

nojb · 2023-09-20T19:13:30Z

I investigated a little more: the problem seems to be the linking command for the library examplelib.cma (which is produced by the ctypes stanza):

(cd _build/default && /home/nojebar/.opam/4.14.1/bin/ocamlc.opt \
-strict-sequence -strict-formats -short-paths -keep-locs -w -9-27 \
-g -a -o stubgen/examplelib.cma \
-dllib -lexamplelib_stubs -cclib -lexamplelib_stubs \
stubgen/.examplelib.objs/byte/examplelib.cmo \
stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_types.cmo \
stubgen/.examplelib.objs/byte/examplelib__Type_description.cmo \
stubgen/.examplelib.objs/byte/examplelib__Types_generated.cmo \ 
stubgen/.examplelib.objs/byte/examplelib__Function_description.cmo \ 
stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_functions__Function_description__Functions.cmo \ 
stubgen/.examplelib.objs/byte/examplelib__C.cmo \
-cclib -L/home/nojebar/t/libexample -cclib -lexample)

I think this linking command is missing a -dllib -lexample (similar to the -dllib -lexamplelib_stubs).

nojb · 2023-09-20T20:55:57Z

I think this linking command is missing a -dllib -lexample (similar to the -dllib -lexamplelib_stubs).

For future reference, my understanding is as follows:

Given a stanza of the form:

(library
 (name mylib)
 (ctypes
  (external_library_name foo))))

Dune will query pkg-config and retrieve a number of link-time flags for foo. Among these flags, those of the form -lxxx should be passed to the OCaml compiler when linking mylib.cma as -dllib -lxxx. And those of the form -Lxxx should be passed to ocamlc when building an executable that links against mylib.cma as -I xxx. Neither of these is done today.

Note that this is only a heuristic, and it is not guaranteed to work in general, but should work for the most common cases.

frejsoya · 2023-09-20T22:37:33Z

Oops. Only read the latest comment not the first. I found the same.

The command args create examplelib.cma uses -cclib, which is only used when doing custom linking. custom vs. byte is not interchangeable.

Reference build command.
Note the file is created, similarly utop.bc in later stage. But ocamlrun fails.


ocamlc.opt
-w
@1..3@5..28@30..39@43@46..47@49..57@61..62@67@69-40
-strict-sequence
-strict-formats
-short-paths
-keep-locs
-w
-9-27
-g
-a
-o
stubgen/examplelib.cma
-dllib
-lexamplelib_stubs
-cclib
-lexamplelib_stubs
stubgen/.examplelib.objs/byte/examplelib.cmo
stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_types.cmo
stubgen/.examplelib.objs/byte/examplelib__Type_description.cmo
stubgen/.examplelib.objs/byte/examplelib__Types_generated.cmo
stubgen/.examplelib.objs/byte/examplelib__Function_description.cmo
stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_functions__Function_description__Functions.cmo
stubgen/.examplelib.objs/byte/examplelib__C.cmo
-cclib
-L/Users/frej/Projects/dune/test/blackbox-tests/test-cases/ctypes/libexample
-cclib
-lexample
```

frejsoya · 2023-09-21T08:43:42Z

So this is one working reference example. There might be other ways to achieve the same.

There is a difference when handling paths between dll_<>.so made by ocamlmklib which ocamlrun can lookup up, and paths for finding regular shared libraries that dllexample_stubs.so links to, in the working case below dllexamplelib_stubs.so knows what lib to find look for unknown symbols. So libexample.so is explicitly mentioned.

otool -L stubgen/dllexamplelib_stubs.so
stubgen/dllexamplelib_stubs.so:
	stubgen/dllexamplelib_stubs.so (compatibility version 0.0.0, current version 0.0.0)
	libexample.so (compatibility version 0.0.0, current version 0.0.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.100.3)

Note from ocamlc manpage:

-dllib -llibname
Arrange for the C shared library dlllibname.so to be loaded dynamically by the run-time system ocamlrun(1) at program start-up time.

For me it looks like that path is for finding dllstubs_example.so, when dlopen(dllstubs_example.so) is set different from the set of path to find libexample.so..

Ensure dllstubs_example.so understands it needs to load lib example. Adding adding linker flags -L,-l to ocamlmklib
At runtime be able to the library paths to find paths to search for libexample.so when libexample.so is not in the existing system path (Ie. {DYLD,LD}_LIBRARY_PATH).
- Adding using -rpath with should work with ocamlmklib, so it the nonstandard location is encoded .so. But I could not make it work.
- Modifying {DYLD,LD}_LIBRARY_PATH at runtime. That is: DYLD_LIBRARY_PATH=${LIB_EXAMPLE_PATH} ocamlrun -I stubgen .utop/utop.bc

#!/bin/bash
cd _build/default
pwd
rm -f -v stubgen/dllexamplelib_stubs.so || true
rm -f -v stubgen/examplelib.cma || true
set -x
set -e
LIB_EXAMPLE_PATH=/Users/frej/Projects/dune/_build/default/test/blackbox-tests/test-cases/ctypes/libexample/
LIB_PATH="-L/Users/frej/Projects/dune/_build/default/test/blackbox-tests/test-cases/ctypes/libexample/"
LIB="-lexample"
#build example.cma
/Users/frej/.opam/5.0.0/bin/ocamlc.opt -w @1..3@5..28@30..39@43@46..47@49..57@61..62@67@69-40 -strict-sequence -strict-formats -short-paths -keep-locs -w -9-27 -g -a -o stubgen/examplelib.cma -dllib -lexamplelib_stubs -cclib -lexamplelib_stubs stubgen/.examplelib.objs/byte/examplelib.cmo stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_types.cmo stubgen/.examplelib.objs/byte/examplelib__Type_description.cmo stubgen/.examplelib.objs/byte/examplelib__Types_generated.cmo stubgen/.examplelib.objs/byte/examplelib__Function_description.cmo stubgen/.examplelib.objs/byte/examplelib__Libexample__c_generated_functions__Function_description__Functions.cmo stubgen/.examplelib.objs/byte/examplelib__C.cmo -cclib -L/Users/frej/Projects/dune/test/blackbox-tests/test-cases/ctypes/libexample -cclib -lexample

#build dllexamplelib_stubs.so
# ADDING LINK FLAGS when building  dllexample_stubs.so object.
/Users/frej/.opam/5.0.0/bin/ocamlmklib.opt -verbose -g -o stubgen/examplelib_stubs stubgen/libexample__c_cout_generated_functions__Function_description__Functions.o ${LIB_PATH} ${LIB}

/Users/frej/.opam/5.0.0/bin/ocamlc.opt -verbose -w @1..3@5..28@30..39@43@46..47@49..57@61..62@67@69-40 -strict-sequence -strict-formats -short-paths -keep-locs -w -24 -g -o .utop/utop.bc /Users/frej/.opam/5.0.0/lib/ocaml/compiler-libs/ocamlcommon.cma /Users/frej/.opam/5.0.0/lib/ocaml/compiler-libs/ocamlbytecomp.cma /Users/frej/.opam/5.0.0/lib/ocaml/compiler-libs/ocamltoplevel.cma /Users/frej/.opam/5.0.0/lib/findlib/findlib.cma /Users/frej/.opam/5.0.0/lib/findlib/findlib_top.cma /Users/frej/.opam/5.0.0/lib/logs/logs.cma /Users/frej/.opam/5.0.0/lib/lwt/lwt.cma /Users/frej/.opam/5.0.0/lib/logs/logs_lwt.cma /Users/frej/.opam/5.0.0/lib/ocplib-endian/ocplib_endian.cma /Users/frej/.opam/5.0.0/lib/ocplib-endian/bigstring/ocplib_endian_bigstring.cma /Users/frej/.opam/5.0.0/lib/ocaml/unix/unix.cma /Users/frej/.opam/5.0.0/lib/ocaml/threads/threads.cma /Users/frej/.opam/5.0.0/lib/lwt/unix/lwt_unix.cma -I /Users/frej/.opam/5.0.0/lib/lwt/../stublibs /Users/frej/.opam/5.0.0/lib/react/react.cma /Users/frej/.opam/5.0.0/lib/lwt_react/lwt_react.cma /Users/frej/.opam/5.0.0/lib/result/result.cma /Users/frej/.opam/5.0.0/lib/uutf/uutf.cma /Users/frej/.opam/5.0.0/lib/uucp/uucp.cma /Users/frej/.opam/5.0.0/lib/uuseg/uuseg.cma /Users/frej/.opam/5.0.0/lib/zed/zed.cma /Users/frej/.opam/5.0.0/lib/trie/trie.cma /Users/frej/.opam/5.0.0/lib/mew/mew.cma /Users/frej/.opam/5.0.0/lib/mew_vi/mew_vi.cma /Users/frej/.opam/5.0.0/lib/lambda-term/lambda_term.cma -I /Users/frej/.opam/5.0.0/lib/lambda-term/../stublibs /Users/frej/.opam/5.0.0/lib/xdg/xdg.cma -I /Users/frej/.opam/5.0.0/lib/xdg/../stublibs /Users/frej/.opam/5.0.0/lib/utop/uTop.cma /Users/frej/.opam/5.0.0/lib/stdlib-shims/stdlib_shims.cma /Users/frej/.opam/5.0.0/lib/integers/integers.cma -I /Users/frej/.opam/5.0.0/lib/integers/../stublibs /Users/frej/.opam/5.0.0/lib/bigarray-compat/bigarray_compat.cma /Users/frej/.opam/5.0.0/lib/ctypes/ctypes.cma -I /Users/frej/.opam/5.0.0/lib/ctypes/../stublibs /Users/frej/.opam/5.0.0/lib/ocaml/str/str.cma /Users/frej/.opam/5.0.0/lib/ctypes/stubs/ctypes_stubs.cma stubgen/examplelib.cma -I stubgen .utop/.utop.eobjs/byte/dune__exe__Utop.cmo -linkall -warn-error -31


DYLD_LIBRARY_PATH=${LIB_EXAMPLE_PATH} ocamlrun -I stubgen .utop/utop.bc

Use custom build mode for mdx to work with cstubs.

7cb1a74

Change the hardcoded link flags to use custom mode.

Add testcase for mdx

24737d5

frejsoya closed this Sep 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use custom build mode for mdx to work with cstubs. #8704

Use custom build mode for mdx to work with cstubs. #8704

frejsoya commented Sep 19, 2023 •

edited

Loading

frejsoya commented Sep 19, 2023

rgrinberg commented Sep 19, 2023

frejsoya commented Sep 20, 2023 •

edited

Loading

Alizter commented Sep 20, 2023

nojb commented Sep 20, 2023

frejsoya commented Sep 20, 2023 •

edited

Loading

nojb commented Sep 20, 2023

nojb commented Sep 20, 2023

nojb commented Sep 20, 2023

frejsoya commented Sep 20, 2023 •

edited

Loading

frejsoya commented Sep 21, 2023 •

edited

Loading

Use custom build mode for mdx to work with cstubs. #8704

Use custom build mode for mdx to work with cstubs. #8704

Conversation

frejsoya commented Sep 19, 2023 • edited Loading

frejsoya commented Sep 19, 2023

rgrinberg commented Sep 19, 2023

frejsoya commented Sep 20, 2023 • edited Loading

Alizter commented Sep 20, 2023

nojb commented Sep 20, 2023

frejsoya commented Sep 20, 2023 • edited Loading

nojb commented Sep 20, 2023

nojb commented Sep 20, 2023

nojb commented Sep 20, 2023

frejsoya commented Sep 20, 2023 • edited Loading

frejsoya commented Sep 21, 2023 • edited Loading

frejsoya commented Sep 19, 2023 •

edited

Loading

frejsoya commented Sep 20, 2023 •

edited

Loading

frejsoya commented Sep 20, 2023 •

edited

Loading

frejsoya commented Sep 20, 2023 •

edited

Loading

frejsoya commented Sep 21, 2023 •

edited

Loading