Make --export-dynamic the default option when linking executables.

f0285de
Opened by Parakleta at 2022-05-15 14:48:56

With #[no_mangle] pub fn Rust exports a dynamic symbol for the defined function in an executable, but this behaviour is somewhat magical and not transitive.

If --export-dynamic is made the default then all symbols that are marked as visible in any dependency of the executable will also be exported as a dynamic symbol. This is important for libraries that support plugins where the plugin needs to be able to call back into a function defined in the library. The case I am dealing with is Lua plugins which must have access to the Lua runtime functions when loaded, but presumably the same issues exist for other pluggable libraries (for example SQLite loadable extensions).

So long as Rust correctly manage its symbol visibility, and authors of static libraries correctly manage the symbol visibility of the C functions in their libraries, this change should have no down-sides.

I believe currently Rust does not quite manage symbol visibility correctly because it leaves every local absolute symbol visible. I'm not sure if there is a reason for this or it is just an oversight, but this would probably need to be resolved.

On the issue of the #[no_mangle] pub fn I'm not sure why this behaviour is not transitive, but I believe this issue should also be fixed. I may not be using it correctly, but I cannot seem to make a #[no_mangle] pub fn from a dependency appear in the symbol table of an executable.

I believe this is related to issue #27541.

  1. We probably don't want to blanket pass --export-dynamic as it'd export a huge amount of symbols I think which we would rather the linker strip, but I'd consider it a bug that an unmangled extern function isn't showing up in the final binary (as it does for dylibs/staticlibs).

    Alex Crichton at 2016-06-09 14:41:46

  2. ~~If you use --export-dynamic in a blanket fashion then you can still manage the symbol visibility separately since the flag will only export visible symbols (note that hidden symbols still appear in static libraries since they're not linked yet so there are no problems with hiding everything).~~

    ~~The main issue is to do with transitivity, and I guess there are two approaches.~~

    ~~The first is to have each library (Rust or FFI) be responsible for defining its own symbol visibility and this is applied globally, as in my proposal with --export-dynamic. So long as each library properly hides symbols this should work just fine. This should be the simplest change.~~

    ~~The other approach is to have Rust carry the list of which symbols should be exposed which it seems to do for the #[no_mangle] pub fn magic, but it would need to do this transitively as well. That is, any declared visible symbols from libraries must also be made visible from the executable. This has some advantage in that the symbols made visible from a library could then be hidden in an executable by not fulfilling some transitive requirement (i.e. pub use?) which would ultimately give more control.~~

    ~~The second approach is more powerful and probably the better option, but it has syntax implications for the Rust language which may be harder to implement. Essentially for it to be any different to --export-dynamic there would need to become a distinction between functions that are public for the sake of compile time linking and those that are public for the sake of dynamic linking. Rust would also need to have a way to declare this difference for FFI functions as well since it would have to take ownership of their visibility.~~

    Actually, I've just realised that maybe it doesn't need to be transitive at all, and maybe you don't even need new syntax. What about if just every public symbol in the main executable is converted to a dynamic symbol? Does pub have any meaning otherwise in an executable? This way I can just pub use those symbols I need exported from the executable.

    Maybe I should change the issue to "pub declared in Executables as should be exported as Dynamic Symbols".

    Parakleta at 2016-06-10 00:32:33

  3. Yeah in theory all reachable symbols ("pub to the root") should be exported from an executable. We don't do much with whether symbols are dynamic or not, we largely just configure their linkage visibility to LLVM currently.

    Alex Crichton at 2016-06-10 13:46:49

  4. Unfortunately I don't know anything about how LLVM handles linkage, but the dynamic symbols are important and the result of compiling a Rust program seems inconsistent at the moment. There are quite a few dymanic symbols already coming out, but apart from no_mangle pub I can't see why the others are there.

    I'm not sure what you mean by "pub to the root", is root main.rs or does root contain main.rs? I think what I would consider "pub from root" (i.e. declared pub in the root of main.rs, and so exported from the program) need to be made into dynamic visible symbols (and ideally only those).

    Parakleta at 2016-06-10 14:20:16

  5. The other dynamic symbols are likely coming from dependencies, which are compiled differently. By "pub to the root" I just mean reachable if the executable were actually a crate.

    Alex Crichton at 2016-06-10 23:48:12

  6. https://github.com/rust-lang/rust/pull/85673 will add a -Zexport-executable-symbols flag to export symbols from executables.

    bjorn3 at 2022-05-15 14:48:56