Allow overriding lang-items from different crates

af1e4ff
Opened by Huon Wilson at 2013-11-15 13:24:06

Building on the new optional lang items, this would, for example, allow someone to substitute a new version of malloc (e.g. because they want to count the number of allocations, or have a specialised, high-performance implementation).

Only allowing it cross crate would mean there wouldn't be any confusion about which takes precedence in:

mod foo { #[lang="exchange_malloc"] fn foo_malloc() { } }
mod bar { #[lang="exchange_malloc"] fn bar_malloc() { } }

This possibly couldn't work at all, because the other crates have had the original lang-item compiled into the appropriate places.

  1. I think that this would be fairly difficult to do, and the semantics should definitely be fleshed out before any work is done.

    When translating, if there's a spot which uses the exchange_malloc lang item, then LLVM has to emit some call to something at that point. I think that this means that normal name-mangling couldn't happen for lang items because otherwise you're emitting a call to something you can't override.

    One possible route to take that has other benefits would be to canonicalize all lang_item names to the same value. Something like __lang_item_<name of lang item>. I'm not quite sure how things like tcmalloc/jemalloc work by dynamically loading them at runtime and everything uses them, but this could probably work along the same lines (everyone assumes the memory allocation function is called malloc). Another benefit of this would be that a library crate could be compiled with a dependence on lang items, but not actually define them itself.

    So we could create a libcore which is libstd - librustrt or something like that, but it could still have methods for things like ~str and ~[T] without defining the lang_item for allocation. Whenever you linked against it, you would be required to define the symbol, but at compile-time for the crate it wouldn't matter. Not only would this be hugely beneficial for allocation functions, but it'd be fantastic if libcore could still use fail!(). Programs linking against libcore (like kernels) would have to define what exactly they want to happen whenever a fail!() occurs (or an out-of-bounds exception)

    Basically

    • I'm not sure if this is possible without canonicalizing the names of lang_items (functions make sense, do traits?)
    • If we do that, we could extend this to make missing lang_items even more flexible!

    I'm curious how feasible this is. What do others think?

    Alex Crichton at 2013-07-19 16:20:17

  2. You can't rely on the same trick used by tcmalloc and jemalloc because it implies dynamic linking. It only works if the application dynamically links against the standard C library and it uses weak symbols. The real solution for the allocator is to use the je_ prefix and extend LLVM's detection of malloc and free to them.

    Daniel Micay at 2013-07-19 16:37:12

  3. Triage: I don't know of any progress.

    Huon Wilson at 2013-11-15 12:56:29

  4. I don't think it makes sense to override these. Thanks to inter-procedural optimization, you cannot replace functions defined in a compiled binary with alternate functions. It's possible to override weak symbols defined in a dynamic library like malloc and free, and I think that's enough. Since we aren't using jemalloc anymore there's nothing preventing the usual weak symbol override.

    Daniel Micay at 2013-11-15 13:24:06