LLVM bindings have become incorrect in places.

Opened by Eduard-Mihai Burtescu at 2025-03-01 19:10:12

Because of x86/x86_64 specifics, certain LLVM functions that used to return unsigned int and now return unsigned long long continue to be usable, ending up in truncated return values.

pub fn LLVMABISizeOfType(TD: TargetDataRef, Ty: TypeRef) -> c_uint;

unsigned long long LLVMABISizeOfType(LLVMTargetDataRef TD, LLVMTypeRef Ty);

This is just one example of such incorrectness arising over time, there could be more. Would be nice if we could use bindgen every LLVM upgrade to verify our bindings (or just generate new ones).

Just curious, what's bindgen? Are there docs on how to use it?
Ahmed Charles at 2014-10-06 15:43:04
@ahmedcharles I've updated the description with a link to bindgen.
Eduard-Mihai Burtescu at 2014-10-06 16:21:37
I spoke about this with @eddyb on IRC today. This is especially pernicious, because we could be accidentally getting UB in corner cases.

A tactic to close this bug:
1. Generate headers from LLVM with bindgen
2. Run diff
3. see if any of it is a particular issue.
Humans looking at it will probably be needed, given that bindgen isn't perfect.
Steve Klabnik at 2015-12-31 18:08:16

I took a look at this, starting with the rather naive:

#!/usr/bin/env sh

set -eu

LLVM_CONFIG_PATH=$(find build -name llvm-config -type f | head -n1)
CXX_FLAGS=$($LLVM_CONFIG_PATH --cxxflags)

echo CXX_FLAGS: $CXX_FLAGS

RUST_BACKTRACE=1 bindgen $@ src/rustllvm/rustllvm.h -o src/librustc_llvm/ffi.rs -- $CXX_FLAGS

which produces the output:

$ ./bindgen.sh
CXX_FLAGS: -I/Users/tamird/src/rust/build/x86_64-apple-darwin/llvm/include -ffunction-sections -fdata-sections -fPIC -m64 -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -Werror=date-time -std=c++11 -O3 -DNDEBUG -fno-exceptions -fno-rtti -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
error: invalid argument '-std=c++11' not allowed with 'C/ObjC', err: true
thread 'main' panicked at 'Unable to generate bindings: ()', src/libcore/result.rs:906:4
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic
   6: std::panicking::begin_panic_fmt
   7: rust_begin_unwind
   8: core::panicking::panic_fmt
   9: core::result::unwrap_failed
  10: std::panicking::try::do_call
  11: __rust_maybe_catch_panic
  12: bindgen::main
  13: __rust_maybe_catch_panic
  14: std::rt::lang_start

Tamir Duberstein at 2017-09-13 19:07:09

Using --cflags rather than --cxxflags produces c++ parsing failures:

$ ./bindgen.sh
C_FLAGS: -I/Users/tamird/src/rust/build/x86_64-apple-darwin/llvm/include -ffunction-sections -fdata-sections -fPIC -m64 -stdlib=libc++ -fPIC -Wall -W -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers -pedantic -Wno-long-long -Wcovered-switch-default -Wdelete-non-virtual-dtor -Wstring-conversion -Werror=date-time -O3 -DNDEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:263:9: error: unknown type name '__char16_t'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:264:9: error: unknown type name '__char32_t'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:1: error: unknown type name 'namespace'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:14: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:1: error: unknown type name 'template'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:10: error: expected identifier or '('
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:1: error: unknown type name 'template'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:10: error: expected identifier or '('
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: unknown type name 'namespace'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: unknown type name 'namespace'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:135:38: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:30: error: variable has incomplete type 'void'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:38: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:30: error: variable has incomplete type 'void'
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:38: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:144:38: error: expected ';' after top level declarator
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:149:38: error: expected ';' after top level declarator
fatal error: too many errors emitted, stopping now [-ferror-limit=]
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:263:9: error: unknown type name '__char16_t', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:264:9: error: unknown type name '__char32_t', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:1: error: unknown type name 'namespace', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:403:14: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:1: error: unknown type name 'template', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:598:10: error: expected identifier or '(', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:1: error: unknown type name 'template', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/__config:599:10: error: expected identifier or '(', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: unknown type name 'namespace', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstddef:44:1: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: unknown type name 'namespace', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/cstring:67:1: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:135:38: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:30: error: variable has incomplete type 'void', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:136:38: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:30: error: variable has incomplete type 'void', err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:137:38: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:144:38: error: expected ';' after top level declarator, err: true
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../include/c++/v1/new:149:38: error: expected ';' after top level declarator, err: true
fatal error: too many errors emitted, stopping now [-ferror-limit=], err: true
thread 'main' panicked at 'Unable to generate bindings: ()', src/libcore/result.rs:906:4
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::_print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::begin_panic
   6: std::panicking::begin_panic_fmt
   7: rust_begin_unwind
   8: core::panicking::panic_fmt
   9: core::result::unwrap_failed
  10: std::panicking::try::do_call
  11: __rust_maybe_catch_panic
  12: bindgen::main
  13: __rust_maybe_catch_panic
  14: std::rt::lang_start

Tamir Duberstein at 2017-09-13 19:10:06

Ah, renaming rustllvm.h to rustllvm.hpp gets this further, but we hit https://github.com/rust-lang-nursery/rust-bindgen/issues/492:
```
./bindgen.sh 2>bindgen.stderr
```
gist containing bindgen.sh & bindgen.stderr
Tamir Duberstein at 2017-09-13 22:49:24
If someone's interested in this (like me, originally), I'll save you some time:

The above PR #46353 was rejected due to A) bindgen needing an unwanted dependency and B) the team preferring a validation of the bindings over re-generating the bindings automatically via bindgen. (This is understandable, as the hand-written bindings are way nicer than bindgens output)

ctest was proposed as a crate that allows testing the existing bindings. However, after some research, ctest is pretty much abandoned and even the fork ctest2 suffers from outdated dependencies, which results in the crate practically being unable to handle rustc_codegen_llvm which is the crate that would need to be parsed. I tried.

The way I see it, the validation can't be reliably automated at the moment unless ctest2 gets a major update/rewrite or an alternative pops up (maybe with syn-based parsing).

Edit: Actually, it might be possible. For example. libz-sys uses ctest2. I guess the code in libz-sys is simple enough to be parsed by the outdated syntex crate. Technically, the bindings could be moved to a separate crate (e.g. rustc_llvm_bindings) and if the code is simple/"old" enough, syntex and ctest should be able to handle it.
FreezyLemon at 2022-12-10 20:45:24
I think the bindings should probably move to rustc_llvm, since that's the one that actually prepares the link library, like a typical *-sys crate. That's also where we define our C++ shims, then we would have all LLVM ffi in one place.
Josh Stone at 2022-12-10 21:17:52
Oh, I didn't know a crate like that already existed. I'll take a look, maybe it's possible to get ctest to work with it.
FreezyLemon at 2022-12-11 14:04:40
ctest still fails at parsing even after moving ffi.rs to rustc_llvm. It might be possible by refactoring the code, but it's not trivial. It's honestly not something I'm willing to spend more time on at the moment. It might make more sense to "just wait" and hope ctest gets an update
FreezyLemon at 2022-12-11 15:49:29