issues with ABIs and struct return types

442caf4
Opened by Vladimir Vukicevic at 2024-04-18 00:35:03

With the MSVC x64 ABI, structs are returned in RAX if and only if they are <= 8 bytes in size, and are effectively a POD type. See https://msdn.microsoft.com/en-us/library/7572ztz4.aspx . Unfortunately, I can't figure out how to tell Rust this. In C++, sk_sp<T> has a user-defined constructor, destructor, etc. In Rust, I couldn't figure out any way to do this. It's defined as..

    #[repr(C)]
    #[derive(Debug, Copy, Clone)]
    pub struct sk_sp<T> {
        pub ptr: *mut T,
    }

This causes segfaults because the C++ calling convention isn't upheld. A temporary workaround is to just add another dummy field, making this type bigger than 8 bytes. This happens to work in this case since this type is only ever used as a smart pointer return type; params are passed as basic T*.

Is there any way to tell rustc to treat this type as non-POD when giving it to LLVM? If not, can such a mechanism be added?

  1. Actually, this difference extends to 32bit as well. C++ methods have that slight difference in how they handle return types compared to C/C++ functions. A case where I encountered this issue was actually in binding COM interfaces. 99% of the time they'd just be returning HRESULT which is the same for both methods and functions. However every once in a while a COM method would return an aggregate type that is affected by this slight difference and Rust would have the wrong calling convention. Hell, even C compiled by cl.exe using the official C bindings to the COM interface gets it wrong, so this issue is so subtle that not even Microsoft has done anything about it.

    Related issue https://github.com/rust-lang/rfcs/issues/1342

    Peter Atashian at 2016-12-09 17:30:48

  2. Someone ran into this today https://users.rust-lang.org/t/returning-values-from-rust-to-c/11256

    EDIT: actually reading more, maybe not...

    Steve Klabnik at 2017-06-09 15:33:03

  3. @steveklabnik (Turns out that was indeed this issue: the user added a constructor on the C++ side, making it non-POD, and didn't mention it originally.)

    comex at 2017-06-10 09:40:05

  4. In order to fully solve this we would need two things:

    1. Calling conventions for C++ methods rather than functions. The difference in how they handle struct returns is absolutely crucial for code to get right.
    2. A new repr for structs to indicate that they are not POD.

    Peter Atashian at 2017-06-23 18:43:39

  5. This is also an issue for Linux btw, see https://github.com/rust-lang/rust-bindgen/issues/778.

    Emilio Cobos Álvarez at 2017-06-24 06:37:15

  6. A new repr for structs to indicate that they are not POD.

    The notion of "POD" here is idiosyncratic to the MSVC ABI (e.g. it does not match the C++ notion of "standard-layout"), so care should be taking with naming such a repr to not cause further confusion.

    Benjamin Saunders at 2024-04-18 00:35:03