armv7 exhausts memory on stage0 librustc w/ debuginfo

c686cb0
Opened by Josh Stone at 2024-06-02 23:45:44

I'm having trouble natively compiling rustc armv7-unknown-linux-gnueabihf with --enable-debuginfo, on both the beta and master branches. I haven't seen this on any stable release before. To reproduce the issue, I'm running on Fedora 26 armv7hl, using all default options apart from enabling debuginfo.

Building stage0 compiler artifacts (armv7-unknown-linux-gnueabihf -> armv7-unknown-linux-gnueabihf)
[...]
   Compiling rustc v0.0.0 (file:///home/jistone/rust-beta/src/librustc)
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
error: Could not compile `rustc`.

Caused by:
  process didn't exit successfully: `/home/jistone/rust-beta/build/bootstrap/debug/rustc --crate-name rustc src/librustc/lib.rs [...]` (signal: 6, SIGABRT: process abort signal)

I guess that C++ exception is probably LLVM running out of memory. This excerpt is from beta sources -- on master it also errors, but without the exception info, instead a SIGSEGV. I doubt that difference is very significant, just bad luck for who failed to allocate memory.

In either case, when I watch progress in htop, the process grows right up to 3GB virt size and then dies. That's the limit of the 32-bit user address space, of course. The resident size was just a little over 2GB at that time, so something is holding wasted memory here. It could be jemalloc, or could be extra capacity in vec/map/etc. -- I'm not sure how to tell.

My workaround for now is to use --enable-debuginfo-only-std.

(And FWIW, 32-bit i686 is not having this problem for me.)

  1. Firefox appears to be hitting this now too, while compiling gkrust: https://bugzilla.redhat.com/show_bug.cgi?id=1523912

    Josh Stone at 2018-01-08 17:43:53

  2. This also affects mips and mipsel on Debian which prevents us from bootstrapping the Rust compiler for these architectures. Disabling debug symbols doesn't help unfortunately, unlike on armel and armhf.

    Both 32-bit MIPS architectures have unfortunately a memory limit of 2 GiB virtual address space per process.

    John Paul Adrian Glaubitz at 2018-05-21 21:30:04

  3. On Debian, for some packages, we use a gcc option: --param ggc-min-expand=10 on mips/mipsel.

    Does llvm have some similiar options?

    YunQiang Su at 2018-09-01 03:19:04

  4. Visiting for wg-debugging triage

    Its been so long since this issue was touched, we should have someone go and confirm that it still reproduces. I'm adding E-help-wanted to reflect that.

    @rustbot label: E-help-wanted

    Felix S Klock II at 2022-09-19 14:54:52

  5. I just ran a scratch build on Fedora 35 with full debuginfo enabled, and the 32-bit targets still failed.

    Here's i686 (i686-unknown-linux-gnu): https://koji.fedoraproject.org/koji/taskinfo?taskID=92194813

    error[E0786]: found invalid metadata files for crate `rustc_driver`
  --> src/librustdoc/lib.rs:41:1
   |
41 | extern crate rustc_driver;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: failed to mmap file '/builddir/build/BUILD/rustc-1.64.0-src/build/i686-unknown-linux-gnu/stage1/lib/rustlib/i686-unknown-linux-gnu/lib/librustc_driver-995e686975e44b6b.so': Cannot allocate memory (os error 12)
For more information about this error, try `rustc --explain E0786`.
error: could not compile `rustdoc` due to previous error

    Here's armv7hl (armv7-unknown-linux-gnueabihf): https://koji.fedoraproject.org/koji/taskinfo?taskID=92194812

       Compiling rustdoc v0.0.0 (/builddir/build/BUILD/rustc-1.64.0-src/src/librustdoc)
error[E0786]: found invalid metadata files for crate `rustc_monomorphize` which `rustc_driver` depends on
  --> src/librustdoc/lib.rs:41:1
   |
41 | extern crate rustc_driver;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: failed to mmap file '/builddir/build/BUILD/rustc-1.64.0-src/build/armv7-unknown-linux-gnueabihf/stage1/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_monomorphize-5bd38fc25e73bfbd.rlib': Cannot allocate memory (os error 12)
error[E0786]: found invalid metadata files for crate `rustc_monomorphize` which `rustc_interface` depends on
  --> src/librustdoc/lib.rs:49:1
   |
49 | extern crate rustc_interface;
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: failed to mmap file '/builddir/build/BUILD/rustc-1.64.0-src/build/armv7-unknown-linux-gnueabihf/stage1/lib/rustlib/armv7-unknown-linux-gnueabihf/lib/librustc_monomorphize-5bd38fc25e73bfbd.rlib': Cannot allocate memory (os error 12)
error[E0786]: found invalid metadata files for crate `askama_derive` which `askama` depends on
  --> src/librustdoc/html/layout.rs:10:5
   |
10 | use askama::Template;
   |     ^^^^^^
   |
   = note: failed to mmap file '/builddir/build/BUILD/rustc-1.64.0-src/build/armv7-unknown-linux-gnueabihf/stage1-tools/release/deps/libaskama_derive-20ded99bdf04465e.so': Cannot allocate memory (os error 12)
error[E0786]: found invalid metadata files for crate `askama_derive` which `askama` depends on
  --> src/librustdoc/html/render/print_item.rs:38:5
   |
38 | use askama::Template;
   |     ^^^^^^
   |
   = note: failed to mmap file '/builddir/build/BUILD/rustc-1.64.0-src/build/armv7-unknown-linux-gnueabihf/stage1-tools/release/deps/libaskama_derive-20ded99bdf04465e.so': Cannot allocate memory (os error 12)
error: cannot determine resolution for the derive macro `Template`
  --> src/librustdoc/html/layout.rs:42:10
   |
42 | #[derive(Template)]
   |          ^^^^^^^^
   |
   = note: import resolution is stuck, try simplifying macro imports
error: cannot find attribute `template` in this scope
  --> src/librustdoc/html/layout.rs:43:3
   |
43 | #[template(path = "page.html")]
   |   ^^^^^^^^
error: cannot determine resolution for the derive macro `Template`
  --> src/librustdoc/html/render/print_item.rs:52:10
   |
52 | #[derive(Template)]
   |          ^^^^^^^^
   |
   = note: import resolution is stuck, try simplifying macro imports
error: cannot find attribute `template` in this scope
  --> src/librustdoc/html/render/print_item.rs:53:3
   |
53 | #[template(path = "print_item.html")]
   |   ^^^^^^^^
memory allocation of 2097152 bytes failed
rustc exited with signal: 6 (SIGABRT) (core dumped)
error: could not compile `rustdoc` due to 8 previous errors

    (Note, the build version is the as-yet-unreleased 1.64 only because I was already doing release prep...)

    Josh Stone at 2022-09-19 23:42:03

  6. According to https://salsa.debian.org/rust-team/rust/-/blob/debian/sid/debian/rules?ref_type=heads#L238 this is still a problem on many architectures.

    mirabilos at 2024-06-02 23:45:44