Large array literal causes slow compilation

586a0a9
Opened by Joshua Landau at 2023-11-20 20:01:59

I have a large array of the form

static mut ARRAY: [AtomicU64; LARGE_NUMBER] = [
    ATOMIC_U64_INIT, ATOMIC_U64_INIT, ...
];

where LARGE_NUMBER = 512 * 1024. ATOMIC_U64_INIT is not Copy, so it's not obvious how to express this differently. I don't even think I even legally can use a [u64; LARGE_NUMBER] and transmute on use because u64 has aliasing guarantees that AtomicU64 breaks.

This ends up adding more than 10s to my compile times. time-passes, filtering out passes that take <0.1s above their baseline, shows

time: 0.594; rss: 322MB parsing
time: 0.295; rss: 349MB expansion
time: 0.244; rss: 404MB name resolution
time: 0.126; rss: 549MB lowering ast -> hir
time: 0.104; rss: 358MB region resolution
time: 0.148; rss: 358MB static item recursion checking
time: 0.497; rss: 359MB compute_incremental_hashes_map
time: 1.643; rss: 437MB item-types checking
time: 0.562; rss: 436MB const checking
time: 0.115; rss: 436MB privacy checking
time: 0.568; rss: 492MB MIR dump
time: 0.136; rss: 475MB death checking
time: 0.164; rss: 475MB stability checking
time: 0.559; rss: 475MB lint checking
  time: 3.098; rss: 523MB   translation item collection
time: 5.724; rss: 447MB translation

There's a full version available elsewhere, but it's not really more interesting.

  1. You can absolutely transmute if you use an UnsafeCell, which discard aliasing assumptions.

    #![feature(const_fn, integer_atomics)]

use std::cell::UnsafeCell;
use std::sync::atomic::AtomicU64;

pub struct SyncCell<T>(UnsafeCell<T>);
unsafe impl<T: Sync> Sync for SyncCell<T> {}

static XYZ: SyncCell<[u64; 1024*512]> =
    SyncCell(UnsafeCell::new([0; 1024*512]));

pub fn get_xyz() -> &'static [AtomicU64; 1024*512] {
    unsafe {&*(XYZ.0.get() as *mut _)}
}

fn main() {}

    Ariel Ben-Yehuda at 2016-10-14 09:18:21

  2. That's much better, thanks! I'll leave the issue open for now, since this indicates a bottleneck in translation, but it's no longer an immediate problem.

    Joshua Landau at 2016-10-14 10:34:17

  3. https://gist.github.com/Mark-Simulacrum/a83196ed84920c37804654ec39a7ebe9 is the full time-passes log from today, with the relevant passes:

    time: 1.191; rss: 206MB parsing
time: 1.578; rss: 391MB stability checking
time: 14.975; rss: 494MB        item-types checking                        ***
time: 2.139; rss: 532MB const checking
time: 3.860; rss: 556MB borrow checking
time: 1.065; rss: 556MB lint checking
  time: 2.718; rss: 556MB       translation item collection
time: 11.775; rss: 560MB        translation                                ***

    Mark Rousskov at 2017-05-14 23:46:55

  4. See https://github.com/rust-lang/rust/issues/52868#issue-345825358 for a quick test case demonstrating that compile time scales quadratically with array size.

    Ralf Jung at 2018-08-06 13:56:44

  5. Triage: Still reproduces with nightly-2023-11-17 using the following reproducer taken from #58523.

    pub fn main() {
    let mut _buf = vec![0u8; 1 << 32];
    (&[0u8; (1 << 32) - 8]);
}

    Martin Nordholts at 2023-11-20 20:01:59