Array lengths don't support generic parameters.

3d642aa
Opened by Robin Lambertz at 2024-12-01 15:03:24

It would seem that when using size_of in const fn context, it fails to properly compute the size of generic types.

The following function fails

unsafe fn zeroed<T: Sized>() -> T {
    // First create an array that has the same size as T, initialized at 0
    let x = [0u8; std::mem::size_of::<T>()];
    // Then, transmute it to type T, and return it
    std::mem::transmute(x)
}

The error is the following :

error[E0277]: the trait bound `T: std::marker::Sized` is not satisfied
 --> src/main.rs:3:19
  |
3 |     let x = [0u8; std::mem::size_of::<T>()];
  |                   ^^^^^^^^^^^^^^^^^^^^^^ `T` does not have a constant size known at compile-time
  |
  = help: the trait `std::marker::Sized` is not implemented for `T`
  = help: consider adding a `where T: std::marker::Sized` bound
  = note: required by `std::mem::size_of`

This is very confusing because it complains that T doesn't have std::marker::Sized, even though it does have that bound.

Meta

rustc_version : rustc 1.20.0-nightly (ae98ebfcb 2017-07-20)

  1. EDIT: This can now be made to work on nightly compilers, although it's a bit awkward to use. See this comment.

    It would seem that when using size_of in const fn context, it fails to properly compute the size of generic types.

    The following function fails

    unsafe fn zeroed<T: Sized>() -> T {
    // First create an array that has the same size as T, initialized at 0
    let x = [0u8; std::mem::size_of::<T>()];
    // Then, transmute it to type T, and return it
    std::mem::transmute(x)
}

    The error is the following :

    error[E0277]: the trait bound `T: std::marker::Sized` is not satisfied
 --> src/main.rs:3:19
  |
3 |     let x = [0u8; std::mem::size_of::<T>()];
  |                   ^^^^^^^^^^^^^^^^^^^^^^ `T` does not have a constant size known at compile-time
  |
  = help: the trait `std::marker::Sized` is not implemented for `T`
  = help: consider adding a `where T: std::marker::Sized` bound
  = note: required by `std::mem::size_of`

    This is very confusing because it complains that T doesn't have std::marker::Sized, even though it does have that bound.

    Meta

    rustc_version : rustc 1.20.0-nightly (ae98ebfcb 2017-07-20)

    Robin Lambertz at 2021-09-12 09:38:49

  2. cc @eddyb -- was this intentional?

    Mark Rousskov at 2017-07-26 20:28:53

  3. This is a limitation of array length - it can't use any parameters in scope - which also came up during the stabilization of associated consts (cc @withoutboats do we have a common issue to use for this?) and the decision taken was to stabilize associated consts without it.

    The exact error here comes from the type parameter being resolved, but the ty::Generics and ty::GenericPredicates (bounds, including the default Sized on type parameters) of the array length are actually both empty. They have to be (for now), otherwise we get cycle errors.

    Eduard-Mihai Burtescu at 2017-07-27 05:00:29

  4. This example triggers a const-evaluation error instead (try on playpen):

    fn test<T: ?Sized>() {
    [0u8; std::mem::size_of::<&T>()];
}

    cc @GuillaumeGomez @oli-obk Why is the const-eval error printing broken still 😢?

    Eduard-Mihai Burtescu at 2017-07-27 05:04:02

  5. This makes me sad, as it's the one thing I wanted to do with https://github.com/rust-lang/rust/pull/42859 :cry:

    scottmcm at 2017-07-31 04:19:33

  6. Weird thing: You can... um... kind of work around it. (but perhaps you shouldn't)

    pub trait ConstSizeOf: Sized {
    const SIZE: usize = ::std::mem::size_of::<Self>();
}

impl<T> ConstSizeOf for T { }

    You get a, uh... warning... that sounds extremely serious and that seems to suggest that this will have a very high probability of blowing up in your face.

       Compiling playground v0.0.1 (file:///playground)
warning: constant evaluation error: the type `Self` has an unknown layout
 --> src/main.rs:2:25
  |
2 |     const SIZE: usize = ::std::mem::size_of::<Self>();
  |                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: #[warn(const_err)] on by default

    Finished dev [unoptimized + debuginfo] target(s) in 0.52 secs
     Running `target/debug/playground`

    But, um...

    it, uh...seems to work.

    fn main() {
    // using println instead of assert_eq! to make sure it isn't just optimized
    // away as undefined behavior
    println!("{:?}", <i8>::SIZE); // prints 1
    println!("{:?}", <i16>::SIZE); // prints 2
    println!("{:?}", <i32>::SIZE); // prints 4
    println!("{:?}", <i64>::SIZE); // prints 8
    println!("{:?}", <Vec<()>>::SIZE); // prints 24
    println!("{:?}", <Vec<i8>>::SIZE); // prints 24

    fn test_vec<T>() {
        println!("{:?}", Vec::<T>::SIZE);
    }
    fn test_tuple<T>() {
        println!("{:?}", <(T, T)>::SIZE);
    }
    test_vec::<()>(); // prints 24
    test_vec::<i8>(); // prints 24
    test_tuple::<()>(); // prints 0
    test_tuple::<i8>(); // prints 2
}

    Michael Lamparski at 2018-04-14 23:48:50

  7. Oops, never mind.

    // error: T: Sized is not satisfied
fn to_byte_array<T>() -> [u8; T::SIZE] {
    panic!()
}

    I could have sworn I've done something before with associated consts in array types, though...

    Michael Lamparski at 2018-04-14 23:55:43

  8. I think we shoud change the error message, this is very confusing now.

    Duy Do at 2018-04-15 00:57:18

  9. @ExpHP The problem is using type parameters (e.g. your T) in array lengths, everything else works.

    @juchiast The error message is "emergent" from the same reason we can't "just" allow this to work right now, the the only other solution I can think of is checking if type-parameters are "really in scope" but that would probably break existing code that doesn't need to look at type parameters.

    Eduard-Mihai Burtescu at 2018-04-17 10:49:25

  10. #![feature(const_fn)]
pub const fn sof<T:?Sized>() -> usize {
    10
}
fn to_byte_array<T>() -> [u8; sof::<T>()] {
     panic!()
}

    Trying this way results in compiler crash in nightly.

    error: internal compiler error: librustc/ty/subst.rs:456: Type parameter `T/#0` (T/0) out of range when substituting (root type=Some(fn() -> usize {sof::<T>})) substs=[]

thread 'main' panicked at 'Box<Any>', librustc_errors/lib.rs:499:9

    Any workarounds known?

    Denis Golovan at 2018-05-24 08:12:03

  11. @MageSlayer Nope, it just can't be supported yet, see https://github.com/rust-lang/rust/issues/43408#issuecomment-381945540 and previous.

    Eduard-Mihai Burtescu at 2018-05-24 08:48:46

  12. Here is my workaround in Serde for [u8; mem::size_of::<T>()]. It supports rustc 1.20+. Note that the type is at least as big as T rather than exactly as big as T, so instead of transmute you would need ptr::write / ptr::read to interact with the bytes.

    https://github.com/serde-rs/serde/blob/d07b62bba481af6603a736f027b5ebbb74a4a63a/serde/src/backport.rs

    David Tolnay at 2019-02-17 22:27:22

  13. Although it is surprising that this is not a standard language feature, I just discovered the crate generic_array which appears to achieve the desired effect.

    Russell at 2019-11-20 12:45:53

  14. fn test<const N: usize>() { let array = [0; N]; } Fails with error: array lengths can't depend on generic parameters.

    Pietro Gorilskij at 2020-01-26 17:41:19

  15. @gorilskij Hmm, that's a different bug, despite of the phrasing matching this issue.

    @varkor @oli-obk I think we should "just" make Rvalue::Repeat hold a ty::Const for the length, I don't foresee any issues arising from that. (More realistically, it should probably be a "broadcast assignment" statement, but either way, it should hold a ty::Const or rely on the one in the ty::Array type)

    Eduard-Mihai Burtescu at 2020-01-26 17:57:18

  16. @eddyb: I've opened https://github.com/rust-lang/rust/issues/68567 to make sure we don't lose track of that. I'll tackle it soon if no-one else wants to take it.

    varkor at 2020-01-27 12:01:14

  17. This example triggers a const-evaluation error instead (try on playpen):

    fn test<T: ?Sized>() {
    [0u8; std::mem::size_of::<&T>()];
}

    cc @GuillaumeGomez @oli-obk Why is the const-eval error printing broken still cry?

    I don't know if this is a known bug, but this snippet results in an ICE in both stable (rustc 1.40.0 (73528e339 2019-12-16)) and nightly (rustc 1.42.0-nightly (8a79d08fa 2020-01-27)) versions, link to playpen.

    Consoli at 2020-01-28 01:20:30

  18. Current output for some of these cases:

    https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=3b45759dad27210891b7bf46b7c60262

    error: constant expression depends on a generic parameter
 --> src/lib.rs:3:19
  |
3 |     let x = [0u8; std::mem::size_of::<T>()];
  |                   ^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this may fail depending on what value the parameter takes

error: constant expression depends on a generic parameter
 --> src/lib.rs:5:5
  |
5 |     std::mem::transmute(x)
  |     ^^^^^^^^^^^^^^^^^^^
  |
  = note: this may fail depending on what value the parameter takes

    https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=dead88aa04ad632cc47af563f34a8543

    error: constant expression depends on a generic parameter
 --> src/main.rs:2:11
  |
2 |     [0u8; std::mem::size_of::<&T>()];
  |           ^^^^^^^^^^^^^^^^^^^^^^^^^
  |
  = note: this may fail depending on what value the parameter takes

    https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=3447eb55e57acb584ed10a73655d7179 now works

    https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=ffb92ec39b7481bfd1b3f435257ed3ce still ICEs in the same way

    Esteban Kuber at 2020-05-16 01:49:09

  19. Here is an ICE I stumbled upon recently which seems relevant. Reporting as per @estebank's suggestion.

    use std::marker::PhantomData;

struct Buffer<T: ?Sized> {
    buf: [u8; Self::LEN],
    phantom: PhantomData<T>,
}

impl<T: ?Sized> Buffer<T> {
    const LEN: usize = 64;
}

fn main() {
    println!("Hello, world!");
}
    <details> <summary>Backtrace</summary> 
    error: internal compiler error: src/librustc/ty/subst.rs:565: type parameter `T/#0` (T/0) out of range when substituting (root type=Some(T)) substs=[]

thread 'rustc' panicked at 'Box<Any>', <::std::macros::panic macros>:2:4
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/libunwind.rs:86
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.44/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print_fmt
             at src/libstd/sys_common/backtrace.rs:78
   3: <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt
             at src/libstd/sys_common/backtrace.rs:59
   4: core::fmt::write
             at src/libcore/fmt/mod.rs:1063
   5: std::io::Write::write_fmt
             at src/libstd/io/mod.rs:1426
   6: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:62
   7: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:49
   8: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:204
   9: std::panicking::default_hook
             at src/libstd/panicking.rs:224
  10: rustc_driver::report_ice
  11: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:474
  12: std::panicking::begin_panic
  13: rustc_errors::HandlerInner::span_bug
  14: rustc_errors::Handler::span_bug
  15: rustc::util::bug::opt_span_bug_fmt::{{closure}}
  16: rustc::ty::context::tls::with_opt::{{closure}}
  17: rustc::ty::context::tls::with_opt
  18: rustc::util::bug::opt_span_bug_fmt
  19: rustc::util::bug::span_bug_fmt
  20: <rustc::ty::subst::SubstFolder as rustc::ty::fold::TypeFolder>::fold_ty
  21: rustc::ty::fold::TypeFoldable::fold_with
  22: <rustc::ty::subst::SubstFolder as rustc::ty::fold::TypeFolder>::fold_const
  23: rustc::ty::normalize_erasing_regions::<impl rustc::ty::context::TyCtxt>::subst_and_normalize_erasing_regions
  24: rustc_mir::interpret::operand::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::eval_operand
  25: rustc_mir::interpret::step::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::eval_rvalue_into_place
  26: rustc_mir::interpret::step::<impl rustc_mir::interpret::eval_context::InterpCx<M>>::run
  27: rustc_mir::const_eval::eval_queries::const_eval_raw_provider
  28: rustc::ty::query::__query_compute::const_eval_raw
  29: rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::const_eval_raw>::compute
  30: rustc::dep_graph::graph::DepGraph::with_task_impl
  31: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query
  32: rustc_mir::const_eval::eval_queries::const_eval_validated_provider
  33: rustc::ty::query::__query_compute::const_eval_validated
  34: rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::const_eval_validated>::compute
  35: rustc::dep_graph::graph::DepGraph::with_task_impl
  36: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query
  37: rustc_mir::const_eval::eval_queries::const_eval_validated_provider
  38: rustc::ty::query::__query_compute::const_eval_validated
  39: rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::const_eval_validated>::compute
  40: rustc::dep_graph::graph::DepGraph::with_task_impl
  41: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query
  42: rustc::mir::interpret::queries::<impl rustc::ty::context::TyCtxt>::const_eval_resolve
  43: rustc::ty::sty::Const::eval::{{closure}}
  44: rustc::ty::sty::Const::eval
  45: rustc::ty::structural_impls::<impl rustc::ty::fold::TypeFoldable for &rustc::ty::TyS>::super_fold_with
  46: <rustc_infer::traits::project::AssocTypeNormalizer as rustc::ty::fold::TypeFolder>::fold_ty
  47: rustc_infer::traits::project::normalize
  48: rustc_infer::infer::InferCtxt::partially_normalize_associated_types_in
  49: <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
  50: rustc::ty::context::GlobalCtxt::enter_local
  51: rustc_typeck::check::wfcheck::check_item_well_formed
  52: rustc::ty::query::__query_compute::check_item_well_formed
  53: rustc::ty::query::<impl rustc::ty::query::config::QueryAccessors for rustc::ty::query::queries::check_item_well_formed>::compute
  54: rustc::dep_graph::graph::DepGraph::with_task_impl
  55: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query
  56: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::ensure_query
  57: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  58: <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  59: __rust_maybe_catch_panic
             at src/libpanic_unwind/lib.rs:86
  60: rustc_hir::hir::Crate::par_visit_all_item_likes
  61: rustc_session::session::Session::track_errors
  62: rustc_typeck::check_crate
  63: rustc_interface::passes::analysis
  64: rustc::ty::query::__query_compute::analysis
  65: rustc::dep_graph::graph::DepGraph::with_task_impl
  66: rustc::ty::query::plumbing::<impl rustc::ty::context::TyCtxt>::get_query
  67: rustc::ty::context::tls::enter_global
  68: rustc_interface::interface::run_compiler_in_existing_thread_pool
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.43.1 (8d69840ab 2020-05-04) running on x86_64-unknown-linux-gnu

note: compiler flags: -C codegen-units=1 -C debuginfo=2 --crate-type bin

note: some of the compiler flags provided by cargo are hidden

query stack during panic:
#0 [const_eval_raw] const-evaluating `Buffer::buf::{{constant}}#0`
#1 [const_eval_validated] const-evaluating + checking `Buffer::buf::{{constant}}#0`
#2 [const_eval_validated] const-evaluating + checking `Buffer::buf::{{constant}}#0`
#3 [check_item_well_formed] processing `Buffer`
#4 [analysis] running analysis passes on this crate
end of query stack
error: aborting due to previous error
    </details>

    Kuba Valtar at 2020-05-23 19:06:21

  20. Thanks for the great work! Is there any chance this will be available soon?

    Hans at 2020-07-05 10:51:34

  21. Triage:

    The current output of https://github.com/rust-lang/rust/issues/43408#issuecomment-391628161 with the latest nightly:

    error: generic parameters may not be used in const operations
 --> src\main.rs:7:37
  |
7 | fn to_byte_array<T>() -> [u8; sof::<T>()] {
  |                                     ^ cannot perform const operation using `T`
  |
  = note: type parameters may not be used in const expressions
  = help: use `#![feature(const_generics)]` and `#![feature(const_evaluatable_checked)]` to allow generic const expressions

error: aborting due to previous error

    And with #![feature(const_generics)]:

    warning: cannot use constants which depend on generic parameters in types
 --> src\main.rs:8:30
  |
8 | pub fn to_byte_array<T>() -> [u8; sof::<T>()] {
  |                              ^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(const_evaluatable_unchecked)]` on by default
  = warning: this was previously accepted by the compiler but is being phased out; it will become a hard error in a future release!        
  = note: for more information, see issue #76200 <https://github.com/rust-lang/rust/issues/76200>

    It's no longer ICE but rejected by const_evaluatable_checked.

    Yuki Okushi at 2020-12-28 23:16:05

  22. This now appear to work-ish, but is a bit awkward to use:

    #![feature(generic_const_exprs)]

unsafe fn zeroed<T: Sized>() -> T 
where
    [(); std::mem::size_of::<T>()]:
{
    // First create an array that has the same size as T, initialized at 0
    let x = [0u8; std::mem::size_of::<T>()];
    // Then, transmute it to type T, and return it
    std::mem::transmute_copy(&x)
}

    Two things of note:

    • The weird where close with an empty bound is necessary, otherwise I get an error: unconstrained generic constant.
    • Transmute doesn't work, because the compiler fails to prove that [u8; size_of::<T>()] has the same size as T, so we have to work around this by using transmute_copy

    Robin Lambertz at 2021-09-12 09:37:50

  23. ~~Though not relevant to the compiler issue, for the sake of those who may come across this it is obligatory to point out that the above implementation of zeroed using std::mem::transmute_copy can invoke UB if std::mem::align_of::<T>() > 1, which is likely not the intent.~~

    Michael Lamparski at 2021-09-12 21:27:14

  24. Though not relevant to the compiler issue, for the sake of those who may come across this it is obligatory to point out that the above implementation of zeroed using std::mem::transmute_copy can invoke UB if std::mem::align_of::<T>() > 1, which is likely not the intent.

    @ExpHP I believe this is wrong, and my implementation is safe even if T has a higher alignment than 1. Heres what transmute_copy's documentation says:

    This function will unsafely assume the pointer src is valid for size_of::<U> bytes by transmuting &T to &U and then reading the &U (except that this is done in a way that is correct even when &U makes stricter alignment requirements than &T). It will also unsafely create a copy of the contained value instead of moving out of src.

    (Emphasis mine)

    I believe this means that it is correct even when &U (our output type) has a "stricter alignment" (higher alignment) than &[u8]. If this is wrong, then I think the documentation of transmute_copy should be changed to better define what stricter alignment means.

    Robin Lambertz at 2021-09-12 21:46:58

  25. @roblabla is right, transmute_copy uses std::ptr::read_unaligned internally, which will safely read the value of unaligned pointers, and the destination of the copy will be at an aligned location in memory.

    The docs for transmute_copy are sort of incorrect though. It never transmutes &T to &U, it obtains an unaligned pointer *const U by casting src as *const T as *const U, which it then reads using read_unaligned. If it ever did create an unaligned reference &U, then it would be UB, but it does not.

    Adam Gausmann at 2021-09-12 22:24:52

  26. possible solution:

    use std::mem::size_of;



struct Ts<T>(T);

impl <T> Ts<T>{
    const fn new(t:T) -> Self{
        Self(t)
    }
    const fn size(&self) -> usize{
        return size_of::<Self>() ;
    }
}
fn main(){
    const K:usize = Ts::<u32>::new(0).size();
    println!("{K}");
    let s = [0u8; K];
}

    S·c at 2023-06-16 11:01:21

  27. possible solution:

    use std::mem::size_of;



struct Ts<T>(T);

impl <T> Ts<T>{
    const fn new(t:T) -> Self{
        Self(t)
    }
    const fn size(&self) -> usize{
        return size_of::<Self>() ;
    }
}
fn main(){
    const K:usize = Ts::<u32>::new(0).size();
    println!("{K}");
    let s = [0u8; K];
}

    May the struct TSize be added enter std crate ?

    S·c at 2023-06-16 11:06:52

  28. Follow up to https://github.com/rust-lang/rust/issues/43408#issuecomment-629571166

    https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=ffb92ec39b7481bfd1b3f435257ed3ce now works by changing the feature

    Esteban Kuber at 2023-06-16 17:48:04

  29. It's no longer ICE

    @rustbot label -I-ICE

    Martin Nordholts at 2023-07-26 14:00:51