[incremental] skip type-checking

ab48a74
Opened by Niko Matsakis at 2020-05-01 23:29:54

This is a kind of meta-bug aimed at skipping type-checking, which is (to some extent) the "holy grail" of incremental compilation. This bug only contains a list of issues and a task breakdown. For background on the general approach we have in mind, see this gist.

Substeps:

  • [x] https://github.com/rust-lang/rust/issues/45210: Introduce the ensure operation first described here and use it on the typeck_tables_of invocations. -- @theotherjimmy is on it
  • [x] https://github.com/rust-lang/rust/issues/45214: Isolate the "used trait imports" part of typeck tables into a distinct query
  • [ ] Enable serialization and deserialization of incremental results, at least for some types.
    • In particular, we will need to support serializing and re-loading Vec<DefId>. It wouldn't hurt to support () while we're at it.
    • This is likely best tackled by @michaelwoerister or @alexcrichton, but that may not be the case.

Ultimately, all of the the uses of typeck_tables_of must be analyzed and eliminated. These uses are listed here in order to mine for producing more subtasks. =)

<details><p>
  • [ ] src/librustc_passes/consts.rs:142: self.tables = self.tcx.typeck_tables_of(item_def_id);
  • [x] src/librustc_typeck/check_unused.rs:69: let tables = tcx.typeck_tables_of(item_def_id);
  • [ ] src/librustc_save_analysis/dump_visitor.rs:109: let tables = self.tcx.typeck_tables_of(item_def_id);
  • [ ] src/librustc_typeck/collect.rs:1155: return tcx.typeck_tables_of(def_id).node_id_to_type(hir_id);
  • [ ] src/librustc_typeck/collect.rs:1192: tcx.typeck_tables_of(owner).node_id_to_type(hir_id)
  • [ ] src/librustc_typeck/collect.rs:1244: tcx.typeck_tables_of(def_id).closure_tys()[hir_id]
  • [ ] src/librustc_privacy/lib.rs:503: replace(tables, tcx.typeck_tables_of(def_id))
  • [ ] src/librustc_const_eval/pattern.rs:606: self.tables = self.tcx.typeck_tables_of(def_id);
  • [ ] src/librustc_const_eval/eval.rs:381: tables: tcx.typeck_tables_of(def_id),
  • [ ] src/librustc_const_eval/eval.rs:770: let tables = tcx.typeck_tables_of(def_id);
  • [ ] src/librustc_borrowck/borrowck/mod.rs:100: let tables = tcx.typeck_tables_of(owner_def_id);
  • [ ] src/librustc_borrowck/borrowck/mod.rs:198: let tables = tcx.typeck_tables_of(owner_def_id);
  • [ ] src/librustc/cfg/construct.rs:57: let tables = tcx.typeck_tables_of(owner_def_id);
  • [x] src/librustc/ty/mod.rs:2108: self.typeck_tables_of(self.hir.body_owner_def_id(body))
    • not a direct user, see calls of body_tables() below
  • [ ] src/librustc/middle/intrinsicck.rs:136: let tables = self.tcx.typeck_tables_of(owner_def_id);
  • [ ] src/librustc_typeck/check/mod.rs:734: tcx.typeck_tables_of(body_owner_def_id);
  • [ ] src/librustc_typeck/check/mod.rs:756: tcx.typeck_tables_of(def_id).generator_sigs()[hir_id].map(|s| ty::Binder(s))
  • [ ] src/librustc_typeck/check/mod.rs:764: tcx.typeck_tables_of(def_id).closure_kinds()[hir_id].0
  • [ ] src/librustc_typeck/check/mod.rs:857: return tcx.typeck_tables_of(outer_def_id);
  • [ ] src/librustc_typeck/check/mod.rs:1152: tcx.typeck_tables_of(tcx.hir.local_def_id(it.id));
  • [ ] src/librustc_typeck/check/mod.rs:1574: tcx.typeck_tables_of(tcx.hir.local_def_id(e.node_id));
  • [ ] src/librustc_mir/transform/generator.rs:712: let interior = *tcx.typeck_tables_of(def_id).generator_interiors().get(hir_id).unwrap();
  • [ ] src/librustc_mir/hair/cx/mod.rs:96: tables: tcx.typeck_tables_of(src_def_id),
  • [ ] src/tools/clippy/clippy_lints/src/consts.rs:302: tables: self.tcx.typeck_tables_of(def_id),
  • [ ] src/librustc_privacy/lib.rs:518: let orig_tables = replace(&mut self.tables, self.tcx.body_tables(body));
  • [ ] src/librustc_privacy/lib.rs:672: let orig_tables = replace(&mut self.tables, self.tcx.body_tables(body));
  • [ ] src/librustc_driver/pretty.rs:505: self.tables.set(self.tcx.body_tables(id));
  • [ ] src/librustc_const_eval/check_match.rs:53: tables: self.tcx.body_tables(b),
  • [ ] src/librustc/lint/context.rs:636: self.tables = self.tcx.body_tables(body);
  • [ ] src/librustc/lint/context.rs:696: self.tables = self.tcx.body_tables(body_id);
  • [ ] src/librustc/middle/liveness.rs:530: let tables = ir.tcx.body_tables(body);
  • [ ] src/librustc/middle/reachable.rs:101: self.tables = self.tcx.body_tables(body);
  • [ ] src/librustc/middle/dead.rs:218: self.tables = self.tcx.body_tables(body);
  • [ ] src/librustc_mir/build/mod.rs:106: let gen_ty = tcx.body_tables(body_id).node_id_to_type(fn_hir_id);
  • [ ] src/librustc_mir/build/mod.rs:247: let closure_ty = tcx.body_tables(body_id).node_id_to_type(closure_expr_hir_id);
  • [ ] src/tools/clippy/clippy_lints/src/attrs.rs:172: is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value)
  • [ ] src/tools/clippy/clippy_lints/src/attrs.rs:180: ImplItemKind::Method(_, eid) => is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value),
  • [ ] src/tools/clippy/clippy_lints/src/attrs.rs:189: is_relevant_expr(tcx, tcx.body_tables(eid), &tcx.hir.body(eid).value)
  • [ ] src/tools/clippy/clippy_lints/src/functions.rs:154: let tables = cx.tcx.body_tables(body.id());
</p></details>

  1. I have implemented ensure on a branch. I'll make a PR

    Jimmy Brisson at 2017-10-11 18:00:22

  2. @theotherjimmy great, see #45210 too =)

    Niko Matsakis at 2017-10-11 18:01:08

  3. @nikomatsakis for markdown working under spoilers, please also use <p> tag:

    <details><p>
# newline required after <p>
# ... checkboxes here ...
</p></details>

    hcpl at 2017-10-11 18:11:04

  4. Added #45214, another step along the way.

    Niko Matsakis at 2017-10-11 18:16:04

  5. The PR I promised is #45228

    Jimmy Brisson at 2017-10-12 03:31:26

  6. For my next trick: Isolate the "used trait imports" part of typeck tables into a distinct query

    Jimmy Brisson at 2017-10-12 03:41:04

  7. It looks like @cjkenn has #45214, "used trait imports". I'll take a a look at serialization of Vec<DefId> and ().

    Jimmy Brisson at 2017-10-14 00:36:05

  8. Status of the issue:

    1. The "used trait imports" query exists and is used;
    2. Saving Vecs seems to be possible, but is not used right now;
    3. Some parts of the compiler use the query for only one field at a time (see below).

    About usage of typeck_tables_of and body_tables:

    <details><p>
    • for ensuring it runs
    librustc_typeck/check/mod.rs:1751:            tcx.typeck_tables_of(def_id);
librustc_typeck/check/mod.rs:1755:            tcx.typeck_tables_of(tcx.hir().local_def_id(it.hir_id));
librustc_typeck/check/mod.rs:2673:            tcx.typeck_tables_of(tcx.hir().local_def_id(e.hir_id));
    • for closure_kind_origins
    librustc_mir/borrow_check/diagnostics/conflict_errors.rs:194:                    let tables = self.infcx.tcx.typeck_tables_of(id.expect_local());
librustc_mir/borrow_check/diagnostics/mod.rs:104:                        self.infcx.tcx.typeck_tables_of(did).closure_kind_origins().get(hir_id)
librustc_mir/borrow_check/diagnostics/mod.rs:127:                    self.infcx.tcx.typeck_tables_of(did).closure_kind_origins().get(hir_id)
    • for concrete_opaque_types
    librustc_mir/borrow_check/type_check/mod.rs:1236:            &tcx.typeck_tables_of(anon_owner_def_id.expect_local()).concrete_opaque_types;
librustc_typeck/collect/type_of.rs:123:                            &tcx.typeck_tables_of(owner.expect_local()).concrete_opaque_types
librustc_typeck/collect/type_of.rs:431:            if !self.tcx.typeck_tables_of(def_id).concrete_opaque_types.contains_key(&self.def_id) {
    • for liberated_fn_sigs
    librustc_typeck/collect.rs:1478:                    let fn_sig = tcx.typeck_tables_of(def_id).liberated_fn_sigs()[hir_id];
    • for tainted_by_errors
    librustc_mir/const_eval/eval_queries.rs:294:            if let Some(error_reported) = tcx.typeck_tables_of(def_id).tainted_by_errors {
librustc_mir/interpret/eval_context.rs:407:                if let Some(error_reported) = self.tcx.typeck_tables_of(did).tainted_by_errors {
librustc_typeck/collect/type_of.rs:141:                                tcx.typeck_tables_of(owner.expect_local()).tainted_by_errors
    • for upvars_list
    librustc_mir/interpret/validity.rs:203:                    let tables = self.ecx.tcx.typeck_tables_of(def_id);
    • for user_provided_sigs
    librustc_mir/borrow_check/type_check/input_output.rs:39:            let typeck_tables = self.tcx().typeck_tables_of(self.mir_def_id.expect_local());
    • for node_type and node_type_opt
    librustc_mir/borrow_check/diagnostics/conflict_errors.rs:883:                                    .typeck_tables_of(def_id)
librustc_mir/borrow_check/diagnostics/mutability_errors.rs:510:            let tables = self.infcx.tcx.typeck_tables_of(def_id);
librustc_mir/borrow_check/universal_regions.rs:501:                    let tables = tcx.typeck_tables_of(self.mir_def_id.expect_local());
librustc_typeck/collect/type_of.rs:606:    let ty = tcx.diagnostic_only_typeck_tables_of(def_id).node_type(body_id.hir_id);
librustc_mir_build/build/mod.rs:89:                    let gen_ty = tcx.body_tables(body_id).node_type(id);
librustc_mir_build/build/mod.rs:144:                let gen_ty = tcx.body_tables(body_id).node_type(id);
librustc_mir_build/build/mod.rs:211:    let closure_ty = tcx.body_tables(body_id).node_type(closure_expr_id);
    • for general usage
    librustc_mir/borrow_check/mod.rs:140:    let tables = tcx.typeck_tables_of(def_id);
librustc_mir_build/hair/cx/mod.rs:59:        let tables = tcx.typeck_tables_of(src_def_id);
librustc_passes/intrinsicck.rs:135:        let tables = self.tcx.typeck_tables_of(owner_def_id);
librustc_passes/liveness.rs:685:        let tables = ir.tcx.typeck_tables_of(def_id);
librustc_privacy/lib.rs:357:    if tcx.has_typeck_tables(def_id) { tcx.typeck_tables_of(def_id) } else { empty_tables }
librustc_save_analysis/dump_visitor.rs:113:            self.tcx.typeck_tables_of(item_def_id)
librustc_trait_selection/traits/error_reporting/suggestions.rs:1246:                query_tables = self.tcx.typeck_tables_of(generator_did.expect_local());
librustc_driver/pretty.rs:333:            self.tables.set(self.tcx.body_tables(id));
librustc_lint/late.rs:108:        self.context.tables = self.context.tcx.body_tables(body);
librustc_lint/late.rs:184:        self.context.tables = self.context.tcx.body_tables(body_id);
librustc_mir_build/hair/pattern/check_match.rs:30:        MatchVisitor { tcx, tables: tcx.body_tables(body_id), param_env: tcx.param_env(def_id) };
librustc_passes/dead.rs:223:        self.tables = self.tcx.body_tables(body);
librustc_passes/reachable.rs:85:        self.tables = self.tcx.body_tables(body);
librustc_privacy/lib.rs:1092:        let orig_tables = mem::replace(&mut self.tables, self.tcx.body_tables(body));
librustc_privacy/lib.rs:1234:        let orig_tables = mem::replace(&mut self.tables, self.tcx.body_tables(body));
    </p></details>

    I think making closure_kind_origins, concrete_opaque_types and tainted_by_errors standalone queries may be worth it. liberated_fn_sigs, upvars_list and user_provided_sigs are only used once. Removing the body_tables alias may be worth it too.

    Camille Gillot at 2020-05-01 09:09:33

  9. I .. think this is fixed?

    Niko Matsakis at 2020-05-01 17:04:05

  10. But maybe there's room to do better

    Niko Matsakis at 2020-05-01 17:04:26

  11. @cjgillot it seems like it'd be good to have some concrete examples of cases where we don't get re-use, but should/could? (Not sure if you had some you were using for analysis.)

    Niko Matsakis at 2020-05-01 17:06:27