Tracking issue for RFC 1977: public & private dependencies

b0cddd6
Opened by srrrse at 2024-12-19 15:42:29

This is the tracking issue for rust-lang/rfcs#1977 - public and private dependencies in cargo.

See also https://github.com/rust-lang/cargo/issues/6129 for tracking on the Cargo side.

Unresolved Questions

  • [ ] Determining the impact & migration path
  • [ ] The default for unspecified pub/priv dependencies

Steps

Implementation history

  • https://github.com/rust-lang/rust/pull/57586
  • https://github.com/rust-lang/rust/pull/59335
  • https://github.com/rust-lang/cargo/pull/6653
  • https://github.com/rust-lang/cargo/pull/6772
  • https://github.com/rust-lang/cargo/pull/6962
  1. Summary

    RFC (original, superseded): #1977 RFC: #3516 Cargo tracking issue: https://github.com/rust-lang/cargo/issues/6129 Issues: https://github.com/rust-lang/rust/issues?q=is%3Aissue+is%3Aopen+label%3AF-public_private_dependencies Cargo issues: https://github.com/rust-lang/cargo/issues?q=is%3Aopen+is%3Aissue+label%3AZ-public-dependency Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#public-dependency

    This feature enables the ability to track which dependencies are publicly exposed through a library's interface. It has two sides to the implementation: rustc (lint and --extern flag), and cargo (Cargo.toml syntax, and passing --extern flags).

    This feature was originally specified in rust-lang/rfcs#1977, but was later down-scoped in rust-lang/rfcs#3516.

    About tracking issues

    Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label. Discussion comments will get marked as off-topic or deleted. Repeated discussions on the tracking issue may lead to the tracking issue getting locked.

    Unresolved Questions

    Steps

    • [ ] Implementation
      • [x] rust-lang/rust#57586
      • [x] rust-lang/rust#59335
      • [x] #67074
      • [ ] #71043
      • [ ] #119428
      • [x] https://github.com/rust-lang/rust/pull/122665
      • [x] https://github.com/rust-lang/rust/pull/122757
      • [x] rust-lang/cargo#6653
      • [x] rust-lang/cargo#6772
      • [x] rust-lang/cargo#6962
      • [x] rust-lang/cargo#12817
      • [x] rust-lang/cargo#13039
      • [x] rust-lang/cargo#13036
      • [x] rust-lang/cargo#13037
      • [ ] rust-lang/cargo#13038
      • [x] rust-lang/cargo#13308
      • [ ] rust-lang/cargo#14502
      • [ ] ~~Command to update Cargo.lock to minimal versions (rust-lang/cargo#4100)~~
      • [ ] ~~Make cargo publish use the minimal versions allowed by Cargo.toml~~
      • [ ] Ensure quality of error messages is sufficient
      • [ ] rust-lang/cargo#13095
      • [ ] Make exported-private-dependencies allow-by-default pre-2024 and deny-by-default in 2024+ / in the 2024-compatibility lint group
        • [ ] Perform a crater run and evaluate whether the pre-202x edition lint should be allow or warn (context from zulip)
    • [ ] Implement migration lints.
    • [ ] Add documentation to the edition guide.
    • [ ] Ensure ready for Rust 2024 stabilization.

    Non-blocking further improvements

    • rust-lang/cargo#13096

    Changes from RFC

    • RFC did not specify workspace inheritance behavior. With rust-lang/cargo#13125, we disallow it.

    cc @rust-lang/cargo @epage

    Ed Page at 2022-06-16 19:49:10

  2. Maybe it's useful to look at how this problem is approached in Haskell. Here are some relevant slides: https://wiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdf

    Here is an example of a subtle issue that occurs with private dependencies:

    Btw, I found it through this post: http://harry.garrood.me/blog/purescript-why-bower/

    Boscop at 2017-10-27 00:18:57

  3. Would Rust2018 be a good opportunity to make the braking changes to Cargo.toml?

    Jacob Finkelman at 2018-03-24 22:24:47

  4. Maybe. Though such changes should be designed and proposed soon, have an easy migration path, and preferably only affect a minority of existing crates.

    Simon Sapin at 2018-03-25 07:29:57

  5. Thought I just had related to ecosystems like futures that provide a "core" stable crate along with less stable utilities re-exported from a "facade": items from a public dependency re-exported through a private dependency should be treated as public. (This is likely how the implementation would behave anyway, but having a testcase for this would be nice). Basically how I see using a tightly coupled ecosystem like that work is that you would declare both the "facade" crate and "core" crate as dependencies, but only the "core" crate as public:

    [dependencies]
    futures = { version = "10.17", public = false }
    futures-core = { version = "1.2", public = true }
    

    But then, in code you would never import items from the "core" crate, instead you would simply have public APIs that return types which have been re-exported through the "facade" from the "core" crate and rely on the compiler to warn you if you accidentally use something that didn't originate in the "core" crate.

    Nemo157 at 2018-11-29 23:51:01

  6. I don't quite understand something about your example. Can you elaborate on what the Toml and the code in the futures crate look like?

    Jacob Finkelman at 2018-11-29 23:59:44

  7. @Eh2406 the setup's a little annoying to just explain, so here's a full compiling set of example crates: https://github.com/Nemo157/rust-44663-facade-example

    In this example futures-core exports a single trait Future, futures re-exports this trait from futures-core along with a "utility" Foo.

    Now mylib wants to use futures, including returning some Futures from its public API. But futures itself intends to have breaking changes to update utilities, users that care about interoperability should only use types from futures-core in their public APIs. Even as new major versions of futures are released they will all link against the same version of futures-core so the underlying traits will be interoperable.

    So, to have the compiler enforce this restriction the developers of mylib want to use the public/private dependencies feature, they have both futures and futures-core as dependency but only futures-core marked as public. But for ergonomics it's nicer to just use the re-exports directly from futures, instead of having to remember which parts of the API need to be imported from futures and which from futures-core.

    Nemo157 at 2018-11-30 00:20:31

  8. I'd like to work on this.

    Aaron Hill at 2018-12-15 09:59:25

  9. Thanks for offering to work on this! @alexcrichton, do you have any mentoring advice?

    I can at any time get a branch ready for the resolver part of cargo, it will be good enough to start experimentation, although not good enough to be ready to stabilize.

    Jacob Finkelman at 2018-12-16 16:34:25

  10. Sure! @Aaron1011 so this is split roughly into two features, one being rustc knowing what crates are in the "public interface" and a second being the support in Cargo's resolver to make various decisions differently. Are you interested in one of those in particular?

    Alex Crichton at 2018-12-17 17:16:27

  11. @alexcrichton: I've started working on the rustc part in a branch: https://github.com/Aaron1011/rust/commits/feature/pub-priv-dep, and I plan to submit a (WIP) PR soon. I might also be interested in working on the cargo part as well.

    Aaron Hill at 2018-12-17 17:41:36

  12. @Aaron1011 please cc me when you make that PR. I know nothing about the internals of Rust, so will be of no help, but I do want to follow the discussion.

    I will redouble my efforts to get the Cargo's resolver part mergeable. I have been enjoying ( and hating ) it because it is the hardest algorithmic problem I have ever worked on. I would be happy for some help, perhaps a new perspective will get me un-stuck.

    Jacob Finkelman at 2018-12-17 19:22:55

  13. @Aaron1011 ok and it seems like you're off to a great start! I'd be up for reviewing that PR, and if you've got any questions along the way just let me know

    Alex Crichton at 2018-12-18 16:18:13

  14. There was another usecase for this mentioned on i.rl.o, if cargo doc had a form of "local development" mode to build the documentation you care about when working on a crate, this would allow that mode to know exactly which dependencies it needs to produce the documentation for (all direct dependencies + their public dependencies).

    Similarly, if you're self-hosting documentation for a crate somewhere you may want to be able to generate a documentation bundle including the crate and all its public dependencies so that it is fully self-contained.

    Nemo157 at 2019-01-25 14:55:40

  15. The current command line syntax for passing private and public extern crates to rustc is

    --extern-private name=PATH
    --extern name=PATH
    

    (this differs from what was written in the RFC).

    I think the syntax would benefit from being slightly more general

    --extern:modifier name=PATH
    

    with private and public extern crates being written like this

    --extern:priv name=PATH
    --extern:pub name=PATH
    --extern name=PATH // OK, `pub` is the default
    

    (We may also want other modifiers, or multiple modifiers (--extern:m1:m2) in the future.)

    Vadim Petrochenkov at 2019-03-23 08:58:28

  16. @petrochenkov Do you want me to modify my PR to use that syntax instead?

    Aaron Hill at 2019-03-23 20:19:12

  17. @Aaron1011 A separate PR would probably be better, this is a novel syntax in general, it may require an FCP etc.

    Vadim Petrochenkov at 2019-03-24 16:10:46

  18. The Cargo frontend of the implementation has been merged

    Things still left to do:

    • Decide on how public/privates dependencies and dependency naming will interact.
    • Merge https://github.com/rust-lang/crates.io/pull/1685 (once the above issue is resolved)
    • Decide on a plan to rollout the change to cargo public (picking the minimal versions from Cargo.toml)

    Aaron Hill at 2019-04-30 20:14:55

  19. The discussion of the first point is continuing at https://github.com/rust-lang/cargo/issues/6129#issuecomment-464845052

    minimal versions is being tracked at https://github.com/rust-lang/cargo/issues/5657. It is implemented, but is not wide enough used to be stabilized as a cli flag. I doubt we can add it to cargo publish, without a magere rethink.

    Jacob Finkelman at 2019-05-02 01:13:02

  20. This is now available on the latest nightly (2019-05-07) if anyone wants to experiment with it (docs). Just add cargo-features = ["public-dependency"] to the top of Cargo.toml of your project to get things started. You can then look at the warnings produced by rustc to understand which dependencies are exposed in your public API.

    Eric Huss at 2019-05-07 01:40:04

  21. I like this feature! The produced warning messages were very clear. In 1 out of 5 crates trialed, I decided based on the results to make 2 dependencies private that were needlessly public. This makes me think there is value and wonder if there is an opportunity to stabilize just the information collection part of this sooner (public dependency attribute, rustc warnings, uploads to the crates.io index) and the dependency resolution changes (warnings or any resolution errors) later? In particular, wouldn't that allow progress on information collection and utility without serializing on rust-lang/cargo#5657? Or perhaps the tracking details just need to be updated here?

    Some additional minor feedback below:

    % rustc --version
    rustc 1.36.0-nightly (a784a8022 2019-05-09)
    % cargo --version
    cargo 1.36.0-nightly (759b6161a 2019-05-06)
    
    • If you label something public=true that isn't actually publicly exposed, you currently don't get a warning. Would a warning or lint for that case be an appropriate future addition?

    • With the cargo feature enabled, cargo doc currently fails, due to rustdoc not understanding the --extern-private flags as passed. Workaround is to temporarily remove the feature and all public attributes from deps, or replace --extern-private with --extern and run rustdoc manually.

    cargo doc --no-deps --open
     Documenting barc v1.2.0 (/home/david/src/body-image/barc)
    error: Unrecognized option: 'extern-private'
    
    error: Could not document `barc`.
    
    Caused by:
      process didn't exit successfully: `rustdoc --edition=2018 --crate-name barc barc/src/lib.rs --color always -o /home/david/src/body-image/target/doc --cfg 'feature="body-image"' --cfg 'feature="brotli"' --cfg 'feature="default"' --cfg 'feature="memmap"' --cfg 'feature="mmap"' --cfg 'feature="olio"' -L dependency=/home/david/src/body-image/target/debug/deps --extern body_image=/home/david/src/body-image/target/debug/deps/libbody_image-f6307513eafde4da.rmeta --extern-private brotli=/home/david/src/body-image/target/debug/deps/libbrotli-914be93b87d00a74.rmeta --extern-private bytes=/home/david/src/body-image/target/debug/deps/libbytes-60c7a2c9551c05c4.rmeta --extern-private flate2=/home/david/src/body-image/target/debug/deps/libflate2-3410ad175205616a.rmeta --extern http=/home/david/src/body-image/target/debug/deps/libhttp-1bcf97f938c6fdd7.rmeta --extern-private httparse=/home/david/src/body-image/target/debug/deps/libhttparse-b04e539fbadba356.rmeta --extern-private memmap=/home/david/src/body-image/target/debug/deps/libmemmap-b29e83fdba55bd43.rmeta --extern-private mime=/home/david/src/body-image/target/debug/deps/libmime-df20e72307836353.rmeta --extern-private olio=/home/david/src/body-image/target/debug/deps/libolio-c7a6754af0be8b56.rmeta --extern-private tao_log=/home/david/src/body-image/target/debug/deps/libtao_log-7da4b359386cbcf7.rmeta --extern-private tempfile=/home/david/src/body-image/target/debug/deps/libtempfile-5c5a144b50e3000b.rmeta -Z unstable-options --cfg barc_std_try_from` (exit code: 1)
    

    I'm happy to elaborate here or elsewhere if its useful.

    David Kellum at 2019-05-10 20:55:40

  22. If you label something public=true that isn't actually publicly exposed, you currently don't get a warning. Would a warning or lint for that case be an appropriate future addition?

    That sounds like a good idea to me!

    Implementing that will be a little tricky. Currently, rustc isn't explicitly aware of the underlying public-dependency feature - it simply treats all dependencies as public by default. We would want to avoid showing 'unecessary public dependency' warnings to users who don't have the public-dependency feature enabled, as these warnings would be both useless and unfixable.

    We would probably need to add a -Z public-dependencies flag to rustc to ensure that we only enable the lint when the cargo feature is also enabled.

    Is it necessary to make another RFC for something like this?

    Aaron Hill at 2019-05-10 21:29:02

  23. Or the alternative "--extern(:pub|:private)? name=PATH" syntax proposed by @petrochenkov could also get rustc that detail, if it chose to preserve the distinction for this? It sounds like an implementation detail to me, but what do I know! ☺

    David Kellum at 2019-05-10 23:38:17

  24. I just tried enabling this feature on another lib crate (also for the purpose of collecting information on public deps) and noticed that examples/*.rs are another interesting edge case. If the examples define any pub items (here really just for purpose of example) that reference the lib crate types, these will result in warnings of the pattern:

    warning: type `Foo` from private dependency 'libcrate' in public interface
    

    This is because each example is compiled, including as part of cargo test, with --extern-private libcrate=... Note this dependency isn't written in the Cargo.toml, its just implied for examples. Shouldn't examples be compiled with --extern:pub libcrate=... instead (using newer proposed syntax just for clarity)?

    Cc: @Eh2406 this last case (and possibly the one above about cargo doc) might be more specific to cargo?

    David Kellum at 2019-05-15 19:28:56

  25. The feature's implementation doesn't currently work correctly with transitive dependencies. If we load a crate named foo as a transitive dependency, we still mark it private (or not) by looking at the --extern foo option, even if refers to an unrelated crate and --extern options in general are entirely irrelevant to transitive dependencies.

    The dependency private-ness should be written into metadata and read from there for transitively loaded crates.

    This brings a question - what if the same crate is private direct dependency and a public transitive dependency (or vice versa)? Should it be treated as private or public?

    (I'm currently refactoring code in that area and may fix this issue as well.)

    Vadim Petrochenkov at 2019-10-15 17:37:57

  26. @Aaron1011 By the way, do you still have plans to implement the --extern:modifier syntax from https://github.com/rust-lang/rust/issues/44663#issuecomment-475853173?

    Vadim Petrochenkov at 2019-10-15 17:39:07

  27. @petrochenkov I like the idea of the modifier syntax, but I'm curious how that would work with getopts. If the intent would be to support multiple modifiers, would every permutation need to be registered as separate flags?

    Eric Huss at 2019-10-15 21:59:36

  28. how that would work with getopts.

    No idea :) I didn't look into implementation details, only suggested a syntax that's more generic than --extern-private.

    Vadim Petrochenkov at 2019-10-15 22:08:39

  29. I have posted a proposal for a new --extern flag syntax at #67074.

    Eric Huss at 2019-12-06 02:30:44

  30. I'm a bit confused about how re-exports are being treated. From the RFC: "Q: Can I export a type from a private dependency as my own?" says NO. However, it looks like various ways of re-exporting do not generate a warning:

    • pub extern crate private_crate; or pub use private_crate;
    • pub use private_crate::some_item;

    Is that intentional?

    Eric Huss at 2019-12-07 19:22:13

  31. @Eh2406 in https://github.com/rust-lang/rfcs/pull/3146#issuecomment-881957976 wrote

    The problem with "Public and private dependencies" is not the theory. It is fairly clear what we want it to mean. It is implementation that is the problem. In a SAT/SMT reduction of Dependency resolution it adds O( (numbers of packages considered) ^ 2) terms to the problem. More importantly in the Resolver as implemented it takes common use cases from sub second time to multiple hours time. There are a lot of changes to the Resolver that we can not stabilize because of limitations to performance or error messages even though we have a clear understanding of exactly what behavior we want and why.

    Is more written about this? It definitely does make it harder, but I am surprised it is so bad in practice when Cabal etc. do this today. Were you ret-conning existing deps as public deps? (I thought the plan was to keep them private which I have my doubts about but which I thought would not suddenly increase the costs.)

    John Ericson at 2021-07-17 22:04:35

  32. I wrote that from memory, and wanted to go back and reread the records (mostly https://github.com/rust-lang/cargo/issues/6129) before expanding on it. It seems like my memory had lost some of the subtlety of the situation. Sorry.

    There are some open question about behavior, best somerized at https://github.com/rust-lang/cargo/issues/6129#issuecomment-501783242. However, except for important surface level sintacs discussions, the available design space is pretty small. For various reasons there needs to be a "existing behavior" option in addition to "public" or "private" dependencies.

    As to the runtime overhead for existing dependencies it is not large. As everything is marked "existing behavior", so it can create no conflicts. Even in the current implementation where everything is marked as private, which means that this can only add a conflict for the parent, conflicts tend to be small and simple. However the Resolver is a tricky and organically grown peace of code. I am not willing to stabilize new behavior for it that is not end-to-end property tested. If you turn on public & private dependencies in our test case generator here, it will find a case where things take over 60 seconds to resolve. If you comment out that line or run the tests in release, it will find a case where our organically grown form of learned clauses has over simplified the problem, so the Resolver decided there is no valid solution, when there is one and the "Reduction to SAT" test can find it. I burnt myself out trying to find the sweet spot, a learned clause representation sparse enough for efficient back jumping with enough information to not miss cases.

    Since then, I have been spending my time on the PubGrub project in the hope that by copying from the state of the art and rewriting from first principles, we will have a more solid foundation for adding these kinds of features.

    The RFC pre dates the "Prior art" section. Do you have links to info about the corresponding features in "Cabal etc."?

    Jacob Finkelman at 2021-07-18 21:00:27

  33. @Eh2406 Sorry to leave your nice response hanging.

    The RFC pre dates the "Prior art" section. Do you have links to info about the corresponding features in "Cabal etc."?

    Off hand, just https://wiki.haskell.org/wikiupload/b/b4/HIW2011-Talk-Loeh.pdf linked in the other thread, and I found https://opam.ocaml.org/doc/External_solvers.html for OPAM which links some older research. But I would be happy to ask around to get more up to date info for Cabal.

    Since then, I have been spending my time on the PubGrub project in the hope that by copying from the state of the art and rewriting from first principles, we will have a more solid foundation for adding these kinds of features.v

    I read up a bit on that. Interesting and looks quite nice! It does seem the PubGrub algorithm out of the box is just for the all-public-deps case right? So if we switched to that it would be the solid foundation and the new features together?

    John Ericson at 2021-08-03 06:12:04

  34. PubGrub simplifies dependency resolution to a platonic ideal of it self. There is a lot of work to expand it to handle compatibility with all the gnarly details of Cargos resolver. A full discussion of each problem and are current thoughts for how to work around it is somewhat out of scope here, luckily @mpizenberg has written up a guide here with a section on "Advanced usage and limitations". We have a plan for how to reduce Cargo's "one per name and semver compatibility range" to PubGrub's "one per name". Figuring out how to extend that to also have "scoped goals" or "public dependencies", has not yet been figured out.

    Despite having to do a lot of work to get back to where we are, I am hopeful that building off this base will work out better because PubGrub simplifications run deep. It has combined many different concepts (that Cargo's resolver has as different types) into one comparatively simple "incompatibility" type. All the different aspects of the problem (Error reporting, Conflict resolution, Unit propagation, ... ) share that vocabulary type. In Cargo's resolver adding a new "learned clause" kind involves interacting the new enum variant with all the moving parts, and thanks to the end-to-end property tests finding all the corner cases where they interact badly. I am hopeful that in PubGrub it is "express your property's as incompatibilities" (ether by reduction to the existing simplified problem, or by adding a new API) and then all the existing pieces will work with it.

    Maybe someone will find a way to add it before the rebuilding compleats, but PubGrub is where I will be spending my time for now.

    Jacob Finkelman at 2021-08-04 03:30:37

  35. Maybe someone will find a way to add it before the rebuilding compleats, but PubGrub is where I will be spending my time for now.

    Yes I definitely agree with this approach, to be clear.

    We have a plan for how to reduce Cargo's "one per name and semver compatibility range" to PubGrub's "one per name". Figuring out how to extend that to also have "scoped goals" or "public dependencies", has not yet been figured out.

    Heh that's the tricky bit. I would say "one per name" means all public dependencies. Cargo's "one per name and semver compatibility range" to me feel more like an optimization that all-private/unrestricted deps don't cause too much duplicative bloat -- I can't see how having multiple versions that that are semver compatible would actually break things, just cause a nuscience.

    "scoped goals" sounds like private dependencies to me. Or rather, it's the combination of public and private deps that induces local non-global coherence issues. (All private means solving is trivial, all public means PubGrub as written works.)

    I do see how one can turn foo = ">5" into foo-6 = * or foo-7 = * or ..., but I worry it's a bit of a dead end. By all means, do something like that for now if you gets Cargo on PubGrub sooner, but I think changes will need to be made to PubGrub to actually track how packages are instantiated. Trying to defunctionalize something like foo ^>= 7 & bar(foo) ^>= 9 gets too messy.

    John Ericson at 2021-08-08 18:47:43

  36. Hi @Ericson2314 I'm just adding a few notes regarding PubGrub. We have to think more about the public/private feature, but I think it will be doable with a "proxy and bucket" scheme similar to the one presented in the guide. Anyway, we'll have a look at it and report how that goes.

    Matthieu Pizenberg at 2021-08-08 19:32:30

  37. @mpizenberg Thanks for that info! One thing I am slightly unclear on is how the "coherence" of public deps is maintained. I think you might need equality constraints, like a->c = a->b->c for the case where a has a public dep on b and c, and b also has a public dep on c.

    John Ericson at 2021-08-18 22:35:31

  38. @Ericson2314 for more details, I've written a new section in the guide specifically about implementing public/private dependencies. It is currently under review with @Eh2406 so it's not published yet. But if you want to participate in the review of this section it lives here: https://github.com/pubgrub-rs/guide/pull/4

    Matthieu Pizenberg at 2021-08-19 08:17:43

  39. What is the status of this effort? Is someone working on implementing it? What's the next step? I just got bitten by this and would be willing to help out with the implementation.

    Dirkjan Ochtman at 2021-10-25 09:48:00

  40. @djc we have a working prototype for very similar public/private scheme in pubgrub presented here: https://pubgrub-rs-guide.netlify.app/limitations/public_private.html

    So it will be on our radar with @Eh2406 as soon as we make progress on trying integrating pubgrub-rs with cargo, which will be the main focus of our effort in a couple months.

    As for current cargo, I have no information.

    Matthieu Pizenberg at 2021-10-25 10:18:44

  41. I imagine that getting pubgrub-rs to stable Cargo might take relatively long...

    Dirkjan Ochtman at 2021-10-25 10:32:25

  42. This comment up thread still describes the current state of things.

    Jacob Finkelman at 2021-10-25 14:24:04

  43. Thanks! So I've read the RFC in more detail. It seems to me that a lot of the complexity in implementing this RFC is in the impact on the resolver/dependency resolution, but that the other parts of the RFC would be quite valuable on their own and would be substantially less complex to implement. How do people feel about starting with an MVP that's basically only:

    • Dependencies can be marked as private in the Cargo manifest
    • Warnings will be issued during compilation if types or traits from private dependencies appear in public API

    Here's the angle I'm coming from: as the maintainer of a public library with fairly broad usage, exposing API types from a dependency publicly is a hazard because it means your semver-incompatible version bumps (which we generally like to avoid so as to avoid waste of downstream work) are coupled to your dependency's semver-incompatible bumps. Inclusion of duplicate downstream dependencies are something of a separate issue that's easier to diagnose nowadays than it was when the RFC was written (I usually use cargo deny for this).

    In particular, I'm coming at this as the rustls maintainer. For rustls 0.20 we tried to make the webpki and ring dependencies private, and it turns out that I missed one case, meaning we now might need to force another semver-incompatible bump on a large number of downstream dependencies for this reason. While this case is silly because I could have found it by grepping the source code, in general auditing a largish crate for types/traits that appear in the public API is non-trivial, and AIUI the compiler already implements most of this analysis to warn about intra-crate privacy issues.

    Dirkjan Ochtman at 2021-10-26 09:21:11

  44. What about mandating every crate to explicitly depend on public dependencies of their dependencies? That is, if you depend on A which has B as its public dependency (export some B types) you should add B in your cargo.toml explicitly with a explicit version. In this way, problem of finding a solution for a SAT system will reduced to checking a particular solution for a SAT system, which can be done in linear time.

    This is not perfect, but:

    1. In many cases we naturally want it, for example when we want to use util functions of B on the exported types from A.
    2. This restriction can be lifted any time in a backward compatible way.

    HKalbasi at 2022-01-02 20:30:30

  45. @HKalbasi this idea breaks for "facade crates," (like rand, futures, bevy, ...) where they reëxport all of the public dependency's types and you're expected to use them primarily, if not even exclusively, through the facade crate.

    Especially for e.g. rand, where using rand_core is encouraged if you're a library just implementing rand_core traits, whereas consumers want to consume via rand. For rand, the facade is meant to be open, and not opaque to the outside world (as opposed to e.g. bevy, which is primarily only really intended to be consumed from the outside as a whole, through the facade, and the inner crates could be considered an implementation detail).

    Crystal Durham at 2022-01-03 01:33:47

  46. I question the assumption that "public/private" is the right indicator for whether deps should be duplicated.

    It's easy to construct cases where public deps are nevertheless OK to duplicate. E.g. any time the final bin ends up publically depending on a lib at two different versions, but doesn't actually try to pass values between them.

    Conversely, even if a dep is private, it's reasonable to want to avoid unnecessary copies to reduce binary size and compile time. This is mentioned as an aside in various other comments, but it's a real issue.

    What I wish I had is a much simpler system, something like: dependencies never duplicated by default, but the final binary can explicitly allow duplicates in its Cargo.toml. (Or, for easier migration, allow by default but explicitly deny.) cargo update barfs it it can't find a solution, tells you what crates would need duplicates, and you edit Cargo.toml if that's what you want.

    This doesn't change the requirements for the underlying constraint solver, but the input to it gets conceptually simpler.

    Of course it would be better if intermediate libraries could also indicate which duplicates they cared about, but that starts adding complexities again. I'd much rather have a system where the final binary could just declare what to allow, than the current system where I'm fighting with Cargo that thinks it knows better.

    Jeff Petkau at 2022-04-17 03:39:25

  47. It's easy to construct cases where public deps are nevertheless OK to duplicate. E.g. any time the final bin ends up publically depending on a lib at two different versions, but doesn't actually try to pass values between them.

    And it's easy enough to allow the crate depending on two crates publicly exposing conflicting versions of a transitive dep to acknowledge and allow this.

    Conversely, even if a dep is private, it's reasonable to want to avoid unnecessary copies to reduce binary size and compile time.

    Everyone agrees that better controls over duplicated dependencies is a good thing. public/private dependencies is distinct from that, though; it's about allowing more things that should work to compile, and about controlling whether you expose a dependency or encapsulate it fully.

    tells you what crates would need duplicates, and you edit Cargo.toml if that's what you want.

    This doesn't change the requirements for the underlying constraint solver, but the input to it gets conceptually simpler.

    If it's still going to allow the cases where you've explicitly allowed duplicates (and notably, to determine what/how to duplicate), the constraint solver still needs to support working with said duplicates, so you don't simplify anything.

    This is a good tool to have in order to control dependency duplication (it's in cargo deny, in fact), but it doesn't influence what the constraint solver needs to be able to do.[^1]

    [^1]: Well, actually, the constraint solver doesn't even support solving for "fewest duplicates" over the whole tree currently. In fact, that's a new thing to optimize for. It doesn't really add complexity to the existing system — you can just model it as making everything semver compatible — but it is "a new thing the solver would need to do."

    Crystal Durham at 2022-04-17 04:42:56

  48. Visiting during T-compiler backlog bonanza.

    It would be nice if it had a S-tracking-* label that reflected its status.

    Hey @rust-lang/cargo , do you think this can be tagged with S-tracking-unimplemented? Or should it be classified as S-tracking-design-concerns?

    Felix S Klock II at 2022-05-06 14:36:08

  49. We might want to update cargo add so that when it infers a dependency version, it checks from among its dependencies' public dependencies. This will make it more likely that a compatible version will be selected.

    Ed Page at 2022-06-10 22:10:11

  50. Some behaviour from the compiler lint that I'm not sure about. This code produces a public-in-private warning on use_advice, even though that function is not accessible to the public. Is this intentional?

    # Cargo.toml
    cargo-features = ["public-dependency"]
    
    [package]
    name = "testfile"
    version = "0.1.0"
    edition = "2021"
    
    [dependencies]
    wasi = { version = "0.11", default-features = false }
    
    // src/lib.rs
    mod inner_private {
        pub fn use_advice() -> wasi::Advice {
            wasi::ADVICE_NORMAL
        }
    }
    pub fn do_stuff() {
        assert_eq!(wasi::ADVICE_NORMAL, inner_private::use_advice());
    }
    
    warning: type `Advice` from private dependency 'wasi' in public interface
     --> src/lib.rs:4:5
      |
    4 |     pub fn use_advice() -> wasi::Advice {
      |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      |
      = note: `#[warn(exported_private_dependencies)]` on by default
    
    warning: `testfile` (lib) generated 1 warning
    

    This can be worked around by using pub(crate), as in ce607bbcb605944073185c287716adc738685d2e

    Michael Howell at 2023-05-18 03:29:35

  51. We consider any pub item to be public, even if not actually reachable. This is also why for example mod sealed { pub trait Sealed {} } pub trait MyTrait: Sealed {} is allowed despite Sealed not being reachable.

    bjorn3 at 2023-05-18 08:30:25

  52. @Eh2406 and I met to talk about the status of this. This represents just the thoughts of the two of us and not the whole cargo team

    The pub/private RFC is quite old, predating

    • Editions
    • Package renaming
    • cargo add and soon cargo upgrade
    • The current RFC template

    And likely other factors that affect it.

    We are also concerned about the status of the proposed resolver changes. In addition to likely being long tail for work for this feature

    • It doesn't account for renames where someone might intentionally have two public versions of the same crate (e.g. tokio_03 and tokio1)
    • This makes semver compatibility more difficult as adding a public dependency is a breaking change. If the root depends on depA@1 and depB@1 and depA@1 adds a public dependency on depB@2, the proposed resolver change would error
    graph TD;
        root-->depA_v1;
        root-->depB_v1;
        depA_v1-->depB_v2;
    

    The hope was that the resolver changes would help with

    • Allowing the root crate to determine what version of a dependency to use when multiple dependencies depend on it with open ranges
    • Catch errors earlier with better messages when dependencies don't re-export their public dependencies and the root crate needs to keep them in sync
      • "depA::Type is not compatible with depA::Type" isn't helpful

    We propose that someone

    • Write up a new RFC
      • Take into account what has changed since the original
        • cargo add and cargo upgrade keeping public dependencies in mind when selecting versions can help reduce the impact of the loss of the Resolver change
        • An edition could possibly turn the warning of private-dependencies being exposed into an error though automatic migration might be difficult as we'd need to take a rustc warning and change Cargo.toml
      • Move Resolver and Minimal Version changes to Future Possibilities and sketch out what we could possibly do for the Resolver changes to allow them to be added later
        • Mutually-exclusive / global features?
        • Extend workspace.resolver with overrides?
          • Ignore versions (additional use case: cargo audit could help auto-generate some of these)
          • Specify relations between dependencies (imitate original proposed resolver change)
          • Declare a dependency as a singleton (a top-down version of https://github.com/rust-lang/rfcs/pull/3251 or links workaround)
    • Evaluate all backlinks to issues/PRs that reference the tracking issue and RFC for workflows, ensuring they are either covered by the new RFC or that an open Issue exists to represent the workflow we are no longer covering

    Some of the remaining motivations include:

    • private-in-public warning especially for From
      • Also help stdlib from accidentally exposing dependencies, like hashbrown
    • New cargo doc --public for generating docs to publish for a crate that includes its public dependencies
    • cargo doc default case limits deps to only recurse into public dep of deps (opt-in?)
    • cargo add improvements
    • cargo semver check (obi1kenobi/cargo-semver-checks#121 among others)
    • "crabi" might find it useful?

    Ed Page at 2023-06-02 01:21:14

  53. For a new RFC, something that should IMHO be listed as prior art, maybe also be a comparison point ("why is the proposed solution better than") is https://github.com/davidpdrsn/cargo-public-api-crates.

    Jonas Platte at 2023-06-02 06:57:51

  54. FYI I've created a superseding Pre-RFC

    Ed Page at 2023-10-13 17:14:23

  55. Oh awesome! I left a comment over there, and thanks!

    EDIT: this was the wrong thread for this comment and I apologize

    Alex Crichton at 2023-10-14 14:50:38

  56. Hey,the unstable feature was existed and I think This new features @epage mentions may be not include the unstable feature. Is it right? I'm really looking forward to contributing to the new features. :)

    Lin Yihai at 2023-11-19 15:11:48

  57. I've updated the tracking issue to account for rust-lang/rfcs#3516.

    Ed Page at 2023-11-22 03:08:08

  58. @Muscraft the new RFC didn't cover workspace inheritance. rust-lang/cargo#12817 implemented it so it can be specified in the workspace and the package can override it. Does that sound reasonable?

    Ed Page at 2023-11-26 02:59:55

  59. I somewhat lean towards saying the workspace can't specify public because it seems like that is something we don't want people overspecifying. Unsure how bad that'd be for the implementation or what our other support cases are like.

    EDIT: looks like we prevent optional from being in the workspace, so we have precedence for that.

    Ed Page at 2023-11-26 03:03:20

  60. Yup, I agree workspaces should not be allowed to specify public.

    Dirkjan Ochtman at 2023-11-26 09:49:58

  61. @alexcrichton looking closer at this, it seems like it'd be reasonable to block public in workspace.dependencies, like optional. Since you did rust-lang/cargo#12817, any thoughts on this?

    Ed Page at 2023-11-27 16:23:16

  62. @epage are you sure you linked the correct issue/PR?

    Dirkjan Ochtman at 2023-11-27 16:53:47

  63. Fixed. Forgot what repo I was in.

    Ed Page at 2023-11-27 16:57:56

  64. IIRC one of the motivaitons for blocking optional in workspace dependencies was that it had interactions with how features were calculated and it seemed best to not accidentally implicitly add features to crates via workspace dependencies. In my opinion similar logic isn't applicable to public and it's something I'd personally like to see available at the workspace.dependencies level. My usage of workspace dependencies and public/private is typically along the lines of "this is public for all crates using this" or the opposite.

    For example when I pull in anyhow that's because nearly all crates will use anyhow::Result<T> in their APIs. Alternatively crates like libc are likely to be a private dependency.

    So while I don't disagree that it would be conservative to block public, I'd personally still like it to avoid the repetition of writing it everywhere. Please feel free to overrule me however!


    Another possible interesting case study is default-features. That's a boolean setting for workspace dependencies which is a bit fiddly, but I think that primarily stems from its "default on" behavior as opposed to default-off like other boolean flags like optional. In that sense default-features may be different enough to not help guide public too.

    Alex Crichton at 2023-11-27 17:14:11

  65. I feel like because the hazard is on the side of defaulting to public = true, workspace dependencies and things like that have defaulted to requiring workspace = true explicitly in a package manifest. If the workspace makes a dependency public, that will not be clear when reviewing the package manifest, which seems like it might cause issues.

    On the other hand, it might be okay to allow workspace.dependencies that specify workspace = true to inherit the public setting from the workspace too. But, it doesn't seem crazy to suggest that you might want some dependency to be public in maybe some inner-level mostly internal-facing dependencies while you want to hide it from the outer-level public-facing dependency (for easier sharing across a single-organization tree of crates).

    Finally, I also feel like there's a case here for saying, we don't create any new problems by denying the use of public at the workspace level now because we can start enabling it later, whereas allowing it now with semantics that turn out to be problematic would be unfortunate.

    Dirkjan Ochtman at 2023-11-27 17:23:53

  66. In reflecting back, I wish workspace dependencies only focused on the Source and didn't allow specifying the rest. There are times to keep the rest in sync but I feel like that can lead to false sharing which can cause its own problems and having them explicit would make things clearer / more approachable.

    However, the work to get there would take a lot of work and could have some downsides, so I'm not wanting to advocate for it now.

    Where that leaves us for this, i'm less sure. I do find it appealing that erroring now leaves the door open in the future for not erroring.

    Ed Page at 2023-11-27 18:08:48

  67. @Muscraft and I talked some more and I think we'll not allow public at the workspace level.

    • Its a form of false-sharing. Its de-duplication of implementation and not requirements (new uses of a dep could come up that don't need it)
    • It mirrors optional
    • You then can't inherit it as a build or dev dependency

    Maybe if we had "grouped" inheritance (named, mutually exclusive sets of fields/deps that you can inherit) as that would mean you can then define things by policy rather than globally.

    Ed Page at 2023-11-30 21:33:25

  68. Regarding cargo fix support, I was hoping that rustc would give cargo enough information to intercept the warning and add machine-applicable information to it but apparently not:

    {
      "reason": "compiler-message",
      "package_id": "cargo-pub-priv 0.1.0 (path+file:///home/epage/src/personal/dump/cargo-pub-priv)",
      "manifest_path": "/home/epage/src/personal/dump/cargo-pub-priv/Cargo.toml",
      "target": {
        "kind": [
          "lib"
        ],
        "crate_types": [
          "lib"
        ],
        "name": "cargo-pub-priv",
        "src_path": "/home/epage/src/personal/dump/cargo-pub-priv/src/lib.rs",
        "edition": "2021",
        "doc": true,
        "doctest": true,
        "test": true
      },
      "message": {
        "rendered": "warning: trait `FooTrait` from private dependency 'foo' in public interface\n  --> src/lib.rs:11:1\n   |\n11 | pub trait UseFoo: foo::FooTrait {}\n   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\n",
        "children": [],
        "code": {
          "code": "exported_private_dependencies",
          "explanation": null
        },
        "level": "warning",
        "message": "trait `FooTrait` from private dependency 'foo' in public interface",
        "spans": [
          {
            "byte_end": 189,
            "byte_start": 158,
            "column_end": 32,
            "column_start": 1,
            "expansion": null,
            "file_name": "src/lib.rs",
            "is_primary": true,
            "label": null,
            "line_end": 11,
            "line_start": 11,
            "suggested_replacement": null,
            "suggestion_applicability": null,
            "text": [
              {
                "highlight_end": 32,
                "highlight_start": 1,
                "text": "pub trait UseFoo: foo::FooTrait {}"
              }
            ]
          }
        ]
      }
    }
    

    I wonder how receptive the compiler folks would be for adding one-off data like that...

    The next best alternative I can think of is for the lint to only be a warning in 2024, rather than an error. However, I had hoped for it to be a warning on stabilization and we decided not to...

    Ed Page at 2023-11-30 22:43:53

  69. Really terrible idea that might be just enough to get over the hump for cargo fix: we parse message.message looking for "from private dependency '{name}' in public interface".

    Ed Page at 2023-11-30 22:55:52

  70. rust-lang/cargo#13046 added cargo add support, assuming workspace inheritance for public will be removed. If we decide to keep it, we'll need to go back and update it.

    Ed Page at 2023-12-01 17:17:28

  71. @linyihai one item that i didn't create an issue for is

    verify public is respected recursively

    Thats because I'm unsure if this is already covered or not.

    • Do we have a test for it?
    • If not, we should write it
    • If it fails, then it might be a bit of work to fix

    Ed Page at 2023-12-11 21:17:00

  72. Do we have a test for it?

    After Checking the test cases contains cargo-features = ["public-dependency"], I didn't found the scenior.

    If not, we should write it

    @epge I would be pleasure to add it. But I am not sure how to verify public is respected recursively . To be honest, I'm not quite sure what does public is respected recursively means

    Lin Yihai at 2023-12-12 02:50:16

  73. @epge I would be pleasure to add it. But I am not sure how to verify public is respected recursively . To be honest, I'm not quite sure what does public is respected recursively means

    I'm going to give an example that assumes #71043 is resolved. It might be more finicky to create an example without it resolved; I'm unsure.

    pub:

    pub struct A;
    

    parent has a public dependency on pub

    pub use pub::*;
    

    grandparent has a public dependency on parent:

    pub use parent::*;
    

    Does how cargo interact with rustc correctly to handle not warn? What other variations might we run into?

    Ed Page at 2023-12-12 03:02:50

  74. #13046 added cargo add support, assuming workspace inheritance for public will be removed. If we decide to keep it, we'll need to go back and update it.

    nit: this should be https://github.com/rust-lang/cargo/pull/13046 — as-written it refers to #13046 on the rust-lang/rust repo.

    Jon Gjengset at 2023-12-17 15:44:00

  75. RFC did not specify workspace inheritance behavior. With https://github.com/rust-lang/rust/issues/13125, we disallow it.

    (nit: the link should be https://github.com/rust-lang/cargo/pull/13125.)

    Lin Yihai at 2023-12-29 09:22:06

  76. Thanks, fixed!

    Ed Page at 2023-12-29 12:57:27

  77. I think the decision to require a key (public = true) that breaks old cargo versions is a mistake. It shouldn't have been implemented in such a way that it produced a hard error in cargo in the first place, but now many crates in the ecosystem will be prevented from using this highly desirable feature for years.

    At the same time I understand that public = true is the most ergonomic option. Can we at least provide an alternative key, for crates that care about MSRV?

    Tyler Mandry at 2024-01-05 02:15:27

  78. I can understand the desire for this to not require an MSRV change. Part of the problem is historical, that the original feature was designed to be very different than what we have today. Part of the problem is not require an MSRV bump is very out of the ordinary with the only cases I can think of being package.rust-version and lints.

    However, I don't feel a redundant field would be appropriate.

    We could consider dropping the nightly requirement now so we at least have a couple release lead time on this problem (with the risk of stabilzation being even further out than that with the state of the compiler side).

    I do think this conversation represents a deeper problem: a push towards stagnation due to MSRV. I'm working on the update for the MSRV resolver RFC that I'm hoping will help lay out a better path for MSRV policies that can help push us out of this stagnation, shifting costs to those that need old versions rather than being old Rust versions being a millstone around the communities neck.

    Ed Page at 2024-01-05 02:37:16

  79. I started on this because I thought public/private deps would weigh on the ability of a library crate to be linked dynamically within a larger build graph, but I don't believe that to be true anymore. So if the cost of not being able to adopt this feature is only local to a crate, it doesn't matter that much.

    That being said, I still think we can do better in the future.

    Part of the problem is not require an MSRV bump is very out of the ordinary with the only cases I can think of being package.rust-version and lints.

    I can throw any random key in my Cargo.toml and will only get a warning that it wasn't used. That is, I believe, what cargo should have continued doing with the public key. Cargo attempted to approach stability issues the same way the language does, by gating features with hard errors, when in fact I believe it needs a different approach.

    Your RFC gets this right, by phasing in enforcement as a warning (with opt-in, even) and turning it into a hard error over an edition. I can think of other approaches that use this "two factor" opt-in idea to get away without waiting for an edition.

    We could consider dropping the nightly requirement now so we at least have a couple release lead time on this problem (with the risk of stabilzation being even further out than that with the state of the compiler side).

    Yes, I think we should do that in any case.

    I do think this conversation represents a deeper problem: a push towards stagnation due to MSRV. I'm working on the update for the MSRV resolver RFC that I'm hoping will help lay out a better path for MSRV policies that can help push us out of this stagnation, shifting costs to those that need old versions rather than being old Rust versions being a millstone around the communities neck.

    Respectfully, I think we still have to take version skew issues seriously in the design of our package manager. Shifting the cost to the users who want or need the most stability that might help motivate updates in some instances, but in others it will just make Rust a less viable option. Having access to recent versions of ecosystem crates is a security matter.

    There is a fundamental tension between stability and stagnation. How confident can you be that your MSRV RFC will modify the revealed preferences of ecosystem libraries and their users? There will always be a cost to updating unless we take a very hard line stance on backward compatibility, which historically we have not been willing to do (and I don't think we should).

    Tyler Mandry at 2024-01-12 02:57:36

  80. I can throw any random key in my Cargo.toml and will only get a warning that it wasn't used. That is, I believe, what cargo should have continued doing with the public key. Cargo attempted to approach stability issues the same way the language does, by gating features with hard errors, when in fact I believe it needs a different approach.

    Its not as simple as "it was ignored before so it should be fine to not error while unstable". Depending on how the key changes behavior, ignoring it would run counter to the users intention. While we can't go back in time to ensure the key was never allowed, the unstable period offers us the chance to ensure users will get a hard error, telling them that the feature they are trying to use is unsupported and cargo cannot fulfill their intent. This also discourages users mixing nightly and stable code where an upgrade of stable can break them because the stabilized form of the feature is different.

    To balance this out, I was the one who advocated for [lints] to not error on stable (and I went out of my way in the implementation to deal with nightly users in case the format changed). I also was originally advocating for it here. We should be looking for opportunities where it works to not error while its unstable. Its just not a blanket thing we should do.

    Ed Page at 2024-01-12 03:21:47

  81. We discussed the blocking feature gate in today's cargo meeting and I've posted rust-lang/cargo#13307 to clarify the documentation to encourage more non-blocking gates and I've opened rust-lang/cargo#13308 to change our feature gate here.

    Ed Page at 2024-01-16 19:03:49

  82. Thanks @epage, I really appreciate your receptiveness here!

    Tyler Mandry at 2024-02-14 01:23:55

  83. See https://github.com/rust-lang/rust/pull/121710#issuecomment-2016344080.

    The non-blocking gate -Zpublic-dependency turns out blocking bootstrapping rustc, with a flood of exported_private_dependencies lint errors. That's because cargo-features = ["public-dependency"] is scoped to a certain crate, but with -Zpublic-dependency the entire dependency graph starts passing --extern priv:<crate> unconditionally.

    Weihang Lo at 2024-03-23 04:48:46

  84. Probably worth keeping both opt-ins.

    Ed Page at 2024-03-23 16:05:22

  85. @rustbot labels +E-help-wanted

    If this item is going to make it into Rust 2024 (and it's OK if it doesn't), we're going to need some help to check off the remaining items above.

    As @davidtwco describes:

    I think the compiler's implementation needs to change quite a bit. We currently compute the private-ness of a dependency and then that is used by the lint whenever we see an item from that dependency, regardless of the path you access the item from. I think that's confusing but also if an intermediate dependency changes which of its dependencies it re-exports from then something that didn't lint might start linting and all sorts of things like that. However, the implementation of lint uses our visibility infrastructure, at which point it's far too late to be able to know which path you took to reference an item, that information isn't really available after name resolution. I experimented with trying to stash away some information that I could later use to do this, but nothing felt right.

    We'll mark this as E-help-wanted. If you're interested in seeing this happen in time for the edition and pitching in here, please comment on this Zulip thread.

    Travis Cross at 2024-05-28 23:00:26

  86. @rustbot labels -A-edition-2024 +A-maybe-future-edition

    Without an owner driving this work forward, this is looking unlikely for this edition, so let's drop that label and mark this as a possibility for a future edition.

    Travis Cross at 2024-06-04 19:28:44

  87. Here is proof why from field proposed in RFC 3516 (combined with package renaming and depending on different versions of the same package) is absolutely necessary. Let's assume we are authoring binary crate foo 1.0, which depends on libraries bar 1.0 and baz 1.0. Also bar 1.0 depends publicly on quux 1.0 and baz 1.0 depends publicly on quux 2.0. quux 1.0 and quux 2.0 have two incompatible versions of QuuxType. bar 1.0 has function produce_quux, which produces QuuxType. And baz 1.0 has function consume_quux, which consumes QuuxType. We want in our binary crate to produce QuuxType using produce_quux and then pass it to consume_quux. At first sight this seems impossible, but using from fields and other trickery with can solve this problem!

    We should write this Cargo.toml:

    [package]
    name = "foo"
    version = "1.0"
    
    [dependencies]
    bar = "1.0"
    baz = "1.0"
    quux_from_bar = { package = "quux", from = ["bar"] }
    quux_from_baz = { package = "quux", from = ["baz"] }
    

    (Of course, it is possible just to specify versions for quux_from_bar and quux_from_baz explicitly. But this will mean that we should update them manually every time we upgrade bar or baz. This feels wrong.)

    Then we should call produce_quux in our program. This will create type quux_from_bar::QuuxType (and we will be able to name it!). Then we should write our own converter, which will convert this type to quux_from_baz::QuuxType. And we will be able to actually write prototype for this converter: fn convert(x: quux_from_bar::QuuxType) -> quux_from_baz::QuuxType! And then we should pass resulting type to consume_quux.

    So, I hope I provided proof explaining why Cargo.toml above should be supported.

    There are 2 other alternatives:

    • Whole crate quux should be re-exported from bar and baz. But then this should be widely established practice. I. e. everyone should know about it and everyone should be encouraged to reexport their public dependencies (as a whole!). Blog post should be published in https://blog.rust-lang.org/ , which should state that everyone should reexport their public dependencies (there already was precedent in the past for such posts: there was post on lockfiles: https://blog.rust-lang.org/2023/08/29/committing-lockfiles.html ). And this should be also stated explicitly in Rust book and in Rust API Guidelines ( https://rust-lang.github.io/api-guidelines/ ). And we should add a lint. (It is possible that we will settle on name of a module, for example, public_reexports, which will contain all such reexported crates.)
    • Same as previous point, but this reexport should be automatically provided by compiler. I. e. if crate apple depends on crate banana publicly, then the compiler should automatically generate module named banana in apple (or possibly public_reexports::banana), which consists of whole banana API

    So here is my trilemma: either we should support Cargo.toml provided above (with some possible syntactical differences), either we should publish blog post on https://blog.rust-lang.org/ saying everyone should reexport their public dependencies, either the compiler should generate these reexports automatically. (Edit: of course, this is okay to choose multiple options here.)

    Of course, the second approach is the worst, because it pushes a lot of work on all maintainers

    Askar Safin at 2024-12-19 15:11:02

  88. @safinaskar let's be careful in setting up false dichotomies. Those are not the only two options. It would be great to have a path for improving this behavior. However, that idea was marked as a future possibility for a reason. In design and implementation work, we need to stay focused on the minimal set or else we'll never be able to make progress. The previous RFC tried to solve this problem, in a different way, and that was responsible for this RFC languishing for years and precluded us from noticing other issues that blocked this effort (the lint implementation). Let's separate out the conversation of supporting public/private dependencies in the first place from how we can take advantage of it in the future.

    Ed Page at 2024-12-19 15:42:29