create_dir_all can't create directories through a dangling symlink.

910330c
Opened by Paul Dicker at 2023-01-26 20:24:51

Currently create_dir_all first does a stat on the file to see if it is a directory, and if not tries to create the parent dirs. This does not take into account that stat can fail if the path is a dangling symlink. The classic mkdir -p just tries to create a directory and see why it fails. This spares a syscall and handles dangling symlinks transparently.

On Windows this method hit two problems:

  • When we have to recurse all the way op to the disk letter, create_dir_all fails. You can't mkdir c:\ or \\host\share. But we can just stop recursing in time.
  • Windows can't create a directory through a dangling symlink. We would have to read the symlink (and possibly resolve it if relative), and create the directory at the target location ourself.

Would it make sense to go trough this trouble?

I think it is best to only fix this for create_dir_all and to document the normal create_dir can not create a directory trough a dangling symlink on Windows.

  1. I'm not sure I quite understand the Windows parts here, could you elaborate? For example, if we just tried the syscall and then backtracked on Windows, what would the problem be specifically?

    Just doing the syscall and inspecting the error seems pretty reasonable to me for Unix!

    Alex Crichton at 2016-02-12 18:51:31

  2. On Unix this way the code will just work with dangling symlinks.

    On Windows there is no way to create a directory through a dangling symlink. So if creating a directory fails but the parent exist, we have to do another check for dangling symlinks. And if there is one, read the symlink and create the directory at the target.

    So there is no unfixable problem. I am just checking whether we want to fix it, or just say "unsupported on Windows"

    Paul Dicker at 2016-02-12 20:12:49

  3. Are you talking about a situation where a symlink points to a nonexistent file, and then you call create_dir_all on that symlink exactly? And on Unix that will create the directory where the symlink is pointing at, but on Windows it fails?

    (sorry just clarifying)

    Alex Crichton at 2016-02-12 23:02:11

  4. Ah, yes. That is exactly the problem.

    Now it always fails, but is easy fixable on Unix

    Paul Dicker at 2016-02-13 06:11:56

  5. Maybe I should just finish it and make a pull request... :)

    Paul Dicker at 2016-02-13 06:15:42

  6. Seems reasonable! I would be prepared to consider a PR that handles dangling symlinks in create_dir_all as best possible on Unix and Windows.

    David Tolnay at 2017-11-18 06:52:59