> A Win32 path like C:\dir\file gets translated to an NT path like \??\C:\dir\file, where \??\C: is a symlink in Object Manager to a device object like \Device\HarddiskVolume4
Cool! Can the "NT paths" be used directly on Windows? As far as I know the ordinary Win32/NTFS system doesn't even have symlinks and this feels quite a handicap.
Note that this is behind a "developper flag", i.e. it needs elevation if you have not checked "developper mode" in the W10 parameters. (I guess they don't want the cost of supporting ordinary people building circular dependencies)
It depends on which API the application uses to pass paths to the OS and whether its path handling library does internal parsing that is or is not aware of those prefixes.
For example rust has support[0] for the magical file path prefixes[1] that can be used to escape path length restrictions, reserved identifiers and some other limitations. This is used by default whenever you normalize a part, which means it has fewer restrictions than windows explorer has by default.
These require user-mode round trips for path traversal, so comparable to FUSE, I believe.
So it is relatively easy to add hooks of your own that get called as if they were file reads; NTFS calls these "Reparse Points". Linux would do this with named pipes.
I am just a user of these things, haven't dug into any implementation details recently, but I guess that the stacks to support these features in Linux and Windows each have their cache etc design decisions that have led to current state of filesystem performance under typical use-cases these days.
I'm not a super expert but I think that is the case and you even need them if you want to use some advanced features like paths that are longer than MAX_PATH or these virtual mount kinds of things.
But IMO it's ugly as hell and just begging for compatibility problems. So don't go there, at least not to get super long paths. 260 bytes should be enough for anybody.
How did you manage? I've never hit the limit in like 20 years. Maybe 260 bytes means only 130 2-byte characters (I don't even know) and that I might find a little restrictive maybe. In any case I'm tempted to say, just fix your damn paths (if possible). I guess at 80 characters it starts to get unreadable anyway.
Of course I have only encountered filenames this long a couple times but as for full path like C:\something\something this is very easy.
E.g. "How did you manage? I've never hit the limit in like 20 years." is 63 characters and it still looks like a reasonable title for a book. You may have written or downloaded a book and stored it in a folder named this way in multiple formats (i.e. PDF, EPUB etc) and this can already make something like "C:\Documents and Settings\John Doe\My Documents\My Projects\Books\How did you manage - I've never hit the limit in like 20 years\How did you manage - I've never hit the limit in like 20 years.epub" and this already is 199 characters. Just 60 characters left and 60 characters is as short as "C:\Documents and Settings\John Doe\My Documents", and you might in fact have your book file named a longer way like "John Doe - How did you manage? I've never hit the limit in like 20 years. 2nd edition draft". This is a bit clumsy example but it demonstrates how easily reachable the limit actually is in the real life.
This actually forced me to store my data in c:\a\b\c... (where a, b, and c actually were 1-4-letter acronyms) instead of C:\Documents and Settings\John Doe\My Documents\
Yes, this is how you end up with huge file paths. So don't do that. Obviously. It's not only long in storage terms. Nobody can even read it without getting dizzy.
Name it filepathlimit/filepathlimit.epub or something. Done. You can actually read that. Now if you want the prose, open the damn file. Or use the file explorer which might already show you a more complete title based on the file metadata.
And don't do that "C:\Documents and Settings\John Doe\My Documents\My Projects\Books\". There is no point in storing your things deep in a thousand rabbit holes. It's overzealous hierarchy fetishism. There is no point in creating unreadable paths that wrap around lines like wild. Use D:\Books or something. Or D:\John\Books if you must. Use basic common sense.
I don't do, but I work with other people's stuff and almost every non-geek does that.
> And don't do that "C:\Documents and Settings\John Doe\My Documents\My Projects\Books\". There is no point in storing your things deep in a thousand rabbit holes
That's the standard way Windows users are meant to store their data (although I don't). What is "\home\jdoe\" on Linux is "C:\Documents and Settings\John Doe\My Documents\" on Windows ("C:\Documents and Settings\John Doe\" actually but not from an ordinary user point of view).
> It's overzealous hierarchy fetishism.
I actually find it sad we are still using hierarchies for that when we could be just using tags and semantic attributes instead (semantically a document file doesn't even need a "file name", the actual document title stored as an atribute or as a record within the file is enough and actually better). Apparently it seems ordinary people are more comfortable with the folder metaphor and hierarchies so WinFS was cancelled and 3rd party tagging and semantic desktop systems are little know.
It is actually surprisingly common to hit the max path length with build systems. Why?
You're building a project. Your source code is under C:\Users\twenty-ish character username\Documents\Projects\. The project name is a released build which got extracted to something network-configurator-1.4.5, which builds into a build directory underneath that. That's up to 80 characters right there, only 180 remaining.
If you have any process that starts doing something like "place the artifacts needed to build <result>.foo in result/" (say, cmake), and throw in some hierarchy nesting, you can chew up 180 characters remarkably quickly. Something like node.js produces this monstrosity of a path name:
Names that seem reasonable within the context of their folder may turn out to be highly redundant in the context of the entire pathname--but many people would rather prefer to have readable names at each level of the folder hierarchy, which makes it hard to make full pathnames reasonable. Sometimes, these names are required by means of a language implementation or other build system that is uncontrollable by you.
The unreasonable thing here is not any of the components' names, and the problem fundamentally can not be fixed by shortening them. The problem is the insane uncontrolled nesting.
Hardly. "insane uncontrolled nesting" seems quite a natural way to manage complexity an a automated way. I can see no other principle but the Miller's law (the 7±2 "magic number") to label some level of nesting as "insane" and it doesn't apply for automated processing. What are, if any, good reasons to limit a path length so severely other than human readability? If long paths puts non-negligible penalty on performance I'd say the file system is defective by design.
Good reasons not to have many modules: Compile time, Website loads times and sizes, Project maintainability.
Nested dependencies have negative effects on all of those. They encourage uncontrolled addition of modules, and even addition of modules in multiple versions. They lead to wrong "isolationist" thinking.
In other words, they do not manage complexity but produce unneeded complexity.
Not having many modules means having big modules. The bigger a module is (anything bigger than half a screen) the lower my productivity is. Every module should fit into your mind easily.
Yes they can, though one problems is that because NT paths are less restrictive than Win32 paths, you can create files / directories that you then have trouble doing things with using standard tools
Notably the Windows shell does not support longlong file names, like, at all. The Windows shell and Win32 APIs are also easily confused by files using the NTFS POSIX namespace, which may have \ in their name.
Cool! Can the "NT paths" be used directly on Windows? As far as I know the ordinary Win32/NTFS system doesn't even have symlinks and this feels quite a handicap.