-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix some incorrect SpecialFolder entries for Unix #68610
Conversation
Tagging subscribers to this area: @dotnet/area-system-runtime Issue DetailsThis PR:
Fixes #63214
|
5d1a693
to
d7592f0
Compare
Hm, a bit confused on that failed Windows build. Can't see anything I did that broke it, nor see from the logs on where it error'd on my changes. |
It's not you that broke these tests. @MattGal is this from helix? running %HELIX_CORRELATION_PAYLOAD%\scripts\ce25ee46de8f44688fa71d0af86d03fb\execute.cmd in C:\h\w\BEED09AC\w\A71708E3\e max 900 seconds Output: Exit Code:-900 |
Yes we hotfix-rolled-back a change that was causing this last week (#67728) . If you're still seeing it today we should investigate but checking out similar runs I think we're OK. We may need some variant of the fix back though, but based on the date that's all this is. |
Since os-mac-os-x got labeled for this, someone might wanna label this with os-linux as well. |
To enumerate breaking changes, on Mac, AppData goes from @marklio thoughts about the impact of the break here? As always, it's a major update, so if we think it's the right change, it may well be acceptable to correct all these. @dotnet/area-system-runtime thoughts? It's quite remarkable how much discussion this has involved, and this is only some of the proposed changes. #63214 is just one of the long discussions. My inclination is to take this and look for feedback on Preview 5+. |
The code changes look fine to me though. |
rerunning validation |
Added When you commit this breaking change:
Tagging @dotnet/compat for awareness of the breaking change. |
The mobile platforms are probably the most impacted here. cc @marek-safar |
case SpecialFolder.Desktop: | ||
case SpecialFolder.DesktopDirectory: | ||
return ReadXdgDirectory(home, "XDG_DESKTOP_DIR", "Desktop"); | ||
case SpecialFolder.Templates: | ||
return ReadXdgDirectory(home, "XDG_TEMPLATES_DIR", "Templates"); | ||
case SpecialFolder.MyVideos: | ||
return ReadXdgDirectory(home, "XDG_VIDEOS_DIR", "Videos"); | ||
|
||
#if TARGET_OSX |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this ifdef be enabled for all Apple platforms?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I sadly only have one Apple device which is running MacOS.
Don't the mobile platforms get the folder from a different place though? I thought they'd get it from here or is this mono specific?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does not apply for iOS. Is that all other Apple platforms?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From quick skimming the RIDs, there's TVOS and MacCatalyst as well (and simulators for them). MacCatalyst is inheriting from iOS, tvOS is only inheriting from unix. I cannot however find much documentation regarding tvOS' file system right now. Does seem relatively locked down though, with apps not meant to write to local storage and mostly to the cloud instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does not apply for iOS
Ah ok, I have missed that this file is ifdefed out.
Is there a good reason for the default folder handling to differ between macOS and other Apple OSes? Should macOS be switched to just use https://github.com/dotnet/runtime/blob/main/src/mono/System.Private.CoreLib/src/System/Environment.iOS.cs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Am not that familiar with the codebase, but my two cents are, that it probably would be better to put macOS into there as well, rename the file to Environment.Apple
, and put some macOS specific checks in there (for i.e. ApplicationData
, Personal
etc.).
On another note, why does Environment.iOS
hardcode the subdirectories, instead of invoking the respective NSLibrary paths?
For SpecialFolder.Desktop
for example, NSDocumentDirectory/Desktop
is used instead of NSDesktopDirectory
which according to the Apple docs at least work on all Apple OS. This is probably a thing for a separate PR, but was curious nonetheless.
(Also, updated link for Environment.iOS)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that if you're going to do breaking changes, do it right by using NSSearchPathDirectory values as much as possible, like it's done for iOS here:
runtime/src/libraries/System.Private.CoreLib/src/System/Environment.iOS.cs
Lines 84 to 85 in 16cb35f
case SpecialFolder.InternetCache: | |
return Interop.Sys.SearchPath(NSSearchPathDirectory.NSCachesDirectory); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As far as I can see, that'd require changing the method's return value from Nevermind, forgot that string
to string?
(as the Interop.Sys.SearchPath
method can return null), which would be yet another API break. Is that a change worth having?GetFolderPathCoreWithoutValidation
is an internal method.
case SpecialFolder.ApplicationData: | ||
return Path.Combine(home, "Library", "Application Support"); | ||
case SpecialFolder.LocalApplicationData: | ||
return Path.Combine(home, "Library"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Apps store their settings in ApplicationData
and LocalApplicationData
. Changing the locations will cause apps to no longer find their settings.
With no access to the previous location, implementing settings migration is also difficult.
Besides it being more correct, is there a tangible improvement?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With no access to the previous location, implementing settings migration is also difficult.
The previous location was just /home/.config
/ /home/.local/share
so that can be used for migration. Would need to be made clear for a release.
is there a tangible improvement?
It follows Apple's Guidelines, which from what I know are necessary to fulfill in order to publish anything on their App Store.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It follows Apple's Guidelines, which from what I know are necessary to fulfill in order to publish anything on their App Store.
Without this change, .NET apps aren't accepted in the Apple App Store?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not as far as I know, but am also not a macOS developer so never dealt with that in any way.
Based on :
Check whether your app follows guidance in other documentation
I'd assume so at least.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marek-safar your team is obviously familiar with the app store -- thoughts about impact here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
backwards compatibility
Having a good migration story is important, but we can't keep pretending that accreting technical debt instead of fixing things is fine. That's one of the things that brought computing into disrepute in the wider public.
Perhaps the behavior could be predicated on the .NET version targeted, such that only code compiled for the next version gets the correct folders?
It will break things, and the benefit is not clear to me (besides correctness).
Writing things to $HOME
(e. g. ~/.dotnet
) also breaks things, just in different ways. You cannot expect $HOME
to be writable. (Example: I make $HOME
read-only on every machine under my control.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For any breaking change you need to weigh the pros and cons. Calling it technical debt doesn't change that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, and?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changing the SpecialFolder.*ApplicationData folders is making a similar change for .NET apps that run on macOS. It will break things, and the benefit is not clear to me (besides correctness).
Users are equally likely to be broken by unexpected behavior and so it is a question of which is better: Is it better for users with expectations for a particular platform to be broken or for existing users to be broken?
In this case, it would be a behavioral change which already has no guarantee of consistency because it maps to an underlying OS concept and the underlying OS behavior can vary between supported platforms, architectures, or even individual machines. Users who only tested on Windows are likely already "broken" in some fashion on Linux or MacOS and vice-versa. Users who didn't test against scenarios where the paths have been customized for a given OS or that have been symlinked or otherwise redirected may be equally broken.
Because of that, it should be viewed as almost nothing more than a bug fix to match platform specific expectations and to correctly respect platform specific configuration knobs.
If there is a concern around migration of data (a concern which already exists if users decide to change the path themselves, which many power users do), then it would be possible to introduce some "legacy" switch that allows access to the previous behavior (such as a bool
parameter or explicit new SpecialFolder.LegacyXYZ
)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tannergooding Agreed. If special folders are finally updated, let's make sure they are actually correct. Those who incorrectly relied on buggy behavior should make a suggestion on how to address their problems, not the people fixing things.
} | ||
return data; | ||
case SpecialFolder.MyDocuments: // same value as Personal | ||
return ReadXdgDirectory(home, "XDG_DOCUMENTS_DIR", "Documents"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There will be code that uses MyDocuments
for HOME
.
That code will break, and need to be updated to using UserProfile
instead.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With "that code" are you referring to code inside of .NET or user made programs?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@marklio has tricks for investigating the impact of such a change. It would be good to wait for his comment.
`a17e73466ca639388a3d89f4d6be0ab9703802fc` made OSX call the native NSearchPath functions, however a stub was initially called. This fixes it to call the native functions.
Thanks jkotas for the hint, I think I figured out the place where to do that. |
The result from `GetFolderPathCoreWithoutValidation` should not be null for further continuation of the program, which could happen before.
Mono and .NET, before .NET 8, would resolve the System.Environment.SpecialFolder.Personal enum value to the $HOME directory. In .NET 8 this is changing. The Personal folder will now refer to $XDG_DOCUMENTS_DIR if set or $HOME/Documents otherwise. Replace all uses of Environment.SpecialFolder.Personal with Environment.SpecialFolder.UserProfile so the code behaves as it did before. UserProfile still maps to $HOME dotnet/runtime#68610
#16017) Context: dotnet/runtime#68610 Context: dotnet/android-tools@0be567a9 In Mono and .NET prior to .NET 8, the [`System.Environment.SpecialFolder`][0]`.Personal` enum value would refer to `$HOME` on Unix platforms. This will be changing in .NET 8, such that `Environment.SpecialFolder.Personal` will instead refer to `$XDG_DOCUMENTS_DIR` (if set) or `$HOME/Documents`. This is for "semantic compatibility" with .NET on Windows. Replace usage of `Environment.SpecialFolder.Personal` with `Environment.SpecialFolder.UserProfile`, so that our code continues to work as expected under .NET 8. [0]: https://docs.microsoft.com/en-us/dotnet/api/system.environment.specialfolder?view=net-6.0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM. Thanks @Miepee for pushing this forward.
I'll write up the breaking change doc and post it here after I merge.
Great work all! Once this is merged I'll begin implementing my proposed additions to the |
Breaking change document is here: dotnet/docs#31423 |
@eerhardt I think the line
needs to be clarified a bit, because it makes In my documentation I use |
It is both actually. But yeah, would probably be a good idea to clarify this behaviour for user XDG directories. EDIT: realized we're on the PR and not the documentation issue, would probably be better to move the doc discussions to there. |
Well, then that's absolutely wrong and should be corrected. |
This needs corrected on Android as
@Miepee some of our configurations run post PR because we have limited capacity. It is possible to run them on PR's, but someone on the dotnet team would have to trigger them on your behalf. Not much you can do. Thanks for your contribution! |
Cursory glance, this would need to be modified to conform with the breaking change: runtime/src/libraries/System.Private.CoreLib/src/System/Environment.Android.cs Lines 57 to 59 in b2af65a
|
I ain't that familiar with Android's file structure, should I just change |
Yeah, I think that's fine. |
dotnet#68610 moved `Personal`/`MyDocuments` on Unix systems from HOME to the documents folder. However, it didn't do so for Android systems, which was only noticed after tests later failed. This fixes it.
Opened #76250 to fix that. |
#68610 moved `Personal`/`MyDocuments` on Unix systems from HOME to the documents folder. However, it didn't do so for Android systems, which was only noticed after tests later failed. This fixes it.
Context: dotnet/runtime#68610 In .NET 8, a test using `SpecialFolder.Personal` failed with: [FAIL] Create table attempt failed! SQLite.SQLiteException: Could not open database file: /data/user/0/com.xamarin.customlinkdescriptionpreserve/files/Documents/TaskDB.db3 (CannotOpen) at SQLite.SQLiteConnection..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionWithLock..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.Entry..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.GetConnectionAndTransactionLock(SQLiteConnectionString , Object& ) at SQLite.SQLiteConnectionPool.GetConnection(SQLiteConnectionString ) at SQLite.SQLiteAsyncConnection.GetConnection() at SQLite.SQLiteAsyncConnection.<>c__DisplayClass33_0`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].<WriteAsync>b__0() at System.Threading.Tasks.Task`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b__273_0(Object ) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& , Thread ) --- End of stack trace from previous location --- at LinkTestLib.Bug35195.AttemptCreateTable() All regression tests completed. In .NET 8+, we should use `SpecialFolder.UserProfile` instead.
Context: dotnet/runtime#68610 In .NET 8, a test using `SpecialFolder.Personal` failed with: [FAIL] Create table attempt failed! SQLite.SQLiteException: Could not open database file: /data/user/0/com.xamarin.customlinkdescriptionpreserve/files/Documents/TaskDB.db3 (CannotOpen) at SQLite.SQLiteConnection..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionWithLock..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.Entry..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.GetConnectionAndTransactionLock(SQLiteConnectionString , Object& ) at SQLite.SQLiteConnectionPool.GetConnection(SQLiteConnectionString ) at SQLite.SQLiteAsyncConnection.GetConnection() at SQLite.SQLiteAsyncConnection.<>c__DisplayClass33_0`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].<WriteAsync>b__0() at System.Threading.Tasks.Task`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b__273_0(Object ) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& , Thread ) --- End of stack trace from previous location --- at LinkTestLib.Bug35195.AttemptCreateTable() All regression tests completed. In .NET 8+, we should use `SpecialFolder.UserProfile` instead.
…#7481) Context: dotnet/runtime#68610 Context: dotnet/android-tools@0be567a In .NET 8, a test using `SpecialFolder.Personal` failed with: [FAIL] Create table attempt failed! SQLite.SQLiteException: Could not open database file: /data/user/0/com.xamarin.customlinkdescriptionpreserve/files/Documents/TaskDB.db3 (CannotOpen) at SQLite.SQLiteConnection..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionWithLock..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.Entry..ctor(SQLiteConnectionString ) at SQLite.SQLiteConnectionPool.GetConnectionAndTransactionLock(SQLiteConnectionString , Object& ) at SQLite.SQLiteConnectionPool.GetConnection(SQLiteConnectionString ) at SQLite.SQLiteAsyncConnection.GetConnection() at SQLite.SQLiteAsyncConnection.<>c__DisplayClass33_0`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].<WriteAsync>b__0() at System.Threading.Tasks.Task`1[[SQLite.CreateTableResult, SQLite-net, Version=1.7.335.0, Culture=neutral, PublicKeyToken=null]].InnerInvoke() at System.Threading.Tasks.Task.<>c.<.cctor>b__273_0(Object ) at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) --- End of stack trace from previous location --- at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread , ExecutionContext , ContextCallback , Object ) at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& , Thread ) --- End of stack trace from previous location --- at LinkTestLib.Bug35195.AttemptCreateTable() All regression tests completed. In .NET 8+, we should use `SpecialFolder.UserProfile` instead.
This PR:
fixes an incorrect Documents/Personal entry for Linux+Mac (from
HOME
toXDG_DOCUMENTS_DIR
/HOME/Documents
for Linux and Mac respectively)fixes an incorrect Videos entry for Mac (from
HOME/Videos
toHOME/Movies
)fixes an incorrect (Local) ApplicationData entry for Mac (from
XDG_CONFIG_HOME
andXDG_DATA_HOME
toHOME/Library/Application Support
for both ApplicationData and LocalApplicationData)Changes the Unix tests to check that
SpecialFolder.Personal
andSpecialFolder.MyDocuments
are the Documents directory instead ofHOME
.Fixes #63214,
Fixes #68323,
Fixes #40353