PickFolders option for OpenFileDialog

[miloush]

For discussion. Fixes #438. Further discussion in #4039.

Description

Allows developers to ask the user to select a folder, a feature known as FolderBrowserDialog in WinForms.

This PR is the least invasive way of introducing the feature. It closely reflects the underlying API design, i.e. the ability to specify FOS_PICKFOLDERS on the IOpenFileDialog. This is a backwards compatible solution - no public API has been moved or removed and no existing behavior has changed.

Tracking the flags

There is, however, an unfortunate implementation detail - the original authors of the code decided to use a single variable to store legacy OFN_ options, Vista FOS_ options and their own custom options. These options diverged and conflict with each other:

Value	OFN_	FOS_	OPTION_
0x0000_0001	READONLY *
0x0000_0002	OVERWRITEPROMPT	OVERWRITEPROMPT *
0x0000_0004	HIDEREADONLY *	STRICTFILETYPES
0x0000_0008	NOCHANGEDIR *	NOCHANGEDIR *
0x0000_0010	SHOWHELP
0x0000_0020	ENABLEHOOK	PICKFOLDERS *
0x0000_0040	ENABLETEMPLATE	FORCEFILESYSTEM
0x0000_0080	ENABLETEMPLATEHANDLE	ALLNONSTORAGEITEMS
0x0000_0100	NOVALIDATE *	NOVALIDATE *
0x0000_0200	ALLOWMULTISELECT *	ALLOWMULTISELECT *
0x0000_0400	EXTENSIONDIFFERENT
0x0000_0800	PATHMUSTEXIST *	PATHMUSTEXIST *
0x0000_1000	FILEMUSTEXIST	FILEMUSTEXIST *
0x0000_2000	CREATEPROMPT	CREATEPROMPT *
0x0000_4000	SHAREAWARE	SHAREAWARE
0x0000_8000	NOREADONLYRETURN	NOREADONLYRETURN
0x0001_0000	NOTESTFILECREATE	NOTESTFILECREATE
0x0002_0000	NONETWORKBUTTON	HIDEMRUPLACES
0x0004_0000	NOLONGNAMES	HIDEPINNEDPLACES
0x0008_0000	EXPLORER
0x0010_0000	NODEREFERENCELINKS *	NODEREFERENCELINKS *
0x0020_0000	LONGNAMES	OKBUTTONNEEDSINTERACTION
0x0040_0000	ENABLEINCLUDENOTIFY
0x0080_0000	ENABLESIZING
0x0100_0000	USESHELLITEM
0x0200_0000	DONTADDTORECENT	DONTADDTORECENT
0x0400_0000
0x0800_0000
0x1000_0000	FORCESHOWHIDDEN	FORCESHOWHIDDEN
0x2000_0000		DEFAULTNOMINIMODE
0x4000_0000		FORCEPREVIEWPANEON
0x8000_0000		SUPPORTSTREAMABLEITEMS	ADDEXTENSION

* allowed to set in code as flags (i.e. preserved by masking)

Most notably FOS_PICKFOLDERS conflicts with OFN_ENABLEHOOK. The existing code sort of supports only the flags having the same value in both API and passes the combination directly to the APIs. Few conflicting ones are enforced at call time and developers cannot disable them:

For legacy dialogs, that is:

• OFN_EXPLORER
• OFN_ENABLEHOOK
• OFN_ENABLESIZING

For Vista dialogs, that is:

• FOS_DEFAULTNOMINIMODE
• FOS_FORCEFILESYSTEM

Using the existing infrastructure would mean either

1. using OFN_EXPLORER bit to also mean FOS_PICKFOLDERS, or;
2. adding a new custom OPTION_PICKFOLDERS, preferably as one of the unoccupied bits.

The former is messy (either requires casting from FOS or declaring fake OFN constant) and creates confusing code. The latter is in danger that new conflicting flags will be introduced, like it already happened with OPTION_ADDEXTENSION.

I therefore decided for a potentially controversial solution - I split the backing field and track legacy and Vista options separately. The custom option stays in the legacy field, as no new flags are expected in the legacy API. I am willing to implement options 1) or 2) instead if that was preferred.

New visible members

In OpenFileDialog:

• public bool PickFolders { get; set; }

In FileDialog:

• protected int VistaOptions { get; }

CheckFileExists behavior

The OpenFileDialog turns on CheckFileExists by default. The property works as advertised, meaning users cannot confirm anything if only folders are available to pick. For convenience, setting the PickFolders option has a side effect of unsetting the CheckFileExists option.

Existing properties on folder dialog

(excerpt from discussion in #4039)

These are the current public properties on the FileDialog.

Applicable to folder selection:

• CustomPlaces
• DereferenceLinks - whether links to folders are followed or not allowed
• FileName - e.g. C:\Users\Thomas
• FileNames - always has 1 item at the moment
• InitialDirectory
• SafeFileName - e.g. Thomas
• SafeFileNames - always has 1 item at the moment
• Tag
• Title

Doing something, but wouldn't be necessary for folders:

• AddExtension - internal WPF flag to ensure the file/folder name ends with an extension
• CheckFileExists - makes it unable to select a folder, see above
• DefaultExt - the extension to add by default
• ValidateNames - this one is a red flag, the WPF documentation says "indicating whether the dialog box accepts only valid Win32 file names." which is how the OFN_NOVALIDATE flag works in the legacy file dialogs. However, for the Vista dialogs, the FOS_NOVALIDATE flag means "check for situations that would prevent an application from opening the selected file, such as sharing violations or access denied errors."). In addition to the dialog behavior, WPF uses the flag for adding extensions and prompting for overwriting existing files.

Note that DefaultExt with AddExtensions are already available for open file dialog which doesn't make much sense, but they work the same when folders are selected.

Valid but currently without any effect:

• CheckPathExists - only existing folders are available for selection currently
• RestoreDirectory - applicable to legacy dialogs only, not used in Vista dialogs

Invalid (the native call returns an error):

• Filter
• FilterIndex

Existing methods on folder dialog

• OpenFile and OpenFiles throw UnauthorisedAccessException on folders. The existing code should already expect this type of exception when opening files.

Legacy behavior

The code currently throws PlatformNotSupported exception on Windows XP. If that is acceptable, exception message needs to be added. If not, support for SHBrowseForFolder API needs to be implemented, which carries non-trivial number of native types (or WinForms version called instead).

Customer Impact

Without this fix, users have to either use WinForms' FolderBrowserDialog, resort to 3rd party libraries or re-implement the whole IFileDialog API. #438 is currently the top voted issue in the repository.

Regression

No.

Testing

Compiled custom PresentationFramework.dll and tested in 6.0.2 x86 app that both folder browsing and file browsing works as expected.

Risk

One public and one protected member are added. Existing behavior is not affected. Existing members (even private) still carry the same data. Minimal performance impact (most properties now change two fields) and minimal memory impact (one integer with corresponding generic type).

User code that processes "used" file dialogs could now be handed an instance that unexpectedly contains folders in the FileName(s) properties. This, however, would have to be explicitly caused using new code. Unexpected file operations on folders (i.e. trying to open them) yield IO exceptions that the existing code should be already handling.

/cc @Symbai @ThomasGoulet73 @fabiant3

[Symbai]

Although this is probably not getting merged within the next 2 years, I'm okay with it. Thanks for your work and taking over the PR. Hopefully you'll have a stronger endurance on receiving repo owner's feedback, it could take a great while...

[batzen]

May i ask why we even care about non Vista OS? Is any OS version pre Vista still supported by .NET 7?
If not, why should all the legacy stuff be kept and complicate all changes?

[miloush]

@batzen I had two reasons:

1. Keeping the changes and risks minimal enough so that the barrier of merging this is lower. If I hear from the WPF team that it is OK to drop XP functionality I will happily remove it, but I can't make that decision myself.

2. You can still run .NET 7 executables in XP compatibility mode (in the executable properties) and that will hit the legacy code path even on the latest Windows. It is unclear to me how "supported" the compatibility mode is meant to be.

[miloush]

Also note that couple of the members are protected so it would be a breaking change to remove them or change their meaning.

[miloush]

OK I addressed @ThomasGoulet73 feedback but I am no longer happy with the PR. It is failing at being transparent & minimal, without taking advantage of more substantial changes. I will do two new ones, one simple without splitting options and one with breaking changes.