Enhance FileSystemEnumerable to track depth recursion

[iSazonov]

Splitted from #1953

Background and Motivation

In PowerShell Core repo we are trying to migrate to FileSystemEnumerable. Prototype shows great performance improvements.

We need to implement Depth parameter for Get-ChildItem cmdlet (aka dir). The parameter limits a depth of recursion.

This cannot be achieved without creating new classes (based on FileSystemEnumerator and FileSystemEnumerable) and creating non-trivial code as it requires processing a directory tree. All this negates the benefits of this API.

Proposed API

namespace System.IO
{
    public class EnumerationOptions
    {
        /// <summary>
        /// Stop recursion if current depth exceeds the value.
        /// Default is int.MaxValue (no limit).
        /// </summary>
        public int MaxRecursionDepth { get; set; }  // New API
    }
}

Usage Examples

var options = new System.IO.EnumerationOptions
    {
         RecurseSubdirectories = recurse,
         MatchType = MatchType.Win32,
         AttributesToSkip = 0,
         IgnoreInaccessible = false,
         MaxRecursionDepth = 2
    };

var paths = new FileSystemProviderEnumerable<string>(
                directory,
                (ref FileSystemEntry entry) => entry.FileName.ToString()),
                options)
    {
        ShouldIncludePredicate = FileSystemEntryFilter,

        ShouldRecursePredicate = (ref FileSystemEntry entry) => FileSystemRecursionPredicate
    };

Alternative Designs

What is better name - MaxRecursionDepth vs MaxDepthRecursion vs MaxDepthOfRecursion vs MaxDepth ?
(there is JsonSerializerOptions.MaxDepth)

Risks

Minor performance regression due to we have to keep additional uint value in a subdirectory queue.

Implementation details

Current implementation uses Queue to track subdirectories. We could put in the queue current depth +1 value together with subdirectory path. This allow to restore correct depth value at subdirectory dequeue.
And we need add Depth check in recursion condition.

[jozkee]

• Should there be a value to specify "no limit"?
  • if the type were int then -1 could mean that (and throw on any other negative value).
• For the sake of naming discussion, there is a similar API called JsonSerializerOptions.MaxDepth.
• Why default is uint.MaxValue? Perhaps it should be the "no limit" value to avoid breaking behaviors.

[iSazonov]

Should there be a value to specify "no limit"?

• if the type were int then -1 could mean that (and throw on any other negative value).

I believe we have no need to have "no limit" value:

• it is already default behavior
• only if users need to limit recursion depth they will set explicit value.

Also this complicates implementation without need. If we will have "no limit" value we will have to:

• either check the value on every step
• or convert the value to MaxValue.

If we compare with JsonSerializerOptions.MaxDepth the last has a real default = 64 but for recursion default is "no limit" that is MaxValue.

For the sake of naming discussion, there is a similar API called JsonSerializerOptions.MaxDepth.

I added this in OP.

Why default is uint.MaxValue? Perhaps it should be the "no limit" value to avoid breaking behaviors.

The value should be positive by definition - no need to check in implementation.

No breaking changes. On windows long path length is 32,767 characters so max recursion depth is about 16000 and this less then uint.MaxValue.

But I'd agree to follow common pattern if the pattern is int.

Future milestone

My request is for 6.0 milestone since this is useful for PowerSHell and I am ready to pull PR with the API.

[terrajobst]

Video

• We should int
• The default value should be int.MaxValue. It seems 0 should just mean don't recurse at all, rather than no limit.

namespace System.IO
{
    public partial class EnumerationOptions
    {
        public int MaxRecursionDepth { get; set; }
    }
}

[iSazonov]

It seems 0 should just mean don't recurse at all, rather than no limit.

Yes, really it is to return only current directory items.

---

The proposal was updated based on the review.

[iSazonov]

I am ready to pull PR if .Net team approves.
/cc @carlossanlop

[carlossanlop]

I assigned this issue to you, @iSazonov . Thank you for your help! Looking forward to review the PR.