feat(linter): add support for nested config files

[camchenry]

• part of linter: nested file configuration #7408

This is the initial implementation for nested config support, without the extends keyword factoring into it yet. So we only load a single config file for any given file still, but it will always be the one closest to the file being linted.

Performance is somewhat of a big TODO here, obviously there is some inherent cost to adding support for nested configurations since we need to check if any directory contains an .oxlintrc.json file, load it if it exists, and then for every file below that directory check if we should use it. I've left a few TODOs of different ways we might be able to cache more data at the cost of memory but at the potential gain of some CPU time. Optimizing iteration over files/directories is one way in which we might be able to make this faster, so we never need to visit the same directory twice.

For now, this will remain an experimental feature that should not affect the current functionality until we enable it for everyone. PRs stacked on top of this will focus on ensuring that other features work well with this, such as --config, -D <rule>, and then we'll add new features like extends.

The configuration loading all happens upfront currently, while the config resolution happens at lint-time. There is a new nested_configs field on the Linter that passes all of the configs that are available, so it can pick which one to use later. Eventually we'll refactor this to not have two duplicate config fields, but I'm leaving it as-is to ensure we're not messing up the existing functionality.

[camchenry]

• test(linter): remove test dependency on oxlint #9513
• fix(linter): do not output number of rules with nested configs #9476
• feat(linter): allow adding more overrides via extends configs #9475
• feat(linter): allowing plugins to be extended with extends #9473
• fix(linter): support nested extending #9472
• feat(linter): inherit rules via the extended config files #9308
• feat(linter): add support for extends property in oxlintrc #9217
• test(linter): ensure CLI filters take precedence over nested configs #9156
• fix(linter): do not use nested configs with --config option #9155
• feat(linter): add support for nested config files #9153 👈 (View in Graphite)
• feat(oxlint): add --experimental-nested-config option #9152
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• 0-merge - adds this PR to the back of the merge queue
• hotfix - for urgent hot fixes, skip the queue and merge this PR next

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[codspeed-hq]

CodSpeed Performance Report

Merging #9153 will not alter performance

_{Comparing 02-15-feat_linter_add_support_for_nested_config_files (9bc3017) with main (4a5a7cf)}

Summary

✅ 33 untouched benchmarks

[Sysix]

Thanks for the great work ❤️ I added some comments and I have a question:

ESLint then searches up the directory structure, merging any .eslintrc files it finds along the way until reaching either an .eslintrc file with root: true or the root directory.

https://eslint.org/docs/latest/use/configure/configuration-files-deprecated#cascading-and-hierarchy

Is this part missing? I could not find the code where you merge the different configurations like overrides.

[graphite-app]

Merge activity

• Feb 21, 11:01 PM EST: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..
• Feb 22, 4:01 AM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Feb 22, 4:02 AM UTC: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..
• Feb 22, 4:02 AM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Feb 22, 9:08 PM UTC: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..
• Feb 22, 9:08 PM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Feb 22, 9:09 PM UTC: Graphite disabled "merge when ready" on this PR due to: a merge conflict with the target branch; resolve the conflict and try again..
• Feb 22, 9:09 PM UTC: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.
• Feb 22, 10:43 PM UTC: A user added this pull request to the Graphite merge queue.
• Feb 22, 10:48 PM UTC: A user merged this pull request with the Graphite merge queue.

[camchenry]

@Boshen I'll address the merge conflicts and then merge this

EDIT: Actually, I'll see about addressing the comment above on stopping iteration when looping over directories and maybe do some benchmarking tomorrow first.

[camchenry]

I'm going to use this repository as a candidate for benchmarking: https://github.com/microsoft/fluentui, as it has a lot of nested .eslintrc files which should make it a good example. Stats:

• 14,836 lintable files using default config
• 2,805 directories

Benchmark

Setup

Using the microsoft/fluentui repository above at b5fa2f0ae70bc472c0603a3927ac68e65b702525.

For simulating something similar to the existing ESLint setup, I ran the following command to replace every .eslintrc-like file with an .oxlintrc.json file, to simulate what a similar setup with oxlint might look like:

find . -name ".eslintrc*" -exec sh -c 'printf "{\n}\n" > "$(dirname "{}")/.oxlintrc.json"' \;

This resulted in 260 .oxlintrc.json files being created (counted by find . | rg oxlintrc | wc -l), spread across many different packages and directories. I think this is reasonably representative of a large enterprise codebase.

Building oxlint with cargo build --release -p oxlint, using code as of 667768c (without early stop / using two separate directory passes). This version with two passes for directories is called oxlint-two-dir-passes. I've also built oxlint, but with the code changed so that we only iterate over each directory only once, and then search for the config in each directory and build as needed. That looks like:

// get all of the unique directories among the paths to use for search for
// oxlint config files in those directories
// e.g. `/some/file.js` and `/some/other/file.js` would both result in `/some`
for path in &paths {
    if let Some(directory) = path.parent() {
        // TODO(perf): benchmark whether or not it is worth it to produce the config files here without
        // iterating over the directories again. it might cause the cache for paths to be disrupted, but
        // it is unclear whether or not that is a problem in practice.
        if let Ok(config) = Self::find_oxlint_config_in_directory(directory) {
            num_configs += 1;
            nested_oxlintrc.insert(directory, config);
        }
    }
}

The executable version for the single-pass config search is ./target/release/oxlint.

Results

• I would expect that ./oxlint-two-dir-passes and ./target/release/oxlint without nested configs enabled should be exactly the same performance, since they don't enable any additional work. That appears to be true to within 0.2ms, which is pretty much just noise. So that's good.
• The two pass version of config building took 734.9ms +- 4.5ms on average, while the single pass version took 758.3ms +- 8.7ms with n=100 runs.
  • 95% confidence interval of the difference of the two means is [-25.332ms, -21.468ms] and the difference of the two means is -23.400ms.
• In short, the two pass version is faster it would seem.
  • However, I didn't benchmark on any small projects, but I'd mainly like to optimize for large monorepo situations first.
  • I did benchmark the vscode repository as well, which doesn't have any nested configurations. In this case, there wasn't a significant difference between having nested configs enabled or disabled.

[Sysix]

In the worst case, it looks like it is only up to 6% slower, and only about 2% slower on average, which is pretty good for a repo of this size!

I guess this is only when no .oxlintrc.json file is found.
The result would be interesting when we set .eslintrc.json as the default for this repo.

Could you please resolve the conflicts? The changes are good for me now 👍

A future idea:
Skip search-iteration of paths-(files) with the same directory. The result should be known already.

[camchenry]

@Sysix Sorry, you caught me in the middle of benchmarking and I left the comment in a half written state for a few hours 😅 I updated the post with a better benchmark.

Overall, I think performance is looking pretty good (so far), in the range of 5-6% slower so far, so not worth optimizing quite yet.