Pylance reports a false positive problem with os.fspath for an argument of type str | os.PathLike[str]

[willfrey]

Environment data

• Language Server version: 2021.3.4
• OS and version: macOS 11.2.3
• Python version (& distribution if applicable, e.g. Anaconda): 3.9.2

Expected behaviour

Pylance should not report an issue for calling os.fspath(obj) when obj is inferred to be of type str | os.PathLike[str].

FWIW, mypy doesn't complain about this and the mypy's bundled stub appears to be identical to what Pylance is distributed with.

Actual behaviour

Pylance reports:

No overloads for "fspath" match the provided arguments
  Argument types: (str | _PathLike[str])

Logs

Not sure if these are useful but I can include them if you want.

Code Snippet / Additional information

import os
from typing import Union

# Just a wrapper for demonstration purposes.
def fspath(arg: Union[str, "os.PathLike[str]"]) -> str:
    return os.fspath(arg)  # this should be fine

[erictraut]

This behavior is due to the way pyright (the type checker that underlies pylance) handles overloaded functions. Pyright determines a single overload function by checking each overload in turn to see if the arguments match. This is consistent with the handling of overloads in other languages like TypeScript. Mypy, by contrast, expands all of the union types passed to the function call and evaluates each combination of subtypes independently. It then applies all combinations to the full list of overloaded functions and creates a union of the results. The problem with this approach is the combinatoric explosion that can result. For example, if you have 10 arguments each with 10 union subtypes, you will need to evaluate the function 10^10 times to determine the type! In practice, the numbers tend to be smaller than this, but even small numbers become problematic from a performance perspective. For this reason — and because it would be a huge change to the structure of the pyright type checker — we have decided to stick with our current approach.

PEP 484 does not clarify the intended behavior for overloads, so both pyright and mypy comply with the standard in this regard.

If my memory is correct, this issue has been reported only once or twice previously. It comes up very rarely in practice. There is a simple (although somewhat ugly) workaround.

def fspath(arg: Union[str, "os.PathLike[str]"]) -> str:
    if isinstance(arg, str):
        return os.fspath(arg)
    else:
        return os.fspath(arg)

[willfrey]

Yeah, I tried that out to see if it was a problem with the union type and Pylance was sated. I can also silence it with a type: ignore comment. Easy to work around but I thought I’d mention the issue. Thanks for the quick response!

…

On Apr 1, 2021, at 12:16 PM, Eric Traut ***@***.***> wrote:  This behavior is due to the way pyright (the type checker that underlies pylance) handles overloaded functions. Pyright determines a single overload function by checking each overload in turn to see if the arguments match. This is consistent with the handling of overloads in other languages like TypeScript. Mypy, by contrast, expands all of the union types passed to the function call and evaluates each combination of subtypes independently. It then applies all combinations to the full list of overloaded functions and creates a union of the results. The problem with this approach is the combinatoric explosion that can result. For example, if you have 10 arguments each with 10 union subtypes, you will need to evaluate the function 10^10 times to determine the type! In practice, the numbers tend to be smaller than this, but even small numbers become problematic from a performance perspective. For this reason — and because it would be a huge change to the structure of the pyright type checker — we have decided to stick with our current approach. PEP 484 does not clarify the intended behavior for overloads, so both pyright and mypy comply with the standard in this regard. If my memory is correct, this issue has been reported only once or twice previously. It comes up very rarely in practice. There is a simple (although somewhat ugly) workaround. def fspath(arg: Union[str, "os.PathLike[str]"]) -> str: if isinstance(arg, str): return os.fspath(arg) else: return os.fspath(arg) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[erictraut]

Reopening this issue. We've received a sufficient number of bug reports about the overload matching behavior that I decided to change the behavior to more closely match mypy's. This eliminates the need for workarounds like the one I posted above. This change will be in the next version of pylance.

[willfrey]

Thank you!

…

On Apr 25, 2021, at 11:17 AM, Eric Traut ***@***.***> wrote:  Reopening this issue. We've received a sufficient number of bug reports about the overload matching behavior that I decided to change the behavior to more closely match mypy's. This eliminates the need for workarounds like the one I posted above. This change will be in the next version of pylance. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[jakebailey]

This issue has been fixed in version 2021.4.3, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202143-29-april-2021