Type error not caught for tuple of incorrect type

[jli]

Environment data

• Language Server version: 2021.3.2
• OS and version: darwin x64
• Python version (and distribution if applicable, e.g. Anaconda): 3.9.2
• python.analysis.indexing: undefined
• python.analysis.typeCheckingMode: basic

Expected behaviour

Expected type error would be flagged by Pylance.

Actual behaviour

Pylance doesn't raise any errors.

Logs

Python Language Server Log

[Info  - 8:03:11 PM] Pylance language server 2021.3.2 (pyright 85309906) starting
[Info  - 8:03:11 PM] Server root directory: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist
[Info  - 8:03:11 PM] No configuration file found.
[Info  - 8:03:11 PM] Setting pythonPath for service "pylance_errors": "/Users/johnli/bin/python"
Search paths found for configured python interpreter:
  /usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9
  /usr/local/Cellar/python@3.9/3.9.2_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/lib-dynload
  /usr/local/lib/python3.9/site-packages
[Error - 8:03:11 PM] stubPath /Users/johnli/src/pylance_errors/typings is not a valid directory.
[Info  - 8:03:12 PM] Assuming Python version 3.9
[Info  - 8:03:12 PM] Assuming Python platform Darwin
[Info  - 8:03:12 PM] Searching for source files
[Info  - 8:03:12 PM] Found 2 source files
[Info  - 8:03:12 PM] Background analysis(1) root directory: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist
[Info  - 8:03:12 PM] Background analysis(1) started
Background analysis message: setConfigOptions
Background analysis message: setTrackedFiles
Background analysis message: markAllFilesDirty
Background analysis message: ensurePartialStubPackages
Background analysis message: setFileOpened
Background analysis message: analyze
[BG(1)] analyzing: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py ...
[BG(1)]   parsing: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (17ms)
[BG(1)]   parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 2ms] (78ms)
[BG(1)]   binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/builtins.pyi (31ms)
[BG(1)]   binding: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (0ms)
[BG(1)]   checking: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py ...
[BG(1)]     parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi [fs read 0ms] (9ms)
[BG(1)]     binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/_typeshed/__init__.pyi (1ms)
[BG(1)]     parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/typing.pyi [fs read 0ms] (22ms)
[BG(1)]     binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/typing.pyi (9ms)
[BG(1)]     parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stubs/typing-extensions/typing_extensions.pyi [fs read 1ms] (4ms)
[BG(1)]     binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stubs/typing-extensions/typing_extensions.pyi (1ms)
[BG(1)]     parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/abc.pyi [fs read 0ms] (0ms)
[BG(1)]     binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/abc.pyi (0ms)
[BG(1)]   checking: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (83ms)
[BG(1)] analyzing: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (211ms)
Background analysis message: resumeAnalysis
[FG] parsing: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (21ms)
[FG] parsing: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/builtins.pyi [fs read 2ms] (85ms)
[FG] binding: /Users/johnli/.vscode/extensions/ms-python.vscode-pylance-2021.3.2/dist/typeshed-fallback/stdlib/builtins.pyi (29ms)
[FG] binding: /Users/johnli/src/pylance_errors/pylance_false_negative_tuple.py (0ms)

Code Snippet / Additional information

lookup_ints is annotated to return a tuple of strs, but the code returns a tuple of bools. Pylance doesn't raise any errors.

# pylance doesn't raise any errors, even though `bools` contains bool and not str.
def lookup_ints(ints: list[int], id_to_bool: dict[int, bool]) -> tuple[str, ...]:
    # (variable) bools: tuple[bool]
    bools = tuple(id_to_bool[i] for i in ints)
    return bools

id_to_bool = {1: True, 2: False}
result = lookup_ints([1, 2], id_to_bool)
print(type(result), result)   # <class 'tuple'> (True, False)

mypy correctly finds the error:

$ mypy pylance_false_negative_tuple.py
pylance_false_negative_tuple.py:5: error: Incompatible return value type (got "Tuple[bool, ...]", expected "Tuple[str, ...]")
Found 1 error in 1 file (checked 1 source file)

[erictraut]

Thanks for the bug report. This was due to a regression that crept into the code a few months ago. I've fixed the bug and updated our unit tests to ensure that such a regression won't go uncaught in the future.

This will be fixed in the next release of pylance.

[jakebailey]

This issue has been fixed in version 2021.3.4, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/main/CHANGELOG.md#202134-31-march-2021