Pylance very slow #57

syagev · 2020-07-05T10:53:55Z

Environment data

Language Server version: Pylance language server 2020.6.1
VSCode: 1.46.1
OS and version: Windows 10 (18363)
Python version (& distribution if applicable, e.g. Anaconda): 3.6

requirements.txt

attrs==19.3.0
audioread==2.1.8
azure-datalake-store==0.0.48
azure-mgmt-storage==3.3.0
azure-nspkg==3.0.2
azure-storage==0.36.0
azure-storage-blob==2.1.0
beautifulsoup4==4.7.1
bokeh==1.4.0
-e git+https://github.com/Tinche/cattrs.git@v0.9.0#egg=cattrs
cntk-gpu==2.5.1
-e git+https://github.com/syagev/dask.git@optional_stats_2.6.0#egg=dask
distributed==2.5.1
-e git+https://github.com/syagev/fastparquet.git@optional_stats_0.3.2#egg=fastparquet
ffmpeg-python==0.2.0
filelock==3.0.12
future==0.18.2
librosa==0.6.3
llvmlite==0.31.0
lxml==4.4.0
matplotlib==3.2.0
numba==0.48.0
numpy==1.18.2
opencv-contrib-python==4.1.2.30
opencv-python==3.4.3.18
openpyxl==3.0.3
pandas==0.25.3
praatio==3.7.1
PyAudio==0.2.11 ; os_name == 'nt'
pydot-ng==2.0.0
python-snappy==0.5.4 ; os_name != 'nt'
https://princetonlocal.blob.core.windows.net/redist/python_snappy-0.5.4-cp36-cp36m-win_amd64.whl#egg=python-snappy ; os_name == 'nt'
pythonnet==2.4.0 ; os_name == 'nt'
scipy==1.3.3
seaborn==0.10.0
# setuptools upgrade is required to solve tensorboard compatibility issues
setuptools==47.1.1
sounddevice==0.3.14
SoundFile==0.10.3.post1
tensorboardX==1.9
# removed tensorflow 1.8.0 and Keras-Preprocessing==1.1.0, because it conflicted with tensorboard 2.0.0 (which is required to be at least 2.0.0 by labsr)
tensorboard==2.0.0
termcolor==1.1.0
thrift==0.11.0
https://download.pytorch.org/whl/cu101/torch-1.4.0-cp36-cp36m-win_amd64.whl#egg=torch ; os_name == 'nt'
https://download.pytorch.org/whl/cu101/torch-1.4.0-cp36-cp36m-linux_x86_64.whl#egg=torch ; os_name != 'nt'
https://download.pytorch.org/whl/cu101/torchvision-0.5.0-cp36-cp36m-win_amd64.whl#egg=torchvision ; os_name == 'nt'
https://download.pytorch.org/whl/cu101/torchvision-0.5.0-cp36-cp36m-linux_x86_64.whl#egg=torchvision ; os_name != 'nt'
tqdm==4.42.1
traitlets==4.3.3
typed-ast==1.4.1
typing-extensions==3.7.4.1
urllib3==1.25.8
wcwidth==0.1.8
webencodings==0.5.1
Werkzeug==0.16.1
widgetsnbextension==3.5.1
wrapt==1.11.2
zict==1.0.0
onnx==1.7.0
onnxruntime==1.2.0
e2e_simulation/nnlm_pt/panini-2.1.2.tar.gz
xlsxwriter==1.2.9
fsspec==0.6.2
zipp==2.1.0
graphviz==0.13.2
jupyter==1.0.0
jupyter-kernel-gateway==2.4.1
jupytext==1.3.3
mypy==0.761
notebook==5.7.5
ptvsd==4.3.2
pylint==2.4.4
pytest==5.3.5
PyQt5==5.14.1 ; os_name == 'nt'
PyQt5-sip==12.7.1 ; os_name == 'nt'
pyzmq==18.1.1
rope==0.16.0

Expected behaviour

Pyright claims speed, I expected pylance to finish analysis and highlight issues in my code file within a few seconds. I expected it to at least beat the run time of pylint + mypy on that same code file.

Actual behaviour

mypy + pylint complete analysis + highlighting within < 3 seconds, pylance completes within > 10 seconds.

Logs

Python Output Console:

> ~\.virtualenvs\OfflineSystem.36\Scripts\python.exe c:\Users\styagev\.vscode\extensions\ms-python.python-2020.6.91350\pythonFiles\pyvsc-run-isolated.py pylint --extension-pkg-whitelist=cv2,cv2.aruco,numpy.random,torch,torch.nn,tensorflow.logging --generated-members=numpy.*,torch.*,tensorflow.* --rcfile ../pylintrc --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation\meeting_synth\labsr\training\train_sepnet.py
cwd: d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation
> ~\.virtualenvs\OfflineSystem.36\Scripts\python.exe c:\Users\styagev\.vscode\extensions\ms-python.python-2020.6.91350\pythonFiles\pyvsc-run-isolated.py pylint --extension-pkg-whitelist=cv2,cv2.aruco,numpy.random,torch,torch.nn,tensorflow.logging --generated-members=numpy.*,torch.*,tensorflow.* --rcfile ../pylintrc --msg-template='{line},{column},{category},{symbol}:{msg}' --reports=n --output-format=text d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation\meeting_synth\labsr\training\train_sepnet.py
cwd: d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation
> ~\.virtualenvs\OfflineSystem.36\Scripts\python.exe c:\Users\styagev\.vscode\extensions\ms-python.python-2020.6.91350\pythonFiles\pyvsc-run-isolated.py mypy --ignore-missing-imports --follow-imports=silent --show-column-numbers d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation\meeting_synth\labsr\training\train_sepnet.py
cwd: d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation
> ~\.virtualenvs\OfflineSystem.36\Scripts\python.exe c:\Users\styagev\.vscode\extensions\ms-python.python-2020.6.91350\pythonFiles\pyvsc-run-isolated.py mypy --ignore-missing-imports --follow-imports=silent --show-column-numbers d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation\meeting_synth\labsr\training\train_sepnet.py
cwd: d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation
> ~\.virtualenvs\OfflineSystem.36\Scripts\python.exe c:\Users\styagev\.vscode\extensions\ms-python.python-2020.6.91350\pythonFiles\testing_tools\run_adapter.py discover pytest -- --rootdir d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation -s -o python_files=*_pytest.py
cwd: d:\Projects\PrincetonServices\src\OfflineSystem\e2e_simulation
##########Linting Output - mypy##########
Success: no issues found in 1 source file
##########Linting Output - pylint##########
************* Module e2e_simulation.meeting_synth.labsr.training.train_sepnet
78,4,refactor,no-self-use:Method could be a function
92,4,refactor,too-many-arguments:Too many arguments (15/5)
159,4,refactor,too-many-arguments:Too many arguments (15/5)
222,4,refactor,too-many-arguments:Too many arguments (11/5)
222,4,refactor,too-many-locals:Too many local variables (21/15)
479,0,refactor,too-many-instance-attributes:Too many instance attributes (12/7)
498,4,refactor,too-many-arguments:Too many arguments (9/5)
672,4,refactor,too-many-locals:Too many local variables (17/15)
754,4,refactor,useless-return:Useless return at end of function or method

------------------------------------------------------------------

Your code has been rated at 9.70/10 (previous run: 9.70/10, +0.00)

Code Snippet / Additional information

My code base is pretty large, this might be the issue. See screen recording below - recording starts as I hit Save after introducing a stupid syntax error. First highlights are from mypy+pylint while Pylance highlights come much after. Note the hogging of the code.exe process.

The text was updated successfully, but these errors were encountered:

erictraut · 2020-07-05T15:01:39Z

Do you make use of type stubs for your imported third-party libraries, or are you relying on the type checker to extract type information from the library implementations?

My team also has a large python code base, and Pyright (which is the type checker used in Pylance) is many times faster than mypy. So I'm curious what is different about your code base.

By any chance, is your code base publicly available (e.g. in a public github repo)? If so, I can look into this more. If not, I don't think there's much I will be able to do to diagnose further.

syagev · 2020-07-05T15:13:33Z

How do I check that I'm using type-stubs? Also, I should have mentioned I have two 3rd party libraries (dask and fastparquet) that we've forked and we are using our own forked version (hosted on github) in this project.

The code isn't public, but I'll reach out on a different channel to share it privately with you if you'd like.
Thanks for the response!

erictraut · 2020-07-05T23:18:51Z

I think I've found and fixed the issue that was causing the slow performance you were seeing. There were certain code constructs (namely, a large number of conditional statements nested within loops) that were confounding the code flow engine in Pylance. I've added some additional caching to handle this case.

ManPython · 2020-07-08T13:10:15Z

After move to pylance I see that VSC working hardly. Eg. when I saving changes and trying quick test for QT/Pyside2 apps then must wait for VSC do own job (like with pylance) and after some time push "play" to execute code in other case have this type info everytime. I'm new with pylance but before this in case of work with "hudge" MVC projects not meet this type to often, now with small simple apps give this time after time - only 3-4 try again, again, runing app in debug coretly. I'm minimal with other extensions etc..

jakebailey · 2020-07-09T21:22:08Z

This issue has been fixed in version 2020.7.0, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202070-9-july-2020

erictraut added the waiting for user response Requires more information from user label Jul 5, 2020

erictraut added fixed in next version (main) A fix has been implemented and will appear in an upcoming version and removed waiting for user response Requires more information from user labels Jul 5, 2020

jakebailey closed this as completed Jul 9, 2020

mohammedayub44 mentioned this issue Feb 1, 2021

Python Extension taking very long time to show Autocomplete and IntelliSense commands #902

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pylance very slow #57

Pylance very slow #57

syagev commented Jul 5, 2020

erictraut commented Jul 5, 2020

syagev commented Jul 5, 2020

erictraut commented Jul 5, 2020

ManPython commented Jul 8, 2020

jakebailey commented Jul 9, 2020

Pylance very slow #57

Pylance very slow #57

Comments

syagev commented Jul 5, 2020

Environment data

Expected behaviour

Actual behaviour

Logs

Code Snippet / Additional information

erictraut commented Jul 5, 2020

syagev commented Jul 5, 2020

erictraut commented Jul 5, 2020

ManPython commented Jul 8, 2020

jakebailey commented Jul 9, 2020