How to use w/o virtualenv

[lorencarvalho]

Howdy! Thanks for the excellent work on this tool!

I have a peculiar environment and I'd like to get pre-commit working for it. We have many individual repositories, and in order to keep them self-contained we have some automation that installs a "clean" (untainted) python and prevents anything from being installed into it's site-packages.

As such, pre-commit fails to execute due to a lack of virtualenv in our "clean" python:

$ git commit --allow-empty
[INFO] Initializing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: Command: ('/export/apps/python/3.7.0/bin/python3', '-mvirtualenv', '/home/lcarvalh/.cache/pre-commit/repoovkxm3_6/py_env-python3.7', '-p', '/export/apps/python/3.7.0/bin/python3')
Return code: 1
Expected return code: 0
Output: (none)
Errors:
    /export/apps/python/3.7.0/bin/python3: No module named virtualenv

Of course, venv is available, as it's part of the standard library as of 3.3, but unfortunately virtualenv is not. If I install it manually into my local site-packages python3 -m pip install virtualenv --user it works fine:

$ PIP_REQUIRE_VIRTUALENV=false python3 -m pip install virtualenv --user
Collecting virtualenv
  Using cached https://files.pythonhosted.org/packages/db/9e/df208b2baad146fe3fbe750eacadd6e49bcf2f2c3c1117b7192a7b28aec4/virtualenv-16.7.2-py2.py3-none-any.whl
Installing collected packages: virtualenv
Successfully installed virtualenv-16.7.2

$ git commit --allow-empty
[INFO] Installing environment for https://github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
Flake8...............................................(no files to check)Skipped
Detect Private Key...................................(no files to check)Skipped
Check docstring is first.............................(no files to check)Skipped
Check builtin type constructor use...................(no files to check)Skipped
Check for added large files..........................(no files to check)Skipped
Forbid new submodules................................(no files to check)Skipped
Aborting commit due to empty commit message.

if I change pre_commit/languages/python.py:172 to: cmd = (sys.executable, '-mvenv', envdir) then things also work as expected.

Is there a way to configure or modify the default behavior to avoid -mvirtualenv completely? Or at least fall back on or prefer -mvenv?

Thanks in advance! And thanks again for all the great work 🙏

[asottile]

you can override language: python with language: python_venv to use -mvenv

though note that if you don't have -mvirtualenv you're outside the supported setup as that's a dependency of this tool:

pre-commit/setup.cfg

Line 35 in 07f6641

virtualenv>=15.2

[asottile]

For example:

-   repo: https://github.com/pre-commit/pre-commit-hooks
    rev: v2.3.0
    hooks:
    -   id: trailing-whitespace
        language: python_venv

[lorencarvalho]

@asottile wow lightning fast response, thanks so much this is perfect and exactly what I was looking for.

Not worried about the dependency, we package all our tools with https://github.com/linkedin/shiv so it's available to pre-commit itself, just not to the vanilla interpreter.

Thanks again, this is great.

[asottile]

ah makes sense 👍 -- I've seen some other hacks with shiv before to make that work, but requires some tricky PYTHONPATH settings

[lorencarvalho]

come to think of it, if I build pre-commit with shiv's -E option (which just puts shiv's sys.path into $PYTHON_PATH, so subprocesses have access to all the same dependencies) it works 100% ootb.
Sweet!

[asottile]

dope! that's what I was remembering! 🎉

[rpatterson]

you can override language: python with language: python_venv to use -mvenv

For more context, I used the above to workaround broken virtualenv layouts.

[asottile]

@rpatterson probably you fixed it in some other way, python_venv is an alias to python and no longer has different behaviour

it was probably some other change like installing python3-distutils or adding the deadsnakes fixed python3.10 on 22.04