pipenv works with --python 3.9 fails with 3.10 (error message like #5109) #5116

g-vidal · 2022-06-02T09:08:14Z

Issue description

When creating vrt. env. with pipenv and python 3.9 it works but fails if python3.10 is used. The mesages seemed to be the same than #5109 but the workaround provided does not work on my debian :

pipenv --python 3.9
Creating a virtualenv for this project...
Pipfile: /home/********/Jupyter/VenvJupyter/Pipfile
Using /usr/bin/python3.9 (3.9.12) to create virtualenv...
⠹ Creating virtual environment...created virtual environment CPython3.9.12.final.0-64 in 523ms
  creator CPython3Posix(dest=/home/*****/.local/share/virtualenvs/VenvJupyter-tRdDfvhR, clear=False, no_vcs_ignore=False, global=False)                                        
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/*****/.local/share/virtualenv)                                  
    added seed packages: pip==22.0.4, setuptools==62.1.0, wheel==0.37.1                                                                                                        
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator                                                                  
                                                                                                                                                                               
✔ Successfully created virtual environment! 
Virtualenv location: /home/********/.local/share/virtualenvs/VenvJupyter-tRdDfvhR
Creating a Pipfile for this project...

Expected result

Expected same behavior with python3.9 and python3.10

Actual result

When creating vrt. env. with pipenv and python 3.10 it fails (Pipfile nor created and bin folder in subfolder local instead of the venv root folder) :

pipenv --python 3.10
Creating a virtualenv for this project...
Pipfile: /home/*******/Jupyter/VenvJupyter/Pipfile
Using /usr/bin/python3 (3.10.4) to create virtualenv...
⠧ Creating virtual environment...created virtual environment CPython3.10.4.final.0-64 in 132ms
  creator CPython3Posix(dest=/home/********/.local/share/virtualenvs/VenvJupyter-tRdDfvhR, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/********/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==62.1.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment! 
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 8, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/cli/options.py", line 56, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 1637, in invoke
    super().invoke(ctx)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/vendor/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pipenv/cli/command.py", line 193, in cli
    ensure_project(
  File "/usr/local/lib/python3.10/dist-packages/pipenv/core.py", line 553, in ensure_project
    ensure_virtualenv(
  File "/usr/local/lib/python3.10/dist-packages/pipenv/core.py", line 486, in ensure_virtualenv
    do_create_virtualenv(                                                                                                                                                      
  File "/usr/local/lib/python3.10/dist-packages/pipenv/core.py", line 1013, in do_create_virtualenv                                                                            
    project._environment = Environment(                                                                                                                                        
  File "/usr/local/lib/python3.10/dist-packages/pipenv/environment.py", line 70, in __init__                                                                                   
    self._base_paths = self.get_paths()                                                                                                                                        
  File "/usr/local/lib/python3.10/dist-packages/pipenv/environment.py", line 394, in get_paths                                                                                 
    c = subprocess_run(command)                                                                                                                                                
  File "/usr/local/lib/python3.10/dist-packages/pipenv/utils/processes.py", line 75, in subprocess_run                                                                         
    return subprocess.run(                                                                                                                                                     
  File "/usr/lib/python3.10/subprocess.py", line 501, in run                                                                                                                   
    with Popen(*popenargs, **kwargs) as process:                                                                                                                               
  File "/usr/lib/python3.10/subprocess.py", line 966, in __init__                                                                                                              
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1842, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/home/********/.local/share/virtualenvs/VenvJupyter-tRdDfvhR/bin/python'

Steps to replicate

cd in empty folder
pipenv --python 3.10 or pipenv --python 3.9

$ pipenv --support

Pipenv version: '2022.5.2'

Pipenv location: '/usr/local/lib/python3.10/dist-packages/pipenv'

Python location: '/usr/bin/python3'

Python installations found:

3.10.4: /usr/bin/python3
3.10.4: /bin/python3
3.9.12: /usr/bin/python3.9
3.9.12: /bin/python3.9

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.10.4',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.17.0-1-amd64',
 'platform_system': 'Linux',
 'platform_version': '#1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18)',
 'python_full_version': '3.10.4',
 'python_version': '3.10',
 'sys_platform': 'linux'}

System environment variables:

SHELL
SESSION_MANAGER
WINDOWID
QT_ACCESSIBILITY
COLORTERM
SSH_AGENT_LAUNCHER
XDG_MENU_PREFIX
GNOME_DESKTOP_SESSION_ID
GTK_IM_MODULE
LANGUAGE
SSH_AUTH_SOCK
XMODIFIERS
DESKTOP_SESSION
XML_CATALOG_FILES
GTK_MODULES
PWD
XDG_SESSION_DESKTOP
LOGNAME
XDG_SESSION_TYPE
SYSTEMD_EXEC_PID
XAUTHORITY
GJS_DEBUG_TOPICS
GDM_LANG
HOME
USERNAME
IM_CONFIG_PHASE
LANG
LS_COLORS
XDG_CURRENT_DESKTOP
WAYLAND_DISPLAY
INVOCATION_ID
MANAGERPID
CLUTTER_IM_MODULE
GJS_DEBUG_OUTPUT
NVM_DIR
GNOME_SETUP_DISPLAY
SGML_CATALOG_FILES
XDG_SESSION_CLASS
TERM
USER
COLORFGBG
DISPLAY
SHLVL
QT_IM_MODULE
CLASSPATH
XDG_RUNTIME_DIR
JOURNAL_STREAM
XDG_DATA_DIRS
PATH
GDMSESSION
DBUS_SESSION_BUS_ADDRESS
GIO_LAUNCHED_DESKTOP_FILE_PID
GIO_LAUNCHED_DESKTOP_FILE
OLDPWD
_
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PIP_PYTHON_PATH
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/********/e-Book/DocbookXSLT/XSLT10/docbook-xsl-snapshot/tools/bin:/home/********/.local/bin:/home/********/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/snap/bin:/usr/lib/ccache
SHELL: /bin/bash
LANG: fr_FR.UTF-8
PWD: /home/********/Jupyter/VenvJupyter

The text was updated successfully, but these errors were encountered:

matteius · 2022-06-02T12:34:24Z

@g-vidal I am unable to reproduce this as it appears to work fine:

matteius@matteius-VirtualBox:~/pipenv-triage/pipenv-5116$ pipenv --python 3.10
Creating a virtualenv for this project...
Pipfile: /home/matteius/pipenv-triage/pipenv-5116/Pipfile
Using /home/matteius/.pyenv/versions/3.10.2/bin/python3 (3.10.2) to create virtualenv...
⠸ Creating virtual environment...created virtual environment CPython3.10.2.final.0-64 in 551ms
  creator CPython3Posix(dest=/home/matteius/.virtualenvs/pipenv-5116-YASxj25R, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/matteius/.local/share/virtualenv)
    added seed packages: pip==22.0.4, setuptools==62.1.0, wheel==0.37.1
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/matteius/.virtualenvs/pipenv-5116-YASxj25R
Creating a Pipfile for this project...

g-vidal · 2022-06-02T14:10:01Z

Thaks for testing, This is weird !
I tried on two computers my laptop and desktop before posting, and got the same behavior. I am working with bullseye (testing) versions of debian. My python3.10 is 3.10.4 while yours is 3.10.2 would it be the point ?
It seems that things are going wrong because the bin directory of the virtualenv is written in the local subfolder while pipenv looks for it in the root folder of the environment. As quoted in #5109 it sems to be linked with 'posix_prefix' but I have not been able to trace out how, the proposed #5109 trick is not working for me

matteius · 2022-06-04T07:30:36Z

@g-vidal Are you able to test with the master branch?

mstarongithub · 2022-06-06T18:38:25Z

I run into a similar problem. Using Python 3.10, I run into the same No such file or directory: /path/to/venv/missing_folder/bin/python while creating a pipenv for both Python 3.9 and 3.10.

Edit: On Pipenv version 2022.5.2

matteius · 2022-06-07T14:19:57Z

Please try again with 2022.6.7 and let us know if its been resolved.

mstarongithub · 2022-06-07T14:33:18Z

You seem to have fixed the bug. Didn't run into any issues while installing 3.9 or 3.10
👍

matteius · 2022-06-07T14:35:54Z

Thanks @MrEvilOnGitHub that is great to hear -- just want to call out that It was brilliantly understood and fixed by @torsava -- I only pushed the release out.

g-vidal · 2022-06-14T07:30:59Z

Hi @matteius The fix worked perfectly with pipenv2022.6.7 python3.10 on one of my machines :

        Manufacturer: Dell Inc.
        Product Name: Precision WorkStation T3500 
Linux Cerf-Volant 5.17.0-1-amd64 #1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18) x86_64 GNU/Linux

But in another one it failed with the same error:

        Manufacturer: Dell Inc.
        Product Name: Precision 5820 Tower
Linux marjalahti 5.17.0-1-amd64 #1 SMP PREEMPT Debian 5.17.3-1 (2022-04-18) x86_64 GNU/Linux

The installed packages differ from one workstation to the other. I started to compare but it is really a pain to tackle with brute force. python3.9 et python3.10 are installed on both machines, there is no package pipenv in the testing distrib I am using. I install pipenv with sudo pip install. If I do the corrections by hand (add a Pipfile and move the bin and lib folders, things seem to work perfectly).

Do you have any suggestion on how to trace out the origin of the discrepancy between the machines

matteius · 2022-06-14T10:57:27Z

No I am not sure based on just that bit of information @g-vidal . Only thing I can say though is its advised to do a --user install of python packages such as pipenv and not a global sudo install. It does mean adding the local path to your PATH usually, like export PATH=$PATH:~/.local/bin

I will reopen this issue in case anyone else can help you.

g-vidal · 2022-06-15T09:17:36Z

Thanks @matteius. I have solved the situation by reinstalling completely python3.10 and removing all python3.9 breadcrumbs. It works pefectly 👍

matteius · 2022-06-15T11:15:16Z

So are you set with this issue @g-vidal?

g-vidal · 2022-06-15T11:59:57Z

yes @matteius evreything is OK, I cannot explain what happened but if anybody has the same issue my solution has been to clean everything related to python in '.local/lib' ' .local/bin' '/usr/local/bin' '/usr/local/lib' ; then rebuild properly last python version and dependencies on python (tricky exercise but efficient!)

molava1 · 2022-06-26T15:32:51Z

I'm still having issues with the FileNotFoundError: [Errno 2] after '✔ Successfully created virtual environment!'..
I've tried to update my pipenv & python :/
Any further suggestions? :(

g-vidal · 2022-06-26T15:55:19Z

@molava1 It took me some time and some risky uninstall operations to succeed on my debian ! I would not suggest to follow the same path... The origin of the problem for me was a big mess between hazardous python 3.9 and 3.10 installations. There were more than one instance of python, python3, pip, pipenv executables and various python local-libs installations. Whenever things have been cleaned up it worked like a charm. You may try to see with which python or which pip or any other method if your python installation is clean...

maximlomans · 2022-07-25T01:22:10Z

same issue -
" The origin of the problem for me was a big mess between hazardous python 3.9 and 3.10 installations. "

though in my case the reason i think was logging out of user and back into root to do full-upgrade on a rolling edition - launching terminal showed me the traceback on pipenv -
unless it is specific to py3.10

Traceback (most recent call last):
File "/usr/local/bin/pipenv", line 8, in
sys.exit(cli())
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 696, in main
with self.make_context(prog_name, args, **extra) as ctx:
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 621, in make_context
self.parse_args(ctx, args)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 1018, in parse_args
rest = Command.parse_args(self, ctx, args)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 875, in parse_args
parser = self.make_parser(ctx)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 821, in make_parser
for param in self.get_params(ctx):
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 774, in get_params
help_option = self.get_help_option(ctx)
File "/usr/lib/python3/dist-packages/pipenv/cli.py", line 26, in get_help_option
from .import core
File "/usr/lib/python3/dist-packages/pipenv/core.py", line 21, in
import requests
File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/init.py", line 65, in
from . import utils
File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/utils.py", line 27, in
from .cookies import RequestsCookieJar, cookiejar_from_dict
File "/usr/lib/python3/dist-packages/pipenv/vendor/requests/cookies.py", line 172, in
class RequestsCookieJar(cookielib.CookieJar, collections.MutableMapping):
AttributeError: module 'collections' has no attribute 'MutableMapping'

ok
i think t is specific to 3.10 - just did a pip upgrade worked for py2 but failed for py3
2
Successfully installed pip-20.3.4
3
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/init.py)

it's 3:30am i'll fix it tmrw- it's probably what got me in this mess to start with

matteius · 2022-07-25T01:29:01Z

@maximlomans I am currently at this moment working on a PR (unrelated to your comment) that would rely on the vendor'd version of pip inside pipenv for installs -- currently pipenv references outside of itself for pip for installs which causes this issue. I believe my PR when its available will resolve this going forward, so I re-opened this issue to tag my work against.

Edit: though re-reading your stack trace I am less convinced that its what I say it is :-)

maximlomans · 2022-07-25T01:43:50Z

thanks for the quick reply - i should have added the fact that i used get-pip.py script -

matteius · 2022-07-25T01:55:50Z

@maximlomans You could check if installing this branch resolves your issue or not: #5190
Then I would know if I should tag this issue with the news fragment or not.
Edit: Actually its probably a bigger change than just this that I should report on, so I'll tag something else. And after re-reading the whole thread I suspect it won't fix your issue, because it seems everyone else had conflicting system installs of python, is my guess.

maximlomans · 2022-07-25T02:12:31Z

I will do tomorrow - it's past 4;15 am where i am - thanks

maximlomans · 2022-07-25T06:25:04Z

i found something on stackoverflow - it is indeed a 3.10 specific issue;

"https://stackoverflow.com/questions/69512672/getting-attributeerror-module-collections-has-no-attribute-mutablemapping-w"
*
"The problem is caused by an old version of pyparsing that has been vendored into pkg_resources, which is now part of setuptools."
*
"update pip using code bellow"
curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10

; )

maximlomans · 2022-07-26T09:10:21Z

It's OFFICIAL

https://peps.python.org/pep-0632/

"The distutils module [1] has for a long time recommended using the setuptools package [2] instead. Setuptools has recently integrated a complete copy of distutils and is no longer dependent on the standard library [3]. Pip has been silently replacing distutils with setuptools when installing packages for a long time already, and the distutils documentation has stated that it is being phased out since 2014 (or earlier). It is time to remove it from the standard library."

g-vidal · 2022-07-26T09:22:30Z

Thanks for the explanation 👍

matteius · 2022-08-05T22:21:59Z

New version of pipenv 2022.8.5 has been released -- can anyone reproduce this issue on the new version?

g-vidal · 2022-08-06T11:02:22Z

upgrade pipenv with pip OK
created new virtualenv with pipenv 2022.8.5 and Using /usr/bin/python3 (3.10.5) Debian 5.18.5-1 OK
`
pipenv install requests
Creating a virtualenv for this project...
Pipfile: /home/vidal/Pipfile
Using /usr/bin/python3 (3.10.5) to create virtualenv...
⠹ Creating virtual environment...created virtual environment CPython3.10.5.final.0-64 in 136ms
creator CPython3Posix(dest=/home/vidal/.local/share/virtualenvs/vidal-yuttCOqX, clear=False, no_vcs_ignore=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/vidal/.local/share/virtualenv)
added seed packages: pip==22.1.2, setuptools==63.1.0, wheel==0.37.1
activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment!
Virtualenv location: /home/vidal/.local/share/virtualenvs/vidal-yuttCOqX
Installing requests...
Adding requests to Pipfile's [packages]...
✔ Installation Succeeded
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success!
Updated Pipfile.lock (a290a1)!
Installing dependencies from Pipfile.lock (a290a1)...
🐍 ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
`
👍

silvesta · 2023-10-24T14:58:26Z

I ran into this problem when I tried to use the command (pipenv shell) inside a django project folder. Error message does not occure when I cd'd out of the project folder.

matteius added the Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. label Jun 7, 2022

matteius closed this as completed Jun 7, 2022

matteius reopened this Jun 14, 2022

g-vidal closed this as completed Jun 15, 2022

matteius reopened this Jul 25, 2022

matteius self-assigned this Jul 25, 2022

godylockz mentioned this issue Jul 27, 2022

[BUG] Python 3.10 Compatibility nettitude/PoshC2#270

Open

matteius closed this as completed Aug 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv works with --python 3.9 fails with 3.10 (error message like #5109) #5116

pipenv works with --python 3.9 fails with 3.10 (error message like #5109) #5116

g-vidal commented Jun 2, 2022

matteius commented Jun 2, 2022

g-vidal commented Jun 2, 2022

matteius commented Jun 4, 2022

mstarongithub commented Jun 6, 2022 •

edited

Loading

matteius commented Jun 7, 2022

mstarongithub commented Jun 7, 2022

matteius commented Jun 7, 2022

g-vidal commented Jun 14, 2022 •

edited

Loading

matteius commented Jun 14, 2022

g-vidal commented Jun 15, 2022

matteius commented Jun 15, 2022

g-vidal commented Jun 15, 2022

molava1 commented Jun 26, 2022

g-vidal commented Jun 26, 2022

maximlomans commented Jul 25, 2022

matteius commented Jul 25, 2022 •

edited

Loading

maximlomans commented Jul 25, 2022

matteius commented Jul 25, 2022 •

edited

Loading

maximlomans commented Jul 25, 2022

maximlomans commented Jul 25, 2022

maximlomans commented Jul 26, 2022

g-vidal commented Jul 26, 2022

matteius commented Aug 5, 2022

g-vidal commented Aug 6, 2022

silvesta commented Oct 24, 2023

pipenv works with --python 3.9 fails with 3.10 (error message like #5109) #5116

pipenv works with --python 3.9 fails with 3.10 (error message like #5109) #5116

Comments

g-vidal commented Jun 2, 2022

Issue description

Expected result

Actual result

Steps to replicate

matteius commented Jun 2, 2022

g-vidal commented Jun 2, 2022

matteius commented Jun 4, 2022

mstarongithub commented Jun 6, 2022 • edited Loading

matteius commented Jun 7, 2022

mstarongithub commented Jun 7, 2022

matteius commented Jun 7, 2022

g-vidal commented Jun 14, 2022 • edited Loading

matteius commented Jun 14, 2022

g-vidal commented Jun 15, 2022

matteius commented Jun 15, 2022

g-vidal commented Jun 15, 2022

molava1 commented Jun 26, 2022

g-vidal commented Jun 26, 2022

maximlomans commented Jul 25, 2022

matteius commented Jul 25, 2022 • edited Loading

maximlomans commented Jul 25, 2022

matteius commented Jul 25, 2022 • edited Loading

maximlomans commented Jul 25, 2022

maximlomans commented Jul 25, 2022

maximlomans commented Jul 26, 2022

g-vidal commented Jul 26, 2022

matteius commented Aug 5, 2022

g-vidal commented Aug 6, 2022

silvesta commented Oct 24, 2023

mstarongithub commented Jun 6, 2022 •

edited

Loading

g-vidal commented Jun 14, 2022 •

edited

Loading

matteius commented Jul 25, 2022 •

edited

Loading

matteius commented Jul 25, 2022 •

edited

Loading