editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) #4895

trondhindenes · 2021-12-18T10:17:06Z

Issue description

I cloned this repo https://github.com/piccolo-orm/piccolo_api and am trying to install it "from dir" in another project. The fact that the repo contains a requirements folder folder instead of a single file seems to throw pipenv:
ERROR: Can't verify hashes for these file:// requirements because they point to directories:

Expected result

The package installation works

Actual result

package install fails

Steps to replicate

Clone the abovementioned repo, and then set up a new pipenv project in a different folder.
try and install the cloned repo from path (pipenv install <path to cloned repo> )

Pipenv version: '2021.11.23'

The text was updated successfully, but these errors were encountered:

matteius · 2021-12-19T04:59:18Z

I think your issue report may be a duplicate of: #4869

matteius · 2021-12-24T06:32:36Z

I revisted this since I got further with editable installs working tonight on the new version, so I wanted to re-validate the ones that I commented on the other day, including this one. The new error I got seems to be related to the setup.py reading in other files, because when I edited the setup.py to not read from the two files it opens, it gets past this error.

mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$ pipenv install -e ../piccolo_api/
Creating a virtualenv for this project...
Pipfile: /home/mdavis/shared-projects/pipenv-4895/Pipfile
Using /usr/bin/python3 (3.9.7) to create virtualenv...
⠼ Creating virtual environment...created virtual environment CPython3.9.7.final.0-64 in 173ms
  creator CPython3Posix(dest=/home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mdavis/.local/share/virtualenv)
    added seed packages: pip==21.3.1, setuptools==59.2.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P
Creating a Pipfile for this project...
Installing -e ../piccolo_api/...
Adding piccolo-api to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Updated Pipfile.lock (4f22d1)!
Installing dependencies from Pipfile.lock (4f22d1)...
  🎅   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/home/mdavis/.local/lib/python3.9/site-packages/pipenv/vendor/vistir/path.py", line 433, in rmtree
    shutil.rmtree(directory, ignore_errors=ignore_errors, onerror=onerror)
  File "/usr/lib/python3.9/shutil.py", line 726, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/usr/lib/python3.9/shutil.py", line 667, in _rmtree_safe_fd
    onerror(os.rmdir, fullname, sys.exc_info())
  File "/home/mdavis/.local/lib/python3.9/site-packages/pipenv/vendor/vistir/path.py", line 528, in handle_remove_readonly
    raise exc_exception
  File "/usr/lib/python3.9/shutil.py", line 665, in _rmtree_safe_fd
    os.rmdir(entry.name, dir_fd=topfd)
OSError: [Errno 26] Text file busy: 'piccolo_api.egg-info'

See after I edited the setup.py to not read the files:

mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$ pipenv --rm
Removing virtualenv (/home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P)...
mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$ rm Pipfile
mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$ rm Pipfile.lock 
mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$ pipenv install -e ../piccolo_api/
Creating a virtualenv for this project...
Pipfile: /home/mdavis/shared-projects/pipenv-4895/Pipfile
Using /usr/bin/python3 (3.9.7) to create virtualenv...
⠦ Creating virtual environment...created virtual environment CPython3.9.7.final.0-64 in 274ms
  creator CPython3Posix(dest=/home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P, clear=False, no_vcs_ignore=False, global=False)
  seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/mdavis/.local/share/virtualenv)
    added seed packages: pip==21.3.1, setuptools==59.2.0, wheel==0.37.0
  activators BashActivator,CShellActivator,FishActivator,NushellActivator,PowerShellActivator,PythonActivator

✔ Successfully created virtual environment! 
Virtualenv location: /home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P
Creating a Pipfile for this project...
Installing -e ../piccolo_api/...
Adding piccolo-api to Pipfile's [packages]...
✔ Installation Succeeded 
Pipfile.lock not found, creating...
Locking [dev-packages] dependencies...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✔ Success! 
Warning: /home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P/lib/python3.9/site-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: VERSION is an invalid version and will not be supported in a future release
  warnings.warn(
/home/mdavis/.virtualenvs/pipenv-4895-gjeFja_P/lib/python3.9/site-packages/pkg_resources/__init__.py:116: PkgResourcesDeprecationWarning: VERSION is an invalid version and will not be supported in a future release
  warnings.warn(
Updated Pipfile.lock (4f22d1)!
Installing dependencies from Pipfile.lock (4f22d1)...
  🎅   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 0/0 — 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
mdavis@matt-VirtualBox:~/shared-projects/pipenv-4895$

matteius · 2021-12-24T07:10:35Z

Ok, so the first time it fails, but generates a directory called piccolo_api/reqlib-metadata and subsequent attempts to install editable succeed until you remove that directory.

matteius · 2022-01-04T22:26:33Z

Re-opening this issue since it more accurately documents the issue in latest version of Pipenv 2021.11.23 and closing the older ticket instead after further discussion on that ticket #4551 prompted this decision.

matteius · 2022-03-12T08:50:48Z

@trondhindenes Seems that the behavior is much improved in pip 22.0.4 -- if you try this branch out I think you will see that: #4969

I find that it always fails the first time and retries and does the second time successfully because piccolo_api requires -no-use-pep517 flag to build properly, which is supplied on second attempt. Here is verbose output:

matteius@matteius-VirtualBox:~/shared-projects/pipenv-triage/pipenv-4895$ pipenv install piccolo_api -v
Installing piccolo_api...
Installing package: piccolo_api
Writing supplied requirement line to temporary file: 'piccolo_api'
Installing 'piccolo-api'
⠧ Installing piccolo-api...$ /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/bin/python -m pip install --verbose --upgrade --exists-action=i -r /tmp/pipenv-qgy6wa6c-requirements/pipenv-gjzvgb0z-requirement.txt -i https://pypi.org/simple
Using source directory: '/home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/src'
Adding piccolo-api to Pipfile's [packages]...
✔ Installation Succeeded 
Installing dependencies from Pipfile.lock (cb1480)...
Writing supplied requirement line to temporary file: './piccolo_api'
Installing 'piccolo-api'
$ /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/bin/python -m pip install --verbose --upgrade --no-deps --exists-action=i -r /tmp/pipenv-qgy6wa6c-requirements/pipenv-wgcb7rqc-requirement.txt -i https://pypi.org/simple
Using source directory: '/home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/src'
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:00
Using pip 22.0.3 from /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/lib/python3.10/site-packages/pip (python 3.10)
Processing ./piccolo_api
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: piccolo-api
  Building wheel for piccolo-api (pyproject.toml): started
  Building wheel for piccolo-api (pyproject.toml): finished with status 'error'
Failed to build piccolo-api
An error occurred while installing ./piccolo_api! Will try again.
Installing initially failed dependencies...
Writing supplied requirement line to temporary file: './piccolo_api'
Installing 'piccolo-api'
$ /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/bin/python -m pip install --verbose --upgrade --no-use-pep517 --no-deps --exists-action=i -r /tmp/pipenv-qgy6wa6c-requirements/pipenv-__9y7jhs-requirement.txt -i https://pypi.org/simple
Using source directory: '/home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/src'
Using pip 22.0.3 from /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/lib/python3.10/site-packages/pip (python 3.10)
Processing ./piccolo_api
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: piccolo-api
  Building wheel for piccolo-api (setup.py): started
  Building wheel for piccolo-api (setup.py): finished with status 'error'
  Running setup.py clean for piccolo-api
Failed to build piccolo-api
Installing collected packages: piccolo-api
  Attempting uninstall: piccolo-api
    Found existing installation: piccolo-api 0.33.1
    Uninstalling piccolo-api-0.33.1:
      Removing file or directory /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/lib/python3.10/site-packages/piccolo_api-0.33.1-py3.10.egg-info
      Removing file or directory /home/matteius/.virtualenvs/pipenv-4895-UU7eR6dL/lib/python3.10/site-packages/piccolo_api/
      Successfully uninstalled piccolo-api-0.33.1
  Running setup.py install for piccolo-api: started
  Running setup.py install for piccolo-api: finished with status 'done'
Successfully installed piccolo-api-0.33.1
  ☤  ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 1/1 — 00:00:05
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.

matteius · 2022-04-21T03:14:36Z

Please check with pipenv==2022.4.20

This comment was marked as outdated.

Sign in to view

PythonCoderAS mentioned this issue Dec 24, 2021

Pipenv unable to install relative path that is a subdirectory of a Pipfile #4900

Closed

matteius changed the title ~~install from dir fails when dir contains a dir called "requirements"~~ editable install fails when setup.py opens references to other files as exemplified by piccolo_api Dec 24, 2021

This comment was marked as outdated.

Sign in to view

matteius changed the title ~~editable install fails when setup.py opens references to other files as exemplified by piccolo_api~~ editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) Dec 24, 2021

matteius mentioned this issue Dec 24, 2021

Pipenv install is not adding dependencies to Pipfile.lock #4703

Closed

matteius added the Type: Duplicate This issue is a duplicate of an already-existing issue. label Dec 24, 2021

This comment was marked as outdated.

Sign in to view

matteius closed this as completed Dec 24, 2021

matteius reopened this Jan 4, 2022

matteius removed the Type: Duplicate This issue is a duplicate of an already-existing issue. label Jan 4, 2022

matteius mentioned this issue Jan 4, 2022

Pipenv 2020.11.15 does not resolve local package dependencies if complex setup.py #4551

Closed

matteius added the Type: Possible Bug This issue describes a possible bug in pipenv. label Jan 4, 2022

matteius added the PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. label Mar 12, 2022

matteius closed this as completed Apr 21, 2022

matteius removed the PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. label Apr 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) #4895

editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) #4895

trondhindenes commented Dec 18, 2021 •

edited

Loading

This comment was marked as outdated.

matteius commented Dec 19, 2021

matteius commented Dec 24, 2021 •

edited

Loading

This comment was marked as outdated.

matteius commented Dec 24, 2021

This comment was marked as outdated.

matteius commented Jan 4, 2022

matteius commented Mar 12, 2022

matteius commented Apr 21, 2022

editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) #4895

editable install sometimes fails when setup.py opens other files (exemplified by piccolo_api) #4895

Comments

trondhindenes commented Dec 18, 2021 • edited Loading

Issue description

Expected result

Actual result

Steps to replicate

This comment was marked as outdated.

matteius commented Dec 19, 2021

matteius commented Dec 24, 2021 • edited Loading

This comment was marked as outdated.

matteius commented Dec 24, 2021

This comment was marked as outdated.

matteius commented Jan 4, 2022

matteius commented Mar 12, 2022

matteius commented Apr 21, 2022

trondhindenes commented Dec 18, 2021 •

edited

Loading

matteius commented Dec 24, 2021 •

edited

Loading