pipenv not resolving 2 editable pkgs with a dependency between them #4323

chrisroat · 2020-06-06T06:13:30Z

Issue description

Installing two local editable packages, where one depends on the other, fails to resolve the dependency. I tried both with and without a version requirement on the dependency. (Report below includes the version requirement.)

Expected result

Both packages get installed in my environment.

Actual result

Locking Failed!

It literally has these two lines in the output:

  found candidate pkgb==0.0.1 (constraint was ==0.0.1)

No matching distribution found for pkgb==0.0.1

Full output


$ pipenv install --verbose
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
⠸ Locking...
                          ROUND 1                           
✘ Locking Failed! 
Current constraints:
  pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))

Finding the best candidates:
  found candidate pkgb==0.0.1 (constraint was ==0.0.1)

Finding secondary dependencies:
  pkgb==0.0.1 not in cache, need to check index
CRITICAL:pipenv.patched.notpip._internal.index.package_finder:Could not find a version that satisfies the requirement pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2)) (from versions: none)
Traceback (most recent call last):
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 808, in resolve
    results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 180, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 268, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 383, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 226, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 348, in get_legacy_dependencies
    download_dir, ireq, wheel_cache
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 303, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 339, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 286, in _get_abstract_dist_for
    req.populate_link(self.finder, upgrade_allowed, require_hashes)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/req/req_install.py", line 249, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/index/package_finder.py", line 963, in find_requirement
    'No matching distribution found for %s' % req
pipenv.patched.notpip._internal.exceptions.DistributionNotFound: No matching distribution found for pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 1386, in resolve_deps
    req_dir=req_dir,
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 1093, in actually_resolve_deps
    resolver.resolve()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 818, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: ERROR: No matching distribution found for pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))
ROUND 1                           
Current constraints:
  pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))

Finding the best candidates:
  found candidate pkgb==0.0.1 (constraint was ==0.0.1)

Finding secondary dependencies:
  pkgb==0.0.1 not in cache, need to check index
CRITICAL:pipenv.patched.notpip._internal.index.package_finder:Could not find a version that satisfies the requirement pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2)) (from versions: none)
Traceback (most recent call last):
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 808, in resolve
    results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 180, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 268, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/resolver.py", line 383, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 226, in get_dependencies
    legacy_results = self.get_legacy_dependencies(ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 348, in get_legacy_dependencies
    download_dir, ireq, wheel_cache
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 303, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 339, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/legacy_resolve.py", line 286, in _get_abstract_dist_for
    req.populate_link(self.finder, upgrade_allowed, require_hashes)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/req/req_install.py", line 249, in populate_link
    self.link = finder.find_requirement(self, upgrade)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/patched/notpip/_internal/index/package_finder.py", line 963, in find_requirement
    'No matching distribution found for %s' % req
pipenv.patched.notpip._internal.exceptions.DistributionNotFound: No matching distribution found for pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 807, in <module>
    main()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 803, in main
    parsed.requirements_dir, parsed.packages, parse_only=parsed.parse_only)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 785, in _main
    resolve_packages(pre, clear, verbose, system, write, requirements_dir, packages)
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 753, in resolve_packages
    requirements_dir=requirements_dir,
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/resolver.py", line 736, in resolve
    req_dir=requirements_dir
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 1386, in resolve_deps
    req_dir=req_dir,
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 1093, in actually_resolve_deps
    resolver.resolve()
  File "/home/croat/.local/lib/python3.7/site-packages/pipenv/utils.py", line 818, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: ERROR: No matching distribution found for pkgb==0.0.1 (from -r /tmp/pipenv39ey2xbnrequirements/pipenv-7_6gdduw-constraints.txt (line 2))

Steps to replicate

Three files in 2 subdirectories:

$ ls -R
.:
pkga/  pkgb/

./pkga:
Pipfile  setup.py

./pkgb:
setup.py

$ cat pkga/Pipfile 
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pkga = {editable = true,path = "."}
pkgb = {editable = true,path = "./../pkgb"}

[requires]
python_version = "3.7"

$ cat pkga/setup.py 
import setuptools

setuptools.setup(
    name='pkga',
    version='0.0.1',
    packages=setuptools.find_packages(),
    python_requires='>=3.7',
    install_requires=[
        'pkgb==0.0.1',
    ],
)

$ cat pkgb/setup.py 
import setuptools

setuptools.setup(
    name='pkgb',
    version='0.0.1',
    packages=setuptools.find_packages(),
    python_requires='>=3.7',
)

$ cd pkga; pipenv install
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Building requirements...
Resolving dependencies...
✘ Locking Failed! 
CRITICAL:pipenv.patched.notpip._internal.index.package_finder:Could not find a version that satisfies the requirement pkgb==0.0.1 (from -r /tmp/pipenvrk62qkx_requirements/pipenv-mje5yldw-constraints.txt (line 2)) (from versions: none)

<see full output above>

$ pipenv --support

Pipenv version: '2020.6.2'

Pipenv location: '/home/croat/.local/lib/python3.7/site-packages/pipenv'

Python location: '/usr/bin/python3.7'

Python installations found:

3.7.5: /usr/bin/python3.7m
3.7.5: /usr/bin/python3.7
3.6.9: /usr/bin/python3.6m
3.6.9: /usr/bin/python3
3.6.9: /usr/bin/python3.6
2.7.17: /usr/bin/python2
2.7.17: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.7.5',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.3.0-53-generic',
 'platform_system': 'Linux',
 'platform_version': '#47~18.04.1-Ubuntu SMP Thu May 7 13:10:50 UTC 2020',
 'python_full_version': '3.7.5',
 'python_version': '3.7',
 'sys_platform': 'linux'}

System environment variables:

CLUTTER_IM_MODULE
CONDA_SHLVL
NVM_DIR
CONDA_EXE
XDG_MENU_PREFIX
LANG
HISTIGNORE
HISTCONTROL
DISPLAY
HISTTIMEFORMAT
OLDPWD
GNOME_SHELL_SESSION_MODE
EDITOR
COLORTERM
NVM_CD_FLAGS
BASH_SILENCE_DEPRECATION_WARNING
USERNAME
XDG_VTNR
SSH_AUTH_SOCK
S_COLORS
_CE_M
XDG_SESSION_ID
USER
DESKTOP_SESSION
QT4_IM_MODULE
TEXTDOMAINDIR
GNOME_TERMINAL_SCREEN
PWD
HOME
CONDA_PYTHON_EXE
TEXTDOMAIN
SSH_AGENT_PID
QT_ACCESSIBILITY
XDG_SESSION_TYPE
XDG_DATA_DIRS
_CE_CONDA
XDG_SESSION_DESKTOP
GJS_DEBUG_OUTPUT
GTK_MODULES
VISUAL
WINDOWPATH
TERM
SHELL
VTE_VERSION
QT_IM_MODULE
XMODIFIERS
IM_CONFIG_PHASE
NVM_BIN
XDG_CURRENT_DESKTOP
GPG_AGENT_INFO
GNOME_TERMINAL_SERVICE
XDG_SEAT
SHLVL
GDMSESSION
GNOME_DESKTOP_SESSION_ID
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XDG_RUNTIME_DIR
XAUTHORITY
XDG_CONFIG_DIRS
PATH
PS1
HISTSIZE
GJS_DEBUG_TOPICS
HISTFILESIZE
SESSION_MANAGER
GTK_IM_MODULE
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/croat/miniconda3/condabin:/home/croat/.nvm/versions/node/v14.3.0/bin:/home/croat/google-cloud-sdk/bin:/home/croat/bin:/home/croat/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SHELL: /bin/bash
EDITOR: emacs
LANG: en_US.UTF-8
PWD: /home/croat/multipackage_deps/pkga

Contents of Pipfile ('/home/croat/multipackage_deps/pkga/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pkga = {editable = true,path = "."}
pkgb = {editable = true,path = "./../pkgb"}

[requires]
python_version = "3.7"

The text was updated successfully, but these errors were encountered:

matteius · 2022-08-17T13:50:28Z

I believe this was resolved, can you check with pipenv=2022.8.17?

chrisroat · 2022-08-17T15:13:19Z

I tried the reproducer given in the initial comment; it still fails, albeit with a different error.

output

Pipfile.lock not found, creating...
Locking [packages] dependencies...
Building requirements...
Resolving dependencies...
✘ Locking Failed! 

CRITICAL:pipenv.patched.pip._internal.resolution.resolvelib.factory:Could not find a version that satisfies the requirement pkgb==0.0.1 (from versions: none)
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 825, in _main
[ResolutionFailure]:       pre, clear, verbose, system, write, requirements_dir, packages, dev
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 780, in resolve_packages
[ResolutionFailure]:       requirements_dir=requirements_dir,
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/resolver.py", line 760, in resolve
[ResolutionFailure]:       req_dir=requirements_dir,
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 1135, in resolve_deps
[ResolutionFailure]:       req_dir=req_dir,
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 919, in actually_resolve_deps
[ResolutionFailure]:       resolver.resolve()
[ResolutionFailure]:   File "/home/croat/.local/lib/python3.8/site-packages/pipenv/utils/resolver.py", line 723, in resolve
[ResolutionFailure]:       raise ResolutionFailure(message=str(e))
[pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could not be resolved. You likely have a mismatch in your sub-dependencies.
  You can use $ pipenv install --skip-lock to bypass this mechanism, then run $ pipenv graph to inspect the situation.
  Hint: try $ pipenv lock --pre if it is a pre-release dependency.
ERROR: No matching distribution found for pkgb==0.0.1

$ pipenv --support

Pipenv version: '2022.8.17'

Pipenv location: '/home/croat/.local/lib/python3.8/site-packages/pipenv'

setuptools version: '45.2.0'

Python location: '/usr/bin/python3'

OS Name: 'posix'

User pip version: '22.2.2'

user Python installations found:

3.8.10: /usr/bin/python3.8
3.8.10: /usr/bin/python3
3.7.13: /usr/bin/python3.7m
3.7.13: /usr/bin/python3.7
2.7.18: /usr/bin/python
2.7.18: /usr/bin/python2
2.7.18: /usr/bin/python2.7

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.8.10',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '5.4.0-120-generic',
 'platform_system': 'Linux',
 'platform_version': '#136-Ubuntu SMP Fri Jun 10 13:40:48 UTC 2022',
 'python_full_version': '3.8.10',
 'python_version': '3.8',
 'sys_platform': 'linux'}

System environment variables:

SHELL
HISTCONTROL
CONDA_EXE
_CE_M
HISTSIZE
SSH_AUTH_SOCK
HISTTIMEFORMAT
SSH_AGENT_PID
EDITOR
PWD
LOGNAME
HOME
LANG
SSH_CONNECTION
BASH_SILENCE_DEPRECATION_WARNING
NVM_DIR
TERM
_CE_CONDA
USER
CONDA_SHLVL
VISUAL
SHLVL
NVM_CD_FLAGS
CONDA_PYTHON_EXE
SSH_CLIENT
LC_ALL
XDG_DATA_DIRS
PATH
HISTIGNORE
HISTFILESIZE
NVM_BIN
MAIL
SSH_TTY
_
OLDPWD
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
PIP_PYTHON_PATH
PYTHONDONTWRITEBYTECODE
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/croat/miniconda3/condabin:/home/croat/.nvm/versions/node/v14.3.0/bin:/home/croat/bin:/home/croat/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/snap/bin:/usr/local/go/bin
SHELL: /bin/bash
EDITOR: emacs
LANG: en_US.UTF-8
PWD: /home/croat/pipenv-4323/pkga

Contents of Pipfile ('/home/croat/pipenv-4323/pkga/Pipfile'):

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[dev-packages]

[packages]
pkga = {editable = true,path = "."}
pkgb = {editable = true,path = "./../pkgb"}

[requires]
python_version = "3.7"

matteius · 2022-08-18T01:04:07Z

@chrisroat I wonder if recent work on constraint files is affecting it on this version, would you be able to check if it worked with 2022.8.5?

matteius · 2022-08-18T01:11:19Z

Actually sorry, I just tried it and it still wasn't working in 2022.8.5, it felt really similar to a scrambled_eggs example with two editable packages, but the difference is there wasn't two setup.py and the setup.py did not call out the other package in the install_requires.

matteius · 2022-08-18T01:30:32Z

Yeah, fwiw it works if I take the install_requires `'pkgb==0.0.1' from the other setup.py. It appears that this constraint gets interpreted by the pip resolver + resolvelib to look for that package in the resolution phase and doesn't detect that it was also told to install the editable dependency with the same name.

matteius · 2022-08-18T03:10:50Z

I worked out a PR that fixes the issue at hand, but I am not sure about any subtilties where the install_requires a different version than what your editable dependency provides, because basically I refactor some code so that I can match up if the requirement.name is an editable dependency from the Pipfile to not include that additional constraint to the pip resolver which would never be able to find it.

triage-new-issues bot added the triage label Jun 6, 2020

nnutter mentioned this issue Jul 1, 2022

Local inter-dependency resolution fails #4784

Closed

matteius added the Status: Awaiting Update ⏳ This issue requires more information before assistance can be provided. label Aug 17, 2022

matteius mentioned this issue Aug 18, 2022

Exclude from resolver constraints the editable dependencies that will be installed. #5271

Merged

matteius added PR: awaiting-review The PR related to this issue is awaiting review by a maintainer. and removed triage Contributor Candidate The issue has been identified/triaged and contributions are welcomed/encouraged. labels Aug 18, 2022

matteius self-assigned this Aug 18, 2022

matteius closed this as completed in #5271 Aug 18, 2022

matteius mentioned this issue Aug 22, 2022

Pipenv>=2020 can't install local packages depending between them #4553

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pipenv not resolving 2 editable pkgs with a dependency between them #4323

pipenv not resolving 2 editable pkgs with a dependency between them #4323

chrisroat commented Jun 6, 2020

matteius commented Aug 17, 2022

chrisroat commented Aug 17, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022

pipenv not resolving 2 editable pkgs with a dependency between them #4323

pipenv not resolving 2 editable pkgs with a dependency between them #4323

Comments

chrisroat commented Jun 6, 2020

Issue description

Expected result

Actual result

Steps to replicate

matteius commented Aug 17, 2022

chrisroat commented Aug 17, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022

matteius commented Aug 18, 2022