ImportError: No module named numpy.distutils.core when installing using pip

[thomasgilgenast]

When installing iced using pip:

$ pip install iced
...
Building wheels for collected packages: iced
  Building wheel for iced (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/tgg/venv-test/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-vT3ByC
       cwd: /tmp/pip-install-jD4T4X/iced/
  Complete output (7 lines):
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-jD4T4X/iced/setup.py", line 112, in <module>
      setup_package()
    File "/tmp/pip-install-jD4T4X/iced/setup.py", line 104, in setup_package
      from numpy.distutils.core import setup
  ImportError: No module named numpy.distutils.core
  ----------------------------------------
  ERROR: Failed building wheel for iced
  Running setup.py clean for iced
Failed to build iced
Installing collected packages: iced
    Running setup.py install for iced ... error
    ERROR: Command errored out with exit status 1:
     command: /home/tgg/venv-test/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-N3SMcO/install-record.txt --single-version-externally-managed --compile --install-headers /home/tgg/venv-test/include/site/python2.7/iced
         cwd: /tmp/pip-install-jD4T4X/iced/
    Complete output (7 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-jD4T4X/iced/setup.py", line 112, in <module>
        setup_package()
      File "/tmp/pip-install-jD4T4X/iced/setup.py", line 104, in setup_package
        from numpy.distutils.core import setup
    ImportError: No module named numpy.distutils.core
    ----------------------------------------
ERROR: Command errored out with exit status 1: /home/tgg/venv-test/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jD4T4X/iced/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-N3SMcO/install-record.txt --single-version-externally-managed --compile --install-headers /home/tgg/venv-test/include/site/python2.7/iced Check the logs for full command output.

The issue appears to be related to numpy not being present before iced is installed. This specific example is intended to be a minimal example and is somewhat contrived, but the same issue arises when doing a standard pip install -r requirements.txt with a requirements.txt file that includes both iced and numpy: pip builds wheels for all packages first before installing any packages, so even though the requirements.txt file includes numpy, pip will attempt to build the wheel for iced before numpy is installed.

A typical "old-school" workaround is to perform the installation in two steps, e.g.:

$ pip install $(grep requirements.txt numpy) && pip install -r requirements.txt

It is possible to avoid this extra step and recover the simplicity of "pip install -r requirements.txt" by declaring that the installation depends on numpy using PEP 518. The implementation for this could simply be to add a pyproject.toml file with the following contents:

[build-system]
requires=['setuptools', 'wheel', 'numpy']

This would have no impact on users using older versions of the Python packaging tools (these older versions just ignore pyproject.toml), but users using newer versions would be able to directly install iced using pip without knowing anything about the confusing history of Python packaging.

[thomasgilgenast]

I've added an implementation of adding a minimal pyproject.toml file in #77 to confirm that it works as expected (can pip install iced in an empty virtual environment).

[NelleV]

Thanks for the PR. This will be fixed during the next release of iced.

[NelleV]

(Actually, we still need to add the pyproject.toml to the tarball, so I'm reopening this ticket.)

[thomasgilgenast]

Thanks for catching that and apologies that I did not catch it - I was testing my PR by installing using pip install git+https://... which downloads everything in the repository I believe (ignoring MANIFEST.in).

Just now, I tried first building the sdist (python setup.py sdist) and then pip install iced*.tar.gz, which mimics what pip would do when installing from PyPI, etc. I can confirm that pyproject.toml doesn't get added to the sdist, so the installation fails. Based on this issue and this PR, super-new (since 1 Jan 2020) versions of setuptools do include pyproject.toml by default in sdists, but this is irrelevant for iced because I believe iced does not use setuptools to handle the setup.py sdist command.

I've opened a new PR (sorry for the confusion!) to include pyproject.toml in MANIFEST.in here: #78

[NelleV]

No problem! I was reopening the ticket in order to not forget about adding this to the manifest file.

Thanks a lot for the contributions!