Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Azure ML extension failed to be installed in the docker image #28992

Closed
t3mi opened this issue May 20, 2024 · 4 comments
Closed

Azure ML extension failed to be installed in the docker image #28992

t3mi opened this issue May 20, 2024 · 4 comments
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning az ml Service Attention This issue is responsible by Azure service team.

Comments

@t3mi
Copy link

t3mi commented May 20, 2024 •
edited
Loading

Describe the bug

Installation of ml extension inside latest azure-cli docker image fails with error for missing dependencies.

Related command

docker run -it mcr.microsoft.com/azure-cli:2.60.0 bash

az extension add --name ml

Errors

ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects

Issue script & Debug output

https://files.pythonhosted.org:443 "GET /packages/fb/2b/a64c2d25a37aeb921fddb929111413049fc5f8b9a4c1aefaffaafe768d54/cachetools-5.3.3-py3-none-any.whl HTTP/1.1" 200 9325
Downloading cachetools-5.3.3-py3-none-any.whl (9.3 kB)
Downloading link https://files.pythonhosted.org/packages/fb/2b/a64c2d25a37aeb921fddb929111413049fc5f8b9a4c1aefaffaafe768d54/cachetools-5.3.3-py3-none-any.whl (from https://pypi.org/simple/cachetools/) (requires-python:>=3.7) to /tmp/pip-unpack-lm4i20c_/cachetools-5.3.3-py3-none-any.whl
https://files.pythonhosted.org:443 "GET /packages/13/68/8906226b15ef38e71dc926c321d2fe99de8048e9098b5dfd38343011c886/pyasn1_modules-0.4.0-py3-none-any.whl HTTP/1.1" 200 181220
Downloading pyasn1_modules-0.4.0-py3-none-any.whl (181 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 181.2/181.2 kB 174.5 MB/s eta 0:00:00
Downloading link https://files.pythonhosted.org/packages/13/68/8906226b15ef38e71dc926c321d2fe99de8048e9098b5dfd38343011c886/pyasn1_modules-0.4.0-py3-none-any.whl (from https://pypi.org/simple/pyasn1-modules/) (requires-python:>=3.8) to /tmp/pip-unpack-lm4i20c_/pyasn1_modules-0.4.0-py3-none-any.whl
https://files.pythonhosted.org:443 "GET /packages/49/97/fa78e3d2f65c02c8e1268b9aba606569fe97f6c8f7c2d74394553347c145/rsa-4.9-py3-none-any.whl HTTP/1.1" 200 34315
Downloading rsa-4.9-py3-none-any.whl (34 kB)
Downloading link https://files.pythonhosted.org/packages/49/97/fa78e3d2f65c02c8e1268b9aba606569fe97f6c8f7c2d74394553347c145/rsa-4.9-py3-none-any.whl (from https://pypi.org/simple/rsa/) (requires-python:>=3.6,<4) to /tmp/pip-unpack-lm4i20c_/rsa-4.9-py3-none-any.whl
https://files.pythonhosted.org:443 "GET /packages/23/7e/5f50d07d5e70a2addbccd90ac2950f81d1edd0783630651d9268d7f1db49/pyasn1-0.6.0-py2.py3-none-any.whl HTTP/1.1" 200 85313
Downloading pyasn1-0.6.0-py2.py3-none-any.whl (85 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 85.3/85.3 kB 164.4 MB/s eta 0:00:00
Downloading link https://files.pythonhosted.org/packages/23/7e/5f50d07d5e70a2addbccd90ac2950f81d1edd0783630651d9268d7f1db49/pyasn1-0.6.0-py2.py3-none-any.whl (from https://pypi.org/simple/pyasn1/) (requires-python:>=3.8) to /tmp/pip-unpack-lm4i20c_/pyasn1-0.6.0-py2.py3-none-any.whl
Building wheels for collected packages: psutil
  Created temporary directory: /tmp/pip-wheel-oiv_w1ad
  Destination directory: /tmp/pip-wheel-oiv_w1ad
  Building wheel for psutil (pyproject.toml): started
  Running command Building wheel for psutil (pyproject.toml)
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-311
  creating build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/__init__.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_pssunos.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_psosx.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_psbsd.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_compat.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_pslinux.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_psposix.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_common.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_psaix.py -> build/lib.linux-x86_64-cpython-311/psutil
  copying psutil/_pswindows.py -> build/lib.linux-x86_64-cpython-311/psutil
  creating build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/__init__.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_unicode.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_windows.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_posix.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_osx.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_connections.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_sunos.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/runner.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/__main__.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_linux.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_aix.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_misc.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_process.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_testutils.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_bsd.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_system.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_process_all.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_contracts.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  copying psutil/tests/test_memleaks.py -> build/lib.linux-x86_64-cpython-311/psutil/tests
  running build_ext
  building 'psutil._psutil_linux' extension
  creating build/temp.linux-x86_64-cpython-311
  creating build/temp.linux-x86_64-cpython-311/psutil
  creating build/temp.linux-x86_64-cpython-311/psutil/arch
  creating build/temp.linux-x86_64-cpython-311/psutil/arch/linux
  gcc -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPSUTIL_POSIX=1 -DPSUTIL_SIZEOF_PID_T=4 -DPSUTIL_VERSION=598 -DPy_LIMITED_API=0x03060000 -DPSUTIL_ETHTOOL_MISSING_TYPES=1 -DPSUTIL_LINUX=1 -I/usr/local/include/python3.11 -c psutil/_psutil_common.c -o build/temp.linux-x86_64-cpython-311/psutil/_psutil_common.o
  psutil could not be installed from sources because gcc is not installed. Try running:
    sudo apk add gcc python3-dev musl-dev linux-headers
  error: command 'gcc' failed: No such file or directory
  error: subprocess-exited-with-error
  
  × Building wheel for psutil (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/local/bin/python /usr/local/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmphevkhg0h
  cwd: /tmp/pip-install-2bqu66n2/psutil_6d39ec9e9d2d44008e5b1c1702761c75
  Building wheel for psutil (pyproject.toml): finished with status 'error'
  ERROR: Failed building wheel for psutil
Failed to build psutil
ERROR: Could not build wheels for psutil, which is required to install pyproject.toml-based projects
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 429, in run
    raise InstallationError(
pip._internal.exceptions.InstallationError: Could not build wheels for psutil, which is required to install pyproject.toml-based projects
Removed build tracker: '/tmp/pip-build-tracker-xv4ofcay'

cli.azure.cli.core.extension.operations: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/root/.azure/cliextensions/ml', '/tmp/tmp_jbigcd3/ml-2.26.0-py3-none-any.whl', '-vv', '--disable-pip-version-check', '--no-cache-dir']' returned non-zero exit status 1.
cli.azure.cli.core.extension.operations: Pip failed so deleting anything we might have installed at /root/.azure/cliextensions/ml
cli.azure.cli.core.azclierror: Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 664, in execute
    raise ex
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 731, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 701, in _run_job
    result = cmd_copy(params)
             ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py", line 334, in __call__
    return self.handler(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
           ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/command_modules/extension/custom.py", line 16, in add_extension_cmd
    return add_extension(cli_ctx=cmd.cli_ctx, source=source, extension_name=extension_name, index_url=index_url,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 349, in add_extension
    extension_name = _add_whl_ext(cli_ctx=cmd_cli_ctx, source=source, ext_sha256=ext_sha256,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/azure/cli/core/extension/operations.py", line 172, in _add_whl_ext
    raise CLIError('An error occurred. Pip failed with status code {}. '
knack.util.CLIError: An error occurred. Pip failed with status code 1. Use --debug for more information.

cli.azure.cli.core.azclierror: An error occurred. Pip failed with status code 1. Use --debug for more information.
az_command_data_logger: An error occurred. Pip failed with status code 1. Use --debug for more information.
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x76e7013ed440>]
az_command_data_logger: exit code: 1
cli.__main__: Command ran in 31.891 seconds (init: 2.180, invoke: 29.712)
cli.azure.cli.core._profile: Failed to load installationId from AzureRmSurvey.json. [Errno 2] No such file or directory: '/root/.azure/AzureRmContextSettings.json'
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3621 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/local/bin/python /usr/local/lib/python3.11/site-packages/azure/cli/telemetry/__init__.py /root/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.

Expected behavior

Command succeeds and ml is available.

Environment Summary

azure-cli 2.60.0

core 2.60.0
telemetry 1.1.0

Dependencies:
msal 1.28.0
azure-mgmt-resource 23.1.0b2

Python location '/usr/local/bin/python'
Extensions directory '/root/.azure/cliextensions'

Python (Linux) 3.11.9 (main, Apr 4 2024, 00:38:40) [GCC 13.2.1 20231014]

Legal docs and information: aka.ms/AzureCliLegal

Your CLI is up-to-date.

Additional context

No response
@t3mi t3mi added the bug This issue requires a change to an existing behavior in the product in order to be resolved. label May 20, 2024
@microsoft-github-policy-service microsoft-github-policy-service bot added the customer-reported Issues that are reported by GitHub users external to the Azure organization. label May 20, 2024
@yonzhan
Copy link
Collaborator

yonzhan commented May 20, 2024

Thank you for opening this issue, we will look into it.

@yonzhan yonzhan added Machine Learning az ml Service Attention This issue is responsible by Azure service team. labels May 20, 2024
@bebound
Copy link
Contributor

bebound commented May 21, 2024

See PR description: #27567
Some of these packages might be needed to compile psutil: gcc make libffi-dev musl-dev linux-headers

@bebound bebound closed this as completed May 21, 2024
@t3mi
Copy link
Author

t3mi commented May 21, 2024

@bebound are you proposing to build custom docker image just to install additional azure-cli extension?

@bebound
Copy link
Contributor

bebound commented May 22, 2024

Simply run apk add gcc musl-dev linux-headers before running az extension add.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. customer-reported Issues that are reported by GitHub users external to the Azure organization. Machine Learning az ml Service Attention This issue is responsible by Azure service team.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bebound @t3mi @yonzhan