-
Notifications
You must be signed in to change notification settings - Fork 3.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
{Packaging} Optimize Linux package and docker image by removing py file #25801
base: dev
Are you sure you want to change the base?
Conversation
️✔️acr
️✔️acs
️✔️advisor
️✔️ams
️✔️apim
️✔️appconfig
️✔️appservice
️✔️aro
️✔️backup
️✔️batch
️✔️batchai
️✔️billing
️✔️botservice
️✔️cdn
️✔️cloud
️✔️cognitiveservices
️✔️config
️✔️configure
️✔️consumption
️✔️container
️✔️core
️✔️cosmosdb
️✔️databoxedge
️✔️dla
️✔️dls
️✔️dms
️✔️eventgrid
️✔️eventhubs
️✔️feedback
️✔️find
️✔️hdinsight
️✔️identity
️✔️iot
️✔️keyvault
️✔️kusto
️✔️lab
️✔️managedservices
️✔️maps
️✔️marketplaceordering
️✔️monitor
️✔️netappfiles
️✔️network
️✔️policyinsights
️✔️privatedns
️✔️profile
️✔️rdbms
️✔️redis
️✔️relay
️✔️resource
️✔️role
️✔️search
️✔️security
️✔️servicebus
️✔️serviceconnector
️✔️servicefabric
️✔️signalr
️✔️sql
️✔️sqlvm
️✔️storage
️✔️synapse
️✔️telemetry
️✔️util
️✔️vm
|
Packaging |
# Python image has build-in env $PYTHON_VERSION=3.10.10. | ||
# `ARG PYTHON_VERSION="3.10"` works on ARM64, but it can't override the default value on AMD64. | ||
RUN ./scripts/install_full.sh && python ./scripts/trim_sdk.py \ | ||
&& python ./scripts/use_pyc.py /usr/local/lib/python${PYTHON_VERSION:0:4}/site-packages/ \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would prefer to make the pyc change in a separate PR, just in case something goes wrong and we need to roll back.
# Conflicts: # Dockerfile # scripts/release/debian/build.sh
# If pip's py files are also removed, the error is raised when installing some packages. | ||
# See https://github.com/Azure/azure-cli/pull/25801 for details. | ||
if os.path.join('site-packages', 'pip') in file: | ||
continue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removing py file has side effects.
For example: /opt/az/bin/python3 -m pip install --no-cache pgcli==3.4.1
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 2
╰─> [1 lines of output]
/opt/az/bin/python3: can't open file '/opt/az/lib/python3.10/site-packages/pip/__pip-runner__.py': [Errno 2] No such file or directory
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
The MSI also keeps py files under pip
folder.
|
||
for f in glob.glob(f'{folder}/**/__pycache__', recursive=True): | ||
# Remove emtpy __pycache__ folder | ||
shutil.rmtree(f) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Need to add -B
in Linux entry point to prevent generating pyc in pip
folder.
Description
We have used
trim_sdk.py
and only keep pyc file on Windows. Apply them on Linux and docker to reduce package size.Trimming SKD is #26172
This is the second part, removing py file on Linux.
Final result:
Ubuntu 22.04 installed size 576 MB -> 326 MB, become 56% of original size.
RHEL 9 installed size 403 MB -> 172 MB, become 43% of original size.
Docker image size 956 MB -> 714 MB, become 75% of original size.
History Notes
[Component Name 1] BREAKING CHANGE:
az command a
: Make some customer-facing breaking change[Component Name 2]
az command b
: Add some customer-facing featureThis checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.