Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OBS-504: Optimize Socorro builds for better FE dev speed #6908

Merged
merged 5 commits into from
Feb 18, 2025
Merged

OBS-504: Optimize Socorro builds for better FE dev speed #6908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
9d6d79d
OBS-504: reorder docker commands to improve FE build speed
toufali Feb 14, 2025
18710da
OBS-504: exclude devcontainer from default build target
toufali Feb 14, 2025
2134275
OBS-504: reorder devcontainer docker commands
toufali Feb 14, 2025
cb5d665
Update justfile
toufali Feb 18, 2025
99d659c
Update justfile
toufali Feb 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .devcontainer/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,6 @@ COPY docker/set_up_stackwalker.sh /tmp/set_up_stackwalker.sh
RUN /tmp/set_up_stackwalker.sh && \
rm /tmp/set_up_stackwalker.sh

# Install frontend JS deps
COPY --chown=app:app ./webapp/package*.json /webapp-frontend-deps/
RUN cd /webapp-frontend-deps/ && npm install

# Install Python dependencies
COPY requirements.txt /app/
RUN pip install --no-cache-dir --no-deps -r requirements.txt && \
Expand All @@ -44,3 +40,7 @@ ENV PYTHONUNBUFFERED=1 \
CSSMIN_BINARY=/webapp-frontend-deps/node_modules/.bin/cssmin \
NPM_ROOT_PATH=/webapp-frontend-deps/ \
NODE_PATH=/webapp-frontend-deps/node_modules/

# Install frontend JS deps
COPY --chown=app:app ./webapp/package*.json /webapp-frontend-deps/
RUN cd /webapp-frontend-deps/ && npm install
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My understanding is that this will speed up installing new web dependencies, while it slows down updating Python dependencies, right? I don't have an opinion on this, just asking for clarification.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's possible: if changes are made to the python dependencies it will rebuild the JS deps on the subsequent layer. However, my experience suggests the JS deps are minimal and build much faster than the python deps (seconds vs minutes).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggested making this change locally on your machine because it would speed up the loop you were doing often. I didn't intend to suggest we make this change globally.

Typically, we want to order steps in the Dockerfile to improve cache hits when doing typical work. We typically don't make changes to the JS environment, but we do land Python environment PRs from Dependabot and elsewhere, so we'll now be rebuilding the JS environment over and over. It doesn't add a lot on my machine (4s), so I'm game for making this change globally for everyone, but I would suggest we revert it it leads to longer build times for typical work.

Going back to the bigger problem of it taking a long time to build Socorro, it takes my machine 1m40s to do a full no-cache rebuild. It's curious it takes 4+ minutes for you. There might be other things you could look into to speed it up locally.

8 changes: 4 additions & 4 deletions docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,6 @@ COPY docker/set_up_stackwalker.sh /tmp/set_up_stackwalker.sh
RUN /tmp/set_up_stackwalker.sh && \
rm /tmp/set_up_stackwalker.sh

# Install frontend JS deps
COPY --chown=app:app ./webapp/package*.json /webapp-frontend-deps/
RUN cd /webapp-frontend-deps/ && npm install

COPY --chown=app:app requirements.txt /app/
RUN pip install --no-cache-dir --no-deps -r requirements.txt && \
pip check --disable-pip-version-check
Expand All @@ -42,6 +38,10 @@ ENV PYTHONUNBUFFERED=1 \
NPM_ROOT_PATH=/webapp-frontend-deps/ \
NODE_PATH=/webapp-frontend-deps/node_modules/

# Install frontend JS deps
COPY --chown=app:app ./webapp/package*.json /webapp-frontend-deps/
RUN cd /webapp-frontend-deps/ && npm install

# app should own everything under /app in the container
USER app

Expand Down
2 changes: 1 addition & 1 deletion justfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ _env:
fi

# Build docker images
build *args: _env
build *args='app fakesentry oidcprovider elasticsearch postgresql pubsub memcached gcs-emulator': _env
docker compose build --progress plain {{args}}

# Set up Postgres, Elasticsearch, local Pub/Sub, and local GCS services.
Expand Down