Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Turbopack] scale down process pool when module graph is finished #75896

Merged
merged 3 commits into from
Feb 12, 2025
Merged

[Turbopack] scale down process pool when module graph is finished #75896

merged 3 commits into from
Feb 12, 2025

Conversation

sokra
Copy link
Member

@sokra sokra commented Feb 11, 2025 •
edited by padmaia
Loading

What?

in build mode we don't need to process pools anymore after the module graph has finished.

Note: I'm not super happy with this design, but it will work for now... In future we need something more general...

Closes PACK-3948

@ijjk ijjk added created-by: Turbopack team PRs by the Turbopack team. Turbopack Related to Turbopack with Next.js. labels Feb 11, 2025
@sokra sokra changed the title scale down process pool when module graph is finished [Turbopack] scale down process pool when module graph is finished Feb 11, 2025
@sokra Graphite App
Copy link
Member Author

sokra commented Feb 11, 2025 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@sokra sokra mentioned this pull request Feb 11, 2025
Merged
@sokra sokra requested a review from mischnic February 11, 2025 08:40
@sokra sokra marked this pull request as ready for review February 11, 2025 08:40
@ijjk
Copy link
Member

ijjk commented Feb 11, 2025 •
edited
Loading

Tests Passed

@ijjk
Copy link
Member

ijjk commented Feb 11, 2025 •
edited
Loading

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
buildDuration 19.8s 18.2s N/A
buildDurationCached 17.2s 14.6s N/A
nodeModulesSize 393 MB 393 MB ✓
nextStartRea..uration (ms) 472ms 471ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
5306-HASH.js gzip 55 kB 55 kB N/A
8276.HASH.js gzip 169 B 168 B N/A
8377-HASH.js gzip 5.46 kB 5.47 kB N/A
bccd1874-HASH.js gzip 53 kB 53 kB N/A
framework-HASH.js gzip 57.5 kB 57.5 kB N/A
main-app-HASH.js gzip 245 B 245 B ✓
main-HASH.js gzip 34.7 kB 34.7 kB N/A
webpack-HASH.js gzip 1.71 kB 1.71 kB N/A
Overall change 245 B 245 B ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB ✓
Overall change 39.4 kB 39.4 kB ✓
Client Pages
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
_app-HASH.js gzip 193 B 193 B ✓
_error-HASH.js gzip 193 B 193 B ✓
amp-HASH.js gzip 512 B 510 B N/A
css-HASH.js gzip 343 B 342 B N/A
dynamic-HASH.js gzip 1.84 kB 1.84 kB ✓
edge-ssr-HASH.js gzip 265 B 265 B ✓
head-HASH.js gzip 363 B 362 B N/A
hooks-HASH.js gzip 393 B 392 B N/A
image-HASH.js gzip 4.59 kB 4.59 kB N/A
index-HASH.js gzip 268 B 268 B ✓
link-HASH.js gzip 2.35 kB 2.35 kB N/A
routerDirect..HASH.js gzip 328 B 328 B ✓
script-HASH.js gzip 397 B 397 B ✓
withRouter-HASH.js gzip 323 B 326 B N/A
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Overall change 3.59 kB 3.59 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
_buildManifest.js gzip 748 B 748 B ✓
Overall change 748 B 748 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
index.html gzip 524 B 522 B N/A
link.html gzip 539 B 537 B N/A
withRouter.html gzip 520 B 519 B N/A
Overall change 0 B 0 B ✓
Edge SSR bundle Size
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
edge-ssr.js gzip 130 kB 130 kB N/A
page.js gzip 211 kB 211 kB N/A
Overall change 0 B 0 B ✓
Middleware size
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
middleware-b..fest.js gzip 676 B 668 B N/A
middleware-r..fest.js gzip 155 B 156 B N/A
middleware.js gzip 31.3 kB 31.3 kB N/A
edge-runtime..pack.js gzip 844 B 844 B ✓
Overall change 844 B 844 B ✓
Next Runtimes
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
app-page-exp...dev.js gzip 394 kB 394 kB ✓
app-page-exp..prod.js gzip 132 kB 132 kB ✓
app-page-tur..prod.js gzip 145 kB 145 kB ✓
app-page-tur..prod.js gzip 141 kB 141 kB ✓
app-page.run...dev.js gzip 382 kB 382 kB ✓
app-page.run..prod.js gzip 129 kB 129 kB ✓
app-route-ex...dev.js gzip 39.3 kB 39.3 kB ✓
app-route-ex..prod.js gzip 25.7 kB 25.7 kB ✓
app-route-tu..prod.js gzip 25.7 kB 25.7 kB ✓
app-route-tu..prod.js gzip 25.4 kB 25.4 kB ✓
app-route.ru...dev.js gzip 40.9 kB 40.9 kB ✓
app-route.ru..prod.js gzip 25.4 kB 25.4 kB ✓
dist_client_...dev.js gzip 356 B 356 B ✓
dist_client_...dev.js gzip 349 B 349 B ✓
pages-api-tu..prod.js gzip 9.69 kB 9.69 kB ✓
pages-api.ru...dev.js gzip 11.8 kB 11.8 kB ✓
pages-api.ru..prod.js gzip 9.69 kB 9.69 kB ✓
pages-turbo...prod.js gzip 22 kB 22 kB ✓
pages.runtim...dev.js gzip 31.6 kB 31.6 kB ✓
pages.runtim..prod.js gzip 22 kB 22 kB ✓
server.runti..prod.js gzip 61.1 kB 61.1 kB ✓
Overall change 1.67 MB 1.67 MB ✓
build cache Overall increase ⚠️
vercel/next.js canary vercel/next.js sokra/scale-down-pool Change
0.pack gzip 2.11 MB 2.11 MB ⚠️ +704 B
index.pack gzip 76.5 kB 76.5 kB N/A
Overall change 2.11 MB 2.11 MB ⚠️ +704 B
Diff details
Diff for main-HASH.js

Diff too large to display

Commit: be1b4af

mischnic
mischnic reviewed Feb 11, 2025
View reviewed changes
turbopack/crates/turbopack-node/src/pool.rs Outdated
@@ -705,6 +706,9 @@ enum AcquiredPermits {
},
}

static ACTIVE_POOLS: Lazy<Mutex<Vec<Arc<Mutex<BinaryHeap<NodeJsPoolProcess>>>>>> =
Copy link
Contributor

@mischnic mischnic Feb 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this additional Vector needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment:

type IdleProcessesList = Arc<Mutex<BinaryHeap<NodeJsPoolProcess>>>;

/// All non-empty `IdleProcessesList`s of the whole application.
/// This is used to scale down processes globally.
static ACTIVE_POOLS: Lazy<Mutex<Vec<IdleProcessesList>>> = Lazy::new(Default::default);

@sokra sokra changed the base branch from sokra/improve-process-selection to graphite-base/75896 February 11, 2025 09:42
@sokra sokra force-pushed the sokra/scale-down-pool branch from 28a6df0 to 96a0518 Compare February 11, 2025 09:43
@sokra sokra force-pushed the graphite-base/75896 branch from f16aa7b to 10e4a0a Compare February 11, 2025 09:43
@sokra sokra changed the base branch from graphite-base/75896 to canary February 11, 2025 09:43
@sokra sokra force-pushed the sokra/scale-down-pool branch from 96a0518 to 23a86e0 Compare February 11, 2025 09:43
@sokra sokra force-pushed the sokra/scale-down-pool branch from cef8c26 to be1b4af Compare February 12, 2025 10:24
@sokra sokra merged commit bb3773d into canary Feb 12, 2025
131 checks passed
@sokra Graphite App
Copy link
Member Author

sokra commented Feb 12, 2025

Merge activity

  • Feb 12, 6:09 AM EST: A user merged this pull request with Graphite.

@sokra sokra deleted the sokra/scale-down-pool branch February 12, 2025 11:09
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 26, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@mischnic mischnic mischnic approved these changes

Assignees
No one assigned
Labels
created-by: Turbopack team PRs by the Turbopack team. locked Turbopack Related to Turbopack with Next.js.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@sokra @ijjk @mischnic