Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Create synchronous snapshots of recursive AvailableModuleInfo struct #74894

Merged
merged 2 commits into from
Jan 15, 2025
Merged

Create synchronous snapshots of recursive AvailableModuleInfo struct #74894

merged 2 commits into from
Jan 15, 2025

Conversation

wbinnssmith
Copy link
Member

@wbinnssmith wbinnssmith commented Jan 15, 2025 •
edited by padmaia
Loading

This:
- Allows for synchronous lookup of a module’s availability information without async resolving Vcs
- Creates a single structure that combines the entire availability information hierarchy into one. While not strictly necessary, without Vc caching, the hierarchy of structs only provided overhead without advantage

Closes PACK-3730

@wbinnssmith
Create synchronous snapshots of recursive AvailableModuleInfo struct
f42e4af 
This:
	- Allows for synchronous lookup of a module’s availability information without async resolving Vcs
	- Creates a single structure that combines the entire availability information hierarchy into one. While not strictly necessary, without Vc caching, the hierarchy of structs only provided overhead without advantage
@wbinnssmith wbinnssmith requested review from sokra and mischnic January 15, 2025 00:44
@ijjk ijjk added the created-by: Turbopack team PRs by the Turbopack team. label Jan 15, 2025
@ijjk
Copy link
Member

ijjk commented Jan 15, 2025 •
edited
Loading

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
buildDuration 16.5s 15s N/A
buildDurationCached 14.1s 11.8s N/A
nodeModulesSize 418 MB 418 MB ✓
nextStartRea..uration (ms) 420ms 419ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
5306-HASH.js gzip 53.9 kB 53.9 kB N/A
8276.HASH.js gzip 169 B 168 B N/A
8377-HASH.js gzip 5.44 kB 5.44 kB N/A
bccd1874-HASH.js gzip 52.9 kB 52.9 kB N/A
framework-HASH.js gzip 57.5 kB 57.5 kB N/A
main-app-HASH.js gzip 240 B 243 B N/A
main-HASH.js gzip 34.2 kB 34.2 kB N/A
webpack-HASH.js gzip 1.71 kB 1.71 kB N/A
Overall change 0 B 0 B ✓
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB ✓
Overall change 39.4 kB 39.4 kB ✓
Client Pages
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
_app-HASH.js gzip 193 B 193 B ✓
_error-HASH.js gzip 193 B 193 B ✓
amp-HASH.js gzip 512 B 510 B N/A
css-HASH.js gzip 343 B 342 B N/A
dynamic-HASH.js gzip 1.84 kB 1.84 kB ✓
edge-ssr-HASH.js gzip 265 B 265 B ✓
head-HASH.js gzip 363 B 362 B N/A
hooks-HASH.js gzip 393 B 392 B N/A
image-HASH.js gzip 4.57 kB 4.57 kB N/A
index-HASH.js gzip 268 B 268 B ✓
link-HASH.js gzip 2.35 kB 2.34 kB N/A
routerDirect..HASH.js gzip 328 B 328 B ✓
script-HASH.js gzip 397 B 397 B ✓
withRouter-HASH.js gzip 323 B 326 B N/A
1afbb74e6ecf..834.css gzip 106 B 106 B ✓
Overall change 3.59 kB 3.59 kB ✓
Client Build Manifests
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
_buildManifest.js gzip 749 B 747 B N/A
Overall change 0 B 0 B ✓
Rendered Page Sizes
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
index.html gzip 524 B 524 B ✓
link.html gzip 539 B 538 B N/A
withRouter.html gzip 520 B 520 B ✓
Overall change 1.04 kB 1.04 kB ✓
Edge SSR bundle Size
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
edge-ssr.js gzip 129 kB 129 kB N/A
page.js gzip 208 kB 208 kB N/A
Overall change 0 B 0 B ✓
Middleware size
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
middleware-b..fest.js gzip 668 B 667 B N/A
middleware-r..fest.js gzip 155 B 156 B N/A
middleware.js gzip 31.3 kB 31.3 kB N/A
edge-runtime..pack.js gzip 844 B 844 B ✓
Overall change 844 B 844 B ✓
Next Runtimes
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
274-experime...dev.js gzip 322 B 322 B ✓
274.runtime.dev.js gzip 314 B 314 B ✓
app-page-exp...dev.js gzip 374 kB 374 kB ✓
app-page-exp..prod.js gzip 130 kB 130 kB ✓
app-page-tur..prod.js gzip 143 kB 143 kB ✓
app-page-tur..prod.js gzip 139 kB 139 kB ✓
app-page.run...dev.js gzip 362 kB 362 kB ✓
app-page.run..prod.js gzip 126 kB 126 kB ✓
app-route-ex...dev.js gzip 37.6 kB 37.6 kB ✓
app-route-ex..prod.js gzip 25.6 kB 25.6 kB ✓
app-route-tu..prod.js gzip 25.6 kB 25.6 kB ✓
app-route-tu..prod.js gzip 25.4 kB 25.4 kB ✓
app-route.ru...dev.js gzip 39.2 kB 39.2 kB ✓
app-route.ru..prod.js gzip 25.4 kB 25.4 kB ✓
pages-api-tu..prod.js gzip 9.69 kB 9.69 kB ✓
pages-api.ru...dev.js gzip 11.6 kB 11.6 kB ✓
pages-api.ru..prod.js gzip 9.68 kB 9.68 kB ✓
pages-turbo...prod.js gzip 21.7 kB 21.7 kB ✓
pages.runtim...dev.js gzip 27.5 kB 27.5 kB ✓
pages.runtim..prod.js gzip 21.7 kB 21.7 kB ✓
server.runti..prod.js gzip 916 kB 916 kB ✓
Overall change 2.47 MB 2.47 MB ✓
build cache Overall increase ⚠️
vercel/next.js canary vercel/next.js wbinnssmith/available-snapshot Change
0.pack gzip 2.1 MB 2.09 MB N/A
index.pack gzip 74.6 kB 74.8 kB ⚠️ +183 B
Overall change 74.6 kB 74.8 kB ⚠️ +183 B
Diff details
Diff for main-HASH.js

Diff too large to display

Commit: ae20455

bgw
bgw approved these changes Jan 15, 2025
View reviewed changes
Copy link
Member

@bgw bgw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Since AvailableModulesInfo is tiny and Copy, this should be pretty cheap.

turbopack/crates/turbopack-core/src/chunk/available_modules.rs Outdated
pub async fn into_snapshot(&self) -> Result<Vc<AvailableModulesSnapshot>> {
let mut combined = FxIndexMap::default();
if let Some(parent) = self.parent {
for (module, info) in &parent.into_snapshot().await?.modules {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to recursively call a turbo_tasks::function here (makes sense only if we want to cache intermediate results), or would to be better to add an _inner helper for the recursion?

sokra
sokra reviewed Jan 15, 2025
View reviewed changes
turbopack/crates/turbopack-core/src/chunk/available_modules.rs Outdated
Comment on lines 127 to 129
pub struct AvailableModulesSnapshot {
modules: FxIndexMap<ResolvedVc<Box<dyn ChunkableModule>>, AvailableModulesInfo>,
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would have gone for:

struct AvailableModulesSnapshot {
    parent: Option<ReadRef<AvailableModulesSnapshot>>,
    modules: ReadRef<AvailableModuleInfoMap>,
}

which avoids cloning and merging the maps, but we can change that later...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

update it

@ijjk
Copy link
Member

ijjk commented Jan 15, 2025 •
edited
Loading

Tests Passed

@sokra sokra merged commit 8703c05 into canary Jan 15, 2025
130 checks passed
@sokra Graphite App
Copy link
Member

sokra commented Jan 15, 2025

Merge activity

  • Jan 15, 3:36 AM EST: A user merged this pull request with Graphite.

@sokra sokra deleted the wbinnssmith/available-snapshot branch January 15, 2025 08:36
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 29, 2025
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@sokra sokra sokra approved these changes

@bgw bgw bgw approved these changes

@mischnic mischnic Awaiting requested review from mischnic

Assignees
No one assigned
Labels
created-by: Turbopack team PRs by the Turbopack team. locked
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@wbinnssmith @ijjk @sokra @bgw