Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Turbopack] only invalidate fs writes with a different content #75936

Merged
merged 9 commits into from
Feb 14, 2025

Conversation

sokra
Copy link
Member

@sokra sokra commented Feb 12, 2025

What?

There are cases where two write tasks write to the same file. This can happen e. g. when two different image source files with the same content are used and end up with the same content hashed filename.

But since dependency tracking is disabled for one off builds, this results in an invalidation of the first write task when the second write task effect is executed. And an invalidation will panic in dependency-tracking-disabled mode.

This change only invalidates other write tasks when they write a different content and keeps writes with the same content intact.

This should avoid the panic as no task is invalidated.

Closes PACK-3965

@ijjk ijjk added the created-by: Turbopack team PRs by the Turbopack team. label Feb 12, 2025
@sokra sokra changed the title only invalidate fs writes with a different content [Turbopack] only invalidate fs writes with a different content Feb 12, 2025
Copy link
Member Author

sokra commented Feb 12, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@sokra sokra requested a review from mischnic February 12, 2025 01:43
@ijjk
Copy link
Member

ijjk commented Feb 12, 2025

Tests Passed

@ijjk
Copy link
Member

ijjk commented Feb 12, 2025

Stats from current PR

Default Build (Increase detected ⚠️)
General
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
buildDuration 16.3s 14.8s N/A
buildDurationCached 14s 11.6s N/A
nodeModulesSize 393 MB 393 MB
nextStartRea..uration (ms) 407ms 389ms N/A
Client Bundles (main, webpack)
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
5271-HASH.js gzip 55.4 kB 55.4 kB N/A
6228c9d4-HASH.js gzip 56.9 kB 56.9 kB N/A
7048.HASH.js gzip 168 B 168 B
8377-HASH.js gzip 5.46 kB 5.46 kB N/A
framework-HASH.js gzip 57.5 kB 57.5 kB N/A
main-app-HASH.js gzip 244 B 246 B N/A
main-HASH.js gzip 34.9 kB 34.9 kB N/A
webpack-HASH.js gzip 1.71 kB 1.71 kB
Overall change 1.88 kB 1.88 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
polyfills-HASH.js gzip 39.4 kB 39.4 kB
Overall change 39.4 kB 39.4 kB
Client Pages
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
_app-HASH.js gzip 194 B 194 B
_error-HASH.js gzip 193 B 192 B N/A
amp-HASH.js gzip 513 B 511 B N/A
css-HASH.js gzip 342 B 342 B
dynamic-HASH.js gzip 1.84 kB 1.84 kB N/A
edge-ssr-HASH.js gzip 265 B 264 B N/A
head-HASH.js gzip 363 B 360 B N/A
hooks-HASH.js gzip 393 B 390 B N/A
image-HASH.js gzip 4.59 kB 4.59 kB N/A
index-HASH.js gzip 268 B 266 B N/A
link-HASH.js gzip 2.35 kB 2.35 kB
routerDirect..HASH.js gzip 328 B 326 B N/A
script-HASH.js gzip 397 B 397 B
withRouter-HASH.js gzip 325 B 325 B
1afbb74e6ecf..834.css gzip 106 B 106 B
Overall change 3.72 kB 3.72 kB
Client Build Manifests
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
_buildManifest.js gzip 749 B 747 B N/A
Overall change 0 B 0 B
Rendered Page Sizes
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
index.html gzip 523 B 522 B N/A
link.html gzip 539 B 536 B N/A
withRouter.html gzip 520 B 518 B N/A
Overall change 0 B 0 B
Edge SSR bundle Size
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
edge-ssr.js gzip 130 kB 130 kB N/A
page.js gzip 214 kB 214 kB N/A
Overall change 0 B 0 B
Middleware size
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
middleware-b..fest.js gzip 677 B 673 B N/A
middleware-r..fest.js gzip 155 B 156 B N/A
middleware.js gzip 31.6 kB 31.6 kB N/A
edge-runtime..pack.js gzip 844 B 844 B
Overall change 844 B 844 B
Next Runtimes
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
app-page-exp...dev.js gzip 395 kB 395 kB
app-page-exp..prod.js gzip 133 kB 133 kB
app-page-tur..prod.js gzip 145 kB 145 kB
app-page-tur..prod.js gzip 141 kB 141 kB
app-page.run...dev.js gzip 382 kB 382 kB
app-page.run..prod.js gzip 129 kB 129 kB
app-route-ex...dev.js gzip 39.4 kB 39.4 kB
app-route-ex..prod.js gzip 25.7 kB 25.7 kB
app-route-tu..prod.js gzip 25.7 kB 25.7 kB
app-route-tu..prod.js gzip 25.5 kB 25.5 kB
app-route.ru...dev.js gzip 41 kB 41 kB
app-route.ru..prod.js gzip 25.5 kB 25.5 kB
dist_client_...dev.js gzip 356 B 356 B
dist_client_...dev.js gzip 349 B 349 B
pages-api-tu..prod.js gzip 9.72 kB 9.72 kB
pages-api.ru...dev.js gzip 11.8 kB 11.8 kB
pages-api.ru..prod.js gzip 9.72 kB 9.72 kB
pages-turbo...prod.js gzip 22 kB 22 kB
pages.runtim...dev.js gzip 31.6 kB 31.6 kB
pages.runtim..prod.js gzip 22 kB 22 kB
server.runti..prod.js gzip 61.2 kB 61.2 kB
Overall change 1.68 MB 1.68 MB
build cache Overall increase ⚠️
vercel/next.js canary vercel/next.js sokra/disable-fs-invalidation Change
0.pack gzip 2.12 MB 2.12 MB ⚠️ +3.79 kB
index.pack gzip 77.2 kB 77.2 kB N/A
Overall change 2.12 MB 2.12 MB ⚠️ +3.79 kB
Diff details
Diff for 5271-HASH.js

Diff too large to display

Diff for main-HASH.js

Diff too large to display

Commit: 3a45e18

@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from d3df62c to a1146c8 Compare February 12, 2025 10:35
@mischnic

This comment was marked as resolved.

@sokra sokra marked this pull request as ready for review February 12, 2025 15:28
@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from a1146c8 to 85ca1e6 Compare February 13, 2025 15:21
@sokra sokra changed the base branch from canary to sokra/disable-fs-invalidation-preparation_split February 13, 2025 15:21
@sokra sokra force-pushed the sokra/disable-fs-invalidation-preparation_split branch from 84fd11a to c003c90 Compare February 14, 2025 05:05
@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from 85ca1e6 to 5d4a9a0 Compare February 14, 2025 05:05
@sokra sokra added the CI Bypass Graphite Optimization Ignore Graphite CI optimizations, run the full CI suite. https://graphite.dev/docs/stacking-and-ci label Feb 14, 2025 — with Graphite App
@sokra sokra force-pushed the sokra/disable-fs-invalidation-preparation_split branch from c003c90 to 94c8c79 Compare February 14, 2025 05:26
@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from 5d4a9a0 to fa61c93 Compare February 14, 2025 05:26
@sokra sokra changed the base branch from sokra/disable-fs-invalidation-preparation_split to sokra/fix-active-counting February 14, 2025 07:50
@sokra sokra changed the base branch from sokra/fix-active-counting to graphite-base/75936 February 14, 2025 08:35
@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from fa61c93 to 0273c0a Compare February 14, 2025 08:36
@sokra sokra force-pushed the graphite-base/75936 branch from 37a34e3 to f9fcae1 Compare February 14, 2025 08:36
@sokra sokra changed the base branch from graphite-base/75936 to canary February 14, 2025 08:36
@sokra sokra force-pushed the sokra/disable-fs-invalidation branch from 0273c0a to 3a45e18 Compare February 14, 2025 08:36
@sokra sokra merged commit fa2ccaf into canary Feb 14, 2025
132 checks passed
Copy link
Member Author

sokra commented Feb 14, 2025

Merge activity

  • Feb 14, 4:04 AM EST: A user merged this pull request with Graphite.

@sokra sokra deleted the sokra/disable-fs-invalidation branch February 14, 2025 09:04
devjiwonchoi pushed a commit that referenced this pull request Feb 14, 2025
### What?

There are cases where two write tasks write to the same file. This can happen e. g. when two different image source files with the same content are used and end up with the same content hashed filename.

But since dependency tracking is disabled for one off builds, this results in an invalidation of the first write task when the second write task effect is executed. And an invalidation will panic in dependency-tracking-disabled mode.

This change only invalidates other write tasks when they write a different content and keeps writes with the same content intact.

This should avoid the panic as no task is invalidated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Bypass Graphite Optimization Ignore Graphite CI optimizations, run the full CI suite. https://graphite.dev/docs/stacking-and-ci created-by: Turbopack team PRs by the Turbopack team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants