Random build failure due to race condition in handle-externals

[ashi009]

Link to the code that reproduces this issue

https://github.com/ashi009/nextjs-wasm

To Reproduce

next.js/packages/next/src/build/handle-externals.ts

Lines 327 to 346 in a71a764

	if (transpiledPackages && !resolvedExternalPackageDirs) {
	resolvedExternalPackageDirs = new Map()
	// We need to resolve all the external package dirs initially.
	for (const pkg of transpiledPackages) {
	const pkgRes = await resolveExternal(
	dir,
	config.experimental.esmExternals,
	context,
	pkg + '/package.json',
	isEsmRequested,
	optOutBundlingPackages,
	getResolve,
	isLocal ? resolveNextExternal : undefined
	)
	if (pkgRes.res) {
	resolvedExternalPackageDirs.set(pkg, path.dirname(pkgRes.res))
	}
	}
	}

At line 327, the resolvedExternalPackageDirs is initialized at the first invocation of async handleExternals(). However, the content of resolvedExternalPackageDirs will not be populated until line 342.

next.js/packages/next/src/build/handle-externals.ts

Lines 347 to 356 in a71a764

	const resolvedBundlingOptOutRes = resolveBundlingOptOutPackages({
	resolvedRes: res,
	config,
	resolvedExternalPackageDirs,
	isAppLayer,
	externalType,
	isOptOutBundling,
	request,
	transpiledPackages,
	})

Another invocation of async handleExternals() will skip initializing resolvedExternalPackageDirs, as it is already a Map instance. So, it will continue to line 347 and pass resolvedExternalPackageDirs to resolveBundlingOptOutPackages.

RACE: Depending on the execution of the first invocation, the content in resolvedExternalPackageDirs may or may not be populated. The resolvedBundlingOptOutRes is undetermined.

Current vs. Expected behavior

current:

external dependencies may not be resolved when calling resolveBundlingOptOutPackages. Which causes random build failures, like rustwasm/wasm-pack#1402.

expected:

external dependencies must be resolved first.

Provide environment information

Operating System:
  Platform: darwin
  Arch: arm64
  Version: Darwin Kernel Version 24.1.0: Thu Oct 10 21:03:11 PDT 2024; root:xnu-11215.41.3~2/RELEASE_ARM64_T6020
  Available memory (MB): 32768
  Available CPU cores: 12
Binaries:
  Node: 22.11.0
  npm: 10.9.0
  Yarn: 1.22.22
  pnpm: N/A
Relevant Packages:
  next: 15.1.6 // There is a newer version (15.1.7) available, upgrade recommended! 
  eslint-config-next: 15.1.5
  react: 18.3.1
  react-dom: 18.3.1
  typescript: 5.7.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Webpack

Which stage(s) are affected? (Select all that apply)

next build (local), next dev (local), next start (local)

Additional context

No response