Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building new architecture sources on Windows (NDK Bump) #33627

Closed
wants to merge 2 commits into from

Conversation

mganandraj
Copy link
Contributor

On Windows there are limits on number of character in file paths and in command lines
Ref: https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md#Path-Length-Limits

The new architecutre source tree is too deep, and the object file naming conventions in NDK makes the matters worse, by producing incredibly long file paths.

Windows can support long file paths if configured through registry or by prefixing all file paths with a special character sequence. LLVM tools can work with long paths if user configured it but NDK23- uses binutils tools which don't.
This PR bumps NDK when building on Windows

Even with NDKr23, we are hitting the command line limits of 8191 characters !
NDK_APP_LOCAL_COMMANDS should come to rescue, but gradle android plugin has hardcoded the flag to false
Ref: Line#165 in https://android.googlesource.com/platform/tools/base/+/gradle_2.3.0/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/NdkBuildExternalNativeJsonGenerator.java#165

Hence, we can't use the default build directory. But, we can support much longer paths than without the NDK bump.

Changelog:
[Android] [Fixed] - Fix for building new architecture sources on Windows

On Windows there are limits on number of character in file paths and in command lines
Ref: https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md#Path-Length-Limits

The new architecutre source tree is too deep, and the object file naming conventions in NDK makes the matters worse, by producing incredibly long file paths.

Windows can support long file paths if configured through registry or by prefixing all file paths with a special character sequence. LLVM tools can work with long paths if user configured it but NDK23- uses binutils tools which don't.
This PR bumps NDK when building on Windows

Even with NDKr23, we are hitting the command line limits of 8191 characters !
NDK_APP_LOCAL_COMMANDS should come to rescue, but gradle android plugin has hardcoded the flag to false
Ref: Line#165 in https://android.googlesource.com/platform/tools/base/+/gradle_2.3.0/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/NdkBuildExternalNativeJsonGenerator.java#165

Hence, we can't use the default build directory. But, we can support much longer paths than without the NDK bump.

Summary

Make sure react native apps can be build on windows machines.

Changelog

On Windows there are limits on number of character in file paths and in command lines
Ref: https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md#Path-Length-Limits

The new architecutre source tree is too deep, and the object file naming conventions in NDK makes the matters worse, by producing incredibly long file paths.

Windows can support long file paths if configured through registry or by prefixing all file paths with a special character sequence. LLVM tools can work with long paths if user configured it but NDK23- uses binutils tools which don't.
This PR bumps NDK when building on Windows

Even with NDKr23, we are hitting the command line limits of 8191 characters !
NDK_APP_LOCAL_COMMANDS should come to rescue, but gradle android plugin has hardcoded the flag to false
Ref: Line#165 in https://android.googlesource.com/platform/tools/base/+/gradle_2.3.0/build-system/gradle-core/src/main/java/com/android/build/gradle/tasks/NdkBuildExternalNativeJsonGenerator.java#165

Hence, we can't use the default build directory. But, we can support much longer paths than without the NDK bump.

[CATEGORY] [TYPE] - Message

Test Plan

On Windows there are limits on number of character in file paths and in command lines
Ref: https://android.googlesource.com/platform/ndk/+/master/docs/BuildSystemMaintainers.md#Path-Length-Limits
NDK allows circumventing command line limits using response(RSP) files as inputs using NDK_APP_SHORT_COMMANDS flag.

Windows can support long file paths if configured through registry or by prefixing all file paths with a special character sequence
The latter requires changes in NDK. And there are tools in NDK (AR) which is not able to handle long paths (>256) even after setting the registry key.
The new architecutre source tree is too deep, and the object file naming conventions in NDK makes the matters worse, by producing incredibly long file paths.
Other solutions such as symlinking source code etc. didn't work as expected, and makes the build scripts complicated and hard to manage.
This change temporarily works around the issue by placing the temporary build outputs as short a path as possible within the project path.

Changelog:
[Android] [Fixed] - Fix for building new architecture sources on Windows
@facebook-github-bot facebook-github-bot added CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Microsoft Partner: Microsoft Partner labels Apr 13, 2022
@facebook-github-bot facebook-github-bot added the Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team. label Apr 13, 2022
@pull-bot
Copy link

Fails
🚫

node` failed.

Log

Error:  RequestError [HttpError]: Must have admin rights to Repository.
    at /root/react-native/bots/node_modules/@octokit/request/dist-node/index.js:86:21
    at processTicksAndRejections (internal/process/task_queues.js:95:5) {
  status: 403,
  response: {
    url: 'https://api.github.com/repos/facebook/react-native/issues/33627/labels',
    status: 403,
    headers: {
      'access-control-allow-origin': '*',
      'access-control-expose-headers': 'ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Used, X-RateLimit-Resource, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type, X-GitHub-SSO, X-GitHub-Request-Id, Deprecation, Sunset',
      connection: 'close',
      'content-encoding': 'gzip',
      'content-security-policy': "default-src 'none'",
      'content-type': 'application/json; charset=utf-8',
      date: 'Wed, 13 Apr 2022 06:30:51 GMT',
      'referrer-policy': 'origin-when-cross-origin, strict-origin-when-cross-origin',
      server: 'github.com',
      'strict-transport-security': 'max-age=31536000; includeSubdomains; preload',
      'transfer-encoding': 'chunked',
      vary: 'Accept-Encoding, Accept, X-Requested-With',
      'x-accepted-oauth-scopes': '',
      'x-content-type-options': 'nosniff',
      'x-frame-options': 'deny',
      'x-github-media-type': 'github.v3; format=json',
      'x-github-request-id': 'BDBE:1EE6:DFD1BF:194A86F:62566E1B',
      'x-oauth-scopes': 'public_repo',
      'x-ratelimit-limit': '5000',
      'x-ratelimit-remaining': '4992',
      'x-ratelimit-reset': '1649835049',
      'x-ratelimit-resource': 'core',
      'x-ratelimit-used': '8',
      'x-xss-protection': '0'
    },
    data: {
      message: 'Must have admin rights to Repository.',
      documentation_url: 'https://docs.github.com/rest/reference/issues#add-labels-to-an-issue'
    }
  },
  request: {
    method: 'POST',
    url: 'https://api.github.com/repos/facebook/react-native/issues/33627/labels',
    headers: {
      accept: 'application/vnd.github.v3+json',
      'user-agent': 'octokit.js/16.43.2 Node.js/14.18.1 (Linux 5.13; x64)',
      authorization: 'token [REDACTED]',
      'content-type': 'application/json; charset=utf-8'
    },
    body: '{"labels":["Pick Request"]}',
    request: { hook: [Function: bound bound register] }
  }
}
danger-results://tmp/danger-results.json

Generated by 🚫 dangerJS against bea84f6

@mganandraj
Copy link
Contributor Author

@cortinico This change is adding the ndk upgrade above the other PR .. This allows us to be more liberal on the host application path length ..

@react-native-bot react-native-bot added Bug Platform: Android Android applications. labels Apr 13, 2022
@analysis-bot
Copy link

Platform Engine Arch Size (bytes) Diff
ios - universal n/a --

Base commit: 86f8d0b
Branch: main

@cortinico
Copy link
Contributor

@cortinico This change is adding the ndk upgrade above the other PR .. This allows us to be more liberal on the host application path length ..

Thanks for doing this @mganandraj

We ended up picking this other two PRs:

Thanks again for your testing and support in solving this and unblocking 0.68.1 🙏

@cortinico cortinico closed this Apr 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. p: Microsoft Partner: Microsoft Partner Platform: Android Android applications. Shared with Meta Applied via automation to indicate that an Issue or Pull Request has been shared with the team.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants