Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve hermeticity #180

Merged
merged 1 commit into from
Mar 13, 2018
Merged

Improve hermeticity #180

merged 1 commit into from
Mar 13, 2018

Conversation

mrkkrp
Copy link
Member

@mrkkrp mrkkrp commented Mar 12, 2018

Close #143, close #153 (I think this PR provides a better solution to essentially the same problem so the older PR may be discarded).

This PR only makes the rules hermetic with respect to binaries, a separate PR for hermeticity with respect to .so files may be needed, although it's probably of lower priority.

To understand the solution let's recall the ways in which hermeticity may be violated:

  • PATH env variable is provided which points to a directory which contains other binaries in addition to the binary of interest (which is virtually always the case).
  • ctx.actions.run_shell is used. This automatically uses environment of host OS.
  • ctx.actions.run with use_default_shell_env is used, again we touch environment of host OS this way.

It may seem that just specifying things as inputs may be sufficient to resolve the hermeticity problem, but it isn't the case. There is a difference between Bazel's "domain of control" and outside world. Bazel can control visibility of artifacts it creates, i.e. Files, because one can put Files in inputs argument of run and such. ctx.host_fragments.cpp on the other hand provides a bunch of strings with locations of system binaries such as ar and gcc (makes sense, Files cannot have absolute paths in Bazel). This means that Bazel cannot control visibility of ar, gcc, ln, etc. To it they are just strings and not artifacts produced by the build system. This is not very handly from the hermeticity point of view.

The solution is to merge CPP fragments, GHC binaries, and binutils of interest and create symlinks to all of them in a special directory which won't contain anything else besides these symlinks to binaries. What we get this way:

  • No extra binaries are visible. Even if we have to provide PATH, we put there just path to our directory with symlinks, which contains nothing extra, and visibility of what it contains is controlled by Bazel in addition to that.
  • We remove all calls to ctx.actions.run_shell, redundant PATH environment variables, we don't use use_default_shell_env.
  • All binaries always come from tools(ctx).bin_name, e.g. tools(ctx).ghc and are used only as executable argument and in inputs argument to make them visible.

So that's it I guess, should be quite hermetic.

@mrkkrp mrkkrp requested review from Fuuzetsu and mboes March 12, 2018 16:41
@mrkkrp mrkkrp force-pushed the improve-hermeticity branch from 77e370d to 63dd1ad Compare March 12, 2018 16:46
@mrkkrp mrkkrp force-pushed the improve-hermeticity branch from 63dd1ad to 48c0b07 Compare March 13, 2018 12:04
)
set.mutable_insert(symlinks, symlink)

tools_struct_args = {tool.basename.replace("-", "_"): tool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this replace needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because you have binaries named like ghc-pkg and ghc-pkg is not a valid attribute name, so it must be ghc_pkg.

@mrkkrp mrkkrp merged commit fa35c53 into master Mar 13, 2018
@mrkkrp mrkkrp deleted the improve-hermeticity branch March 13, 2018 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rule inputs should be declared more precisely
2 participants