Replace dep jobserver with jobslot

[NobodyXu]

Signed-off-by: Jiahao XU Jiahao_XU@outlook.com

This PR replaces dep jobserver with a fork jobslot.

Advantages over jobserver?

• jobslot contains bug fix for Client::configure is unsafe
• jobslot removed use of signal handling in the helper thread on unix
• jobslot uses winapi on windows instead of manually declaring bindings (some of the bindings seem to be wrong)
• jobslot uses getrandom on windows instead of making homebrew one using raw windows api

[sylvestre]

I am quite reluctant to move to a fork. Especially when the original maintainer is the amazing @alexcrichton

You also don't share enough information about the performance improvement.

[sylvestre]

If rust-lang/cargo#11066 is merged, maybe we will consider merging this PR but we probably won't until this.

[NobodyXu]

You also don't share enough information about the performance improvement.

On unix, jobserver::Client::configure uses std::os::unix::process::CommandExt::pre_exec to register a callback which removes CLOEXEC flags set on the jobserver fds (essentially just pipes).

Since the callback can do whatever it wants, Command::spawn cannot use vfork + exec, thus it will take longer to spawn the new process and also might cause memory overcommit.

Both sccache and sccache-dist are quite large (roughly 20MB), so it is affected by this.
It also depends on the memory taken up by them at runtime, so the more virtual memory it takes, the slower fork + exec.

Though I did not run any benchmarks so I can't say for sure how much of a speedup it would be.

[sylvestre]

Without data, I am even less convinced to merge it than I was before :)

[NobodyXu]

Without data, I am even less convinced to merge it than I was before :)

Well, the main motivation for me to fork it is to fix the bug plus refactoring, the better performance is just a side-effect.
But anyway, the fork at least should not cause any regression in the performance of sccache.

Other improvements in jobslot includes:

• Remove use of signal handling in the helper thread on unix to make it more robust
• Use winapi on windows instead of manually declaring bindings (some of the bindings seem to be wrong)
• Use getrandom on windows instead of making homebrew one using raw windows api

[sylvestre]

Sure, I understand that you think it is the right approach but I am not going to switch sccache to a 2 days-old new crate when I have no idea how long it is going to be maintained (while Alex has been maintaining jobserver for 5 years).

[NobodyXu]

Sure, I understand that you think it is the right approach but I am not going to switch sccache to a 2 days-old new crate when I have no idea how long it is going to be maintained (while Alex has been maintaining jobserver for 5 years).

Yes I fully understand your decision to not switch right now.

Though jobslot has slightly less bus factor (not offensive to anyone) as it is put in an organization cargo-bins and another person in the org also has access to crates.io

Honestly I am astonished how many well-known crates in rust do not receive enough maintainance.
zip, tar, jobserver, etc.
Perhaps it is a good time to put them in rust-lang...

[sylvestre]

Honestly I am astonished how many well-known crates in rust do not receive enough maintainance.

This is the case of opensource as a whole...

[NobodyXu]

This is the case of opensource as a whole...

sad but indeed true.

[sylvestre]

closing for now