Improve usage of file locks to allow better parallel use of nargo

[TomAFrench]

We've currently got a bit of a sledgehammer approach to file locking where ony a single instance of nargo can access a workspace at any given time. This is causing some pain for users of nargo who want to build a workspace in parallel and kick off processing of the artifacts without waiting for the full workspace to build.

We should then restrict the file locking to cover only the directories which multiple nargo instances may clash on. These are the git dependency cache and the target directory afaik. I think a sensible plan is then:

Tasks

Preview Give feedback

1. fix: lock git dependencies folder when resolving workspace #7327
   +0
2. Add file lock on target directory to prevent multiple commands using the same output dir concurrently. #7542
   +0
3. Add CLI flag to specify user-defined target directory location. #7349
   +0
   
4. Remove Nargo.toml lock

[aakoshh]

Regarding these two:

• Add file lock on target directory to prevent multiple commands using the same output dir concurrently.
• Remove Nargo.toml lock

I did it differently in #7345: I still take out locks on the Nargo.toml file, but it's the package specific version, rather than the workspace one. This is because if we use the workspace level output directory for all members, then locking that directory itself would again prevent parallelism, unless we lock some package specific output (e.g. the JSON artefact, the witness file or a special marker); I thought locking the already existing Nargo.toml in the input directory should be sufficient.

If we had the ability to specify the target directory, then we would still face some challenges:

• If they user doesn't override it, then we are still writing to the same workspace directory, so we can't just lock it if we want to support using parallel in the shell to speed things up.
• If the user overrides it but only to a member specific directory, then we still have to worry about using different compilation options for the same package, which is what caused the issue with the tests in the first place: while we were executing with e.g. inliner aggressiveness 0 and forcing brillig, another test was overwriting the artefacts with a different aggressiveness. We would need to lock some compilation flag specific lock file, and save separate artefacts for each configuration.