Allow repository root build directories.

[MIreland]

Describe the feature you'd like to request

Our monorepo is setup to place all build artifacts in a centralized build directory.

/apps
  /app-1
    /src
      - index.ts
  /app-2
    /src
      - index.ts
   ...etc
/build
  /app-1
    - index.js
    - other.js
  /app-2
    - index.js
    - other.js  
  ...etc

It doesn't seem like the outputs directory is supporting root directory build options- I've tried a number of permutations of ../../builds/**/*, etc.

We would like to maintain our centralized build location while taking advantage of turborepo.

Describe the solution you'd like

I would like to be able to provide an outputs glob that processes parent directories, or to have a flag/setting to determine if outputs are relative to the package or the repo root.

Describe alternatives you've considered

If outputs does not allow us to access items from the monorepo root, we will need to make substantial internal infrastructure changes to support a different pattern of build outputs.

[jaredpalmer]

I had a PR that implements this pre-acquisition. Another customer asked for it as well initially but then refactored. In summary, this requires storing all files relative to the monorepo root instead of relative to the package in the fs cache/zip and then unzipping to the root instead of the package directory.

The only headache / potential gotcha is a situation with mispecified outputs that create locks/race conditions against same outputs accidentally shared between 2 tasks

[MIreland]

Makes sense.

Is this something you are still open to adding back in in the near future?

Although I certainly understand the concern, I'm confident we can be careful enough to avoid locks/race conditions.

[jaredpalmer]

10000000000000000000000000%

[ckortekaas]

It's sometimes much easier to manage module resolutions in a large monorepo with lots of packages when using a shared root build directory. Refactoring back into individual package dist dirs has proven very messy, verbose and brittle in comparison.

Is there any chance the old PR could be resuscitated in a not too distant release?

[jaredpalmer]

Planning to look at this next week

[jaredpalmer]

Closed with #861 . Try canary to test

[ckortekaas]

@jaredpalmer Thanks for working on this. I've tried out 1.1.7-canary.1 but I can't get it to successfully cache directories below the current package directory.

For example, a '@orgname/api' package that lives off the monorepo root directory in a /api folder, setting the top-level turbo.json outputs to:

    "@orgname/api#build": {
      "dependsOn": ["^build"],
      "outputs": ["../build/@orgname/api/**"]
    },

It just doesn't cache it. I've tried various permutations (including in the default ^build outputs instead of with # scopes) but caching only works for relative paths inside the same folder (as in previous releases).

There are no errors in the build logs (with -vvv on) and the log has the named ../build/@orgname/api/** path in the list of dirs it says it will cache.

The node_modules/.cache/turbo has a hash dir for the folder (eg /api/.turbo) but it only contains a turbo-build.log file. Other packages with relative output settings (that don't use ../ paths) cache output targets all ok.

Is there something more that needs to be added to the file path Go code to allow it to use the ../ (or ../../ and deeper) syntax?

FWIW, this is using Node 16.14.0 on MacOS 12.2.1 with latest PNPM and using PNPM Workspaces, running pnpm turbo run build from the root dir of the monorepo.

[nathanhammond]

And, if this is still an issue, 1.2.8 should have you covered. If 1.2.8 doesn't take care of you, please file a new issue!

[delanebob]

I'm new enough to Turbo repo and came looking to setup caching for a build like this. I was just wondering if there is a way to get this working with 1 glob output that works dynamically across packages?

I have a folder structure like below with the packages under the dist/static folder:

In my turbo.json I tried something like this first:

"build": {
  "dependsOn": ["^build"],
  "outputs": ["../../dist/static/**/*"]
},

The issue with this was that it seemed to be caching all packages output under each packages turbo repo cache.

So I got it to only cache each individual build folders by doing this:

"build": {
  "dependsOn": ["^build"],
},
"attendance#build": {
  "outputs": ["../../dist/static/attendance/**"]
},
"dashboard#build": {
  "outputs": ["../../dist/static/dashboard/**"]
}

But I said I'd ask to see if there was a better way to make the output more dynamic. So when it grows to something like 30 packages, we won't have to continue repeating the same bits of JSON as we add each new package?

[shinebayar-g]

Does turbo repo support variables like $workspaceRoot & $projectRoot like nx? They make inputs & outputs configuration so much easier and less error prone, specially when we have deeply nested package structure.