pathogen-repo-build: Move build.log to ignored directory

The entire working directory gets zipped and uploaded to AWS Batch when using the `aws-batch` runtime, which includes an early `build.log` file. When the build completes and downloads the build outputs, the early `build.log` overwrites the local `build.log` of the GH Action. This results in a truncated file that does not include the AWS_BATCH_JOB_ID and prevents the generation of the AWS Batch summary. This commit moves the hard-coded `build.log` file to an ignored directory (the same directory that we are using for the nextstrain/.github repo¹). This prevents the `build.log` from being uploaded with the build. Previously tried other paths, but they all caused errors. - "../build.log" - ".." pattern is not allowed for the `upload-artifact` action² - "~/build.log" - `hashfiles` requires files to be in the `GITHUB_WORKSPACE`³ ¹ #44 (comment) ² actions/upload-artifact#176 ³ https://docs.github.com/en/actions/learn-github-actions/expressions#hashfiles
nextstrain · Mar 15, 2024 · 504728b · 504728b
1 parent 5cd0ba1
commit 504728b
Show file tree

Hide file tree

Showing 2 changed files with 18 additions and 17 deletions.
diff --git a/.github/workflows/pathogen-repo-build.yaml b/.github/workflows/pathogen-repo-build.yaml
@@ -98,14 +98,14 @@ on:
           The default paths included in the artifact are:
 
 
-            build.log
+            .git/nextstrain/.github/build.log
             auspice/
             results/
             benchmarks/
             logs/
             .snakemake/log/
 
-          The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
+          The ".git/nextstrain/.github/build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !.git/nextstrain/.github/build.log).
 
           This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
         type: string
@@ -189,14 +189,14 @@ on:
           The default paths included in the artifact are:
 
 
-            build.log
+            .git/nextstrain/.github/build.log
             auspice/
             results/
             benchmarks/
             logs/
             .snakemake/log/
 
-          The "build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !build.log).
+          The ".git/nextstrain/.github/build.log" contains log messages from the `nextstrain build` command. The other paths are common output paths for Nextstrain builds. If a path does not exist in your build, then the action will still succeed and will print out a warning for the non-existent file(s). Use an exclude pattern for any of the default paths that you would like to exclude from the artifact (e.g. !.git/nextstrain/.github/build.log).
 
           This is not supported for builds on AWS Batch because the workflow detaches from the build. Please use the `nextstrain build` command locally to reattach to AWS Batch builds to download outputs.
         type: string
@@ -274,14 +274,14 @@ jobs:
           # shellcheck disable=SC2154
           set -x
 
-          eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
+          eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee "$NEXTSTRAIN_GITHUB_DIR"/build.log
       # Attempt to get the AWS Batch ID even if the run build command failed
-      # as long as the runtime is `aws-batch` and the `build.log` file exists
-      - if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
+      # as long as the runtime is `aws-batch` and the `"$NEXTSTRAIN_GITHUB_DIR"/build.log` file exists
+      - if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles( env.NEXTSTRAIN_GITHUB_DIR/'build.log') != '' }}
         name: Get AWS Batch job id
         id: aws-batch
         run: |
-          echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
+          echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < "$NEXTSTRAIN_GITHUB_DIR"/build.log)" | tee -a "$GITHUB_ENV"
       - if: ${{ always() && env.AWS_BATCH_JOB_ID }}
         name: Generate AWS Batch summary
         run: |
@@ -293,7 +293,7 @@ jobs:
           if-no-files-found: warn
           name: ${{ inputs.artifact-name }}
           path: |
-            build.log
+            ${{ env.NEXTSTRAIN_GITHUB_DIR }}/build.log
             auspice/
             results/
             benchmarks/

diff --git a/.github/workflows/pathogen-repo-build.yaml.in b/.github/workflows/pathogen-repo-build.yaml.in
@@ -128,19 +128,20 @@ on:
 
           The default paths included in the artifact are:
 
-            build.log
+            .git/nextstrain/.github/build.log
             auspice/
             results/
             benchmarks/
             logs/
             .snakemake/log/
 
-          The "build.log" contains log messages from the `nextstrain build` command.
+          The ".git/nextstrain/.github/build.log" contains log messages from the
+          `nextstrain build` command.
           The other paths are common output paths for Nextstrain builds.
           If a path does not exist in your build, then the action will still
           succeed and will print out a warning for the non-existent file(s).
           Use an exclude pattern for any of the default paths that you would like to
-          exclude from the artifact (e.g. !build.log).
+          exclude from the artifact (e.g. !.git/nextstrain/.github/build.log).
 
           This is not supported for builds on AWS Batch because the workflow
           detaches from the build. Please use the `nextstrain build` command
@@ -240,15 +241,15 @@ jobs:
           # shellcheck disable=SC2154
           set -x
 
-          eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee build.log
+          eval "$NEXTSTRAIN_BUILD_COMMAND" |& tee "$NEXTSTRAIN_GITHUB_DIR"/build.log
 
       # Attempt to get the AWS Batch ID even if the run build command failed
-      # as long as the runtime is `aws-batch` and the `build.log` file exists
-      - if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles('build.log') != '' }}
+      # as long as the runtime is `aws-batch` and the `"$NEXTSTRAIN_GITHUB_DIR"/build.log` file exists
+      - if: ${{ always() && inputs.runtime == 'aws-batch' && hashFiles( env.NEXTSTRAIN_GITHUB_DIR/'build.log') != '' }}
         name: Get AWS Batch job id
         id: aws-batch
         run: |
-          echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < build.log)" | tee -a "$GITHUB_ENV"
+          echo "AWS_BATCH_JOB_ID=$(sed -nE 's/.+AWS Batch Job ID\:.+ ([-a-f0-9]+)$/\1/p' < "$NEXTSTRAIN_GITHUB_DIR"/build.log)" | tee -a "$GITHUB_ENV"
 
       - if: ${{ always() && env.AWS_BATCH_JOB_ID }}
         name: Generate AWS Batch summary
@@ -262,7 +263,7 @@ jobs:
           if-no-files-found: warn
           name: ${{ inputs.artifact-name }}
           path: |
-            build.log
+            ${{ env.NEXTSTRAIN_GITHUB_DIR }}/build.log
             auspice/
             results/
             benchmarks/