Create a new builder cmd to support new Docker-Based Builder type

[asraa]

Description

This is a meta/tracking issue to track AIs and dev items to support a SLSA 3 Docker-Based Builder according to
https://docs.google.com/document/d/1ky2yKk0dsSHuO6EjWyWtuyK4QupRD3eoBCzFpwzOWxc/edit?resourcekey=0-KyXXUyad6_XLpa_HPa7eYw#heading=h.x664rqu0e9bo (request access).

Steps and Timeline

Builder binary changes:

1. Create a configuration parser. The configuration file contains options, environment variables, and commands.
2. Create a command to generate the new predicate. We can either refactor the original builder binary to support sub-commands, or create a new builder. I think it's most simple to create a new builder binary under cmd/builder-alpha or something else.

• The first subcommand would be generate-predicate (which only resolves the --config and produces the SLSA predicate). It also takes in the --docker-image and --image-digest and --output-path. It may also take some GitHub context as input through the environment.
• The second subcommand would be a build command that may ingest the previously produced SLSA predicate

3. Create the resuable workflow that invokes the builder binary.

There are still some AIs to resolve, but I think (1) is possible to do now, as well as the scaffolding for the two builder commands.

Tracking issues:

• Reusable workflow: Docker build image workflow slsa-framework/slsa-github-generator#56
• Builder binary: [feature] Create a docker-based builder binary that executes a builder image to produce a binary slsa-framework/slsa-github-generator#1191
• Discussion issue: Discussion: Use a Docker image for building binaries/files prior to generating SLSA provenances  slsa-framework/slsa-github-generator#23

@rbehjati

[rbehjati]

@asraa Thanks for creating the issue and writing down the steps.

With the new predicate format, I think we no longer need the first subcommand (generate-predicate). With the new format, the trusted GitHub builder (the outer builder) has all the information required for generating a complete SLSA provenance. What do you think?

Create a configuration parser. The configuration file contains options, environment variables, and commands.

Our current builder already parses the config file, but does not support options, and environment variables. See

transparent-release/internal/common/common.go

Lines 78 to 90 in 8d11813

	func LoadBuildConfigFromFile(path string) (*BuildConfig, error) {
	tomlTree, err := toml.LoadFile(path)
	if err != nil {
	return nil, fmt.Errorf("couldn't load toml file: %v", err)
	}
	config := BuildConfig{}
	if err := tomlTree.Unmarshal(&config); err != nil {
	return nil, fmt.Errorf("couldn't ubmarshal toml file: %v", err)
	}
	return &config, nil
	}

As a starting point, I suggest we remove the provenance generation from the current builder, and start from implementing the reusable GitHub workflow. Can you advise on how much of the functionality in https://github.com/slsa-framework/slsa-github-generator could be reused?

We can then iterate on the parser and add support for options and environment variables.

[asraa]

I suggest we remove the provenance generation from the current builder, and start from implementing the reusable GitHub workflow. Can you advise on how much of the functionality in https://github.com/slsa-framework/slsa-github-generator could be reused?

Sure! We can start from the skeleton of the workflow and move down. These are the components that can be reused: https://github.com/slsa-framework/slsa-github-generator/tree/main/.github/actions. They're around passing binaries between the VMs/jobs, checking out project repositories, and detecting the reusable workflows environment for the provenance. We can also re-use the signing packages that are used in the builders. We'd still need our own builder that can create the provenance and parse the udpated configuration.

With the new predicate format, I think we no longer need the first subcommand (generate-predicate). With the new format, the trusted GitHub builder (the outer builder) has all the information required for generating a complete SLSA provenance.

I'm a tiny bit confused, just want to clarify. There still needs to be some translation from the inputs to the reusable workflow into the predicate that needs to be signed. Do you mean that instead we can create a "provenance generator" binary for the reusable workflow that does not overlap with the actual builder? If so, then I agree, but still think a binary or code needs to be written to package the inputs into the predicate

[rbehjati]

There still needs to be some translation from the inputs to the reusable workflow into the predicate that needs to be signed. Do you mean that instead we can create a "provenance generator" binary for the reusable workflow that does not overlap with the actual builder? If so, then I agree, but still think a binary or code needs to be written to package the inputs into the predicate

Yes. I agree with that. It is just that with the new format the generate-predicate and build functionalities do not have much in common. So it might be easier to have them as separate commands, instead of merging them into a single tool. Or maybe they both take the same input? The input format is not entirely clear to me yet.

[rbehjati]

Summary of the discussion:

We need three main components:

• Predicate generator (parses the input)
  • Input: workflow inputs (should be convenient for running locally as well)
  • Output: The predicate (see example)
  • Location of the code: use-case specific
  • Open question: Should the command be a workflow input or part of the config?
• Builder:
  • Input: Predicate (could be partial, e.g, missing the builderBinary)
  • Output: The Binary
  • Location of the code: slsa-framework
• Reusable workflow
  • Behaviour: orchestrating the steps: build, signing and uploading the provenance
    • Should this also upload the binary and the provenance as artefacts attached to the GitHub Workflow run?
  • Location of the code: slsa-framework

@asraa Please comment and correct me if anything is missing or incorrect :)

[asraa]

That makes sense to me!

We can follow-up on the predicate input command vs config.

I've updated the issue with the two SLSA features (reusable workflow, builder binary), and we have a tracking milestone there too.

[rbehjati]

@asraa Can we close this now that we have the container-based SLSA3 builder?