FIPS Build

[michel-laterman]

What does this PR do?

Adds FIPS env var to magefile to enable FIPS compliant builds using the microsoft/go toolchain.

This PR will not be sufficient to ensure that every artifact made with these changes are compliant, we still need to verify our crypto use.

Why is it important?

FIPS artifacts must be built with compliant toolchains.

Checklist

• I have read and understood the pull request guidelines of this project.
• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in ./changelog/fragments using the changelog tool
• I have added an integration test or an E2E test

Disruptive User Impact

None

How to test this PR locally

Assuming microsoft go is available, run FIPS=true mage build:binary

[mergify]

This pull request does not have a backport label. Could you fix it @michel-laterman? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-./d./d is the label to automatically backport to the 8./d branch. /d is the digit

[mergify]

backport-v8.x has been added to help with the transition to the new branch 8.x.
If you don't need it please use backport-skip label and remove the backport-8.x label.

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[cmacknz]

This should come with a test that the binary we produce actually works, ideally one that can confirm it properly uses the FIPS OpenSSL in the expected way.

You'll also need to update CI to actually build the FIPS variant, the package steps for testing are in

elastic-agent/.buildkite/integration.pipeline.yml

Line 8 in a29af64

- group: "Integration tests: packaging"

right now

[simitt]

Similar to what @cmacknz already suggested, there should be some tests. IMO there would ideally also be some unit tests for the mage functionality, which shouldn't be too hard given it is regular go code.

[mergify]

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fips-build upstream/fips-build
git merge upstream/main
git push upstream fips-build

[simitt]

Left some more comments, mostly trying to follow the complexity in the mage/release logic or seeing whether it can be reduced a bit to reduce the surface for bugs.

[simitt]

Let's wait for a review from @pchila as this is related to packaging.

[dliappis]

Thanks for iterating.
There seem to be a few failing tests in CI e.g. https://buildkite.com/elastic/elastic-agent-extended-testing/builds/7357#019520ad-6465-47f3-8b18-7fca000d61ba could you please take a look?

[michel-laterman]

The current failure error is:

.../elastic-agent-9.0.0-SNAPSHOT-linux-x86_64-fips/manifest.yaml: no such file or directory

When I download the artifact and extract it, i can see the folder it creates is missing the -fips suffix; i'll try to get this working but I'm having issues stemming from running package on an arm macbook

[pchila]

Good enough to get a first FIPS compliant agent package in order to start testing.
There are some parts that will need modification to integrate with elastic-agent packaging code but that will be done in a follow-up PR that will produce FIPS artifacts along with the current ones.

Left a couple of comments as a reminder of what needs modifying

[michel-laterman]

buildkite test this

[michel-laterman]

buildkite test this

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: 65eae02

Failed CI Steps

• Unit tests - Windows 2016

History

• 💔 Build #17653 failed d548454
• 💔 Build #17587 failed 2d37190
• 💔 Build #17584 failed 2d37190
• 💔 Build #17513 failed 4e7b9e4
• 💔 Build #17508 failed f4b3c6a

cc @michel-laterman

[elastic-sonarqube]

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
100.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

[michel-laterman]

@dliappis, @v1v can I please get a review?

[dliappis]

LGTM