Gramine support: Vault doesn't support being built dynamically linked (error building with -linkshared flag)

[svenkata9]

Describe the bug

I am trying to build a dynamically linked vault binary so that Vault runs with Gramine. I updated the scripts/build.sh to use -linkshared flag, but I encountered the error below. Gramine is a LibOS and Gramine-SGX makes the binary run securely within Intel(R) Software Guard Extensions.

make -j dev
==> Checking that build is using go version >= 1.20.5...
==> Using go version 1.20.7...
==> Removing old directory...
==> Building...
# github.com/hashicorp/vault
2023/08/08 11:53:50 duplicated definition of symbol github.com/zclconf/go-cty/cty/set.Set[go.shape.interface {}].SymmetricDifference.func2, from github.com/hashicorp/hcl/v2/gohcl and github.com/hashicorp/vault/command
make: *** [Makefile:39: dev] Error 1

To Reproduce
Steps to reproduce the behavior:

1. Modify the build script to include -linkshared flag. Shippet below:

 echo "==> Building..."
 ${GO_CMD} build \
     -gcflags "${GCFLAGS}" \
+    -linkshared \
     -ldflags "${LD_FLAGS} -X github.com/hashicorp/vault/version.GitCommit='${GIT_COMMIT}${GIT_DIRTY}' -X github.com/hashicorp/vault/version.BuildDate=${BUILD_DATE}" \
     -o "bin/vault" \
     -tags "${BUILD_TAGS}" \

2. make dev

Expected behavior
Building vault binary with -linkshared should be successful, resulting in a dynamically linked binary.

Environment:

• Go version 1.20.7
• Vault GitHub commit: a29b88b

[VioletHynes]

Hey there! Thanks for submitting this.

We discussed this internally, and we're going to close this one. Broadly, we have no reason to support dynamic linking, and we don't claim to support Vault when built with an updated build.sh. You're free to tinker with it if you'd like to get it working for yourself, but we can't support or help with any arbitrary changes to the build file, and we don't consider this an issue, as it's not something we support.

[svenkata9]

Thanks, @VioletHynes. We were trying to enable this dynamically linked binary from the context of Gramine. To give you some context, Gramine is a LibOS and Gramine-SGX makes the binary run securely within Intel(R) Software Guard Extensions. The statically linked vault binary works fine, but for performance reasons we need to have a dynamically built binary to avoid inline system calls (Gramine intercepts the syscalls and handles it, but that won't be the case with inline syscalls in a statically linked binary).

So, would need your help here to enable this use case to be performant. It could be an optional environment variable that we pass during make or make dev that is passed to the build script and not necessarily making it for all the builds (I just changed it directly to try and build it).

@monavij

[VioletHynes]

Hey! Thanks for the additional context! I'm going to re-open this and edit some parts so it's clearer, I hope that's okay?

It makes little sense for us to support arbitrary changes to the build file, but it does make sense for us to consider improvements that help Vault run with secure OS environments.

I'm going to re-open discussion with the team about this one.

[svenkata9]

Thank you, @VioletHynes. I appreciate it. Please let me know if you need more details.

[svenkata9]

Hi @VioletHynes. Did you discuss with your team on this? Please let me know if you need any info.

[VioletHynes]

Hi there! I did, I was told our partner/alliances folks were in contact with Intel on the issue. I can dig further if that doesn't sound accurate to you, but I'd check with whoever on your team is in contact with HashiCorp first :)

[svenkata9]

Thanks @VioletHynes. I checked internally now and no one that I know and work with, got connected with HashiCorp on this.

Would you mind letting us know the name? If privacy is a concern, I can send you an email to the gmail id that I see in your profile, or you can let me know via email in my profile.

[VioletHynes]

Hey! Thanks for the heads up. I reached out again on the HashiCorp side and our partner folks are digging into this more and should be reaching out again today or very soon.

[svenkata9]

Any update on this @VioletHynes?

[svenkata9]

Hi @VioletHynes, please let us know how we can work together on this to improve the performance of Vault in Gramine-SGX environment. Can we start an email thread?

@monavij

[svenkata9]

Hi @sgmiller, Got to know from my colleagues here that they were in discussion with you on Gramine-SGX before and there were some performance concerns.

Could you take a look into this issue? With this build time flag, the performance with Gramine-SGX will improve because they will get rid of the inline syscalls that static binary will have.

Thanks.

[sgmiller]

Hi @svenkata9 . No, we just evaluated SGX without the Gramine context (Gramine didn't exist at the time) and didn't find that the effort to develop custom code to run in the SGX enclave was worth the effort at the time. We'll discuss whether adopting build modifications are an option, but you're obviously okay maintaining a patch for this.

[heatherezell]

Hi folks - is there anything more that is needed on this issue? Please let me know if we can close this at this time. Thanks!