Skip to content

Commit aa9ed14

Browse files
authored
Merge pull request #2837 from sbueringer/pr-setup-envtest-ct-rel-0.18
[release-0.18] ✨ setup-envtest: download binaries from controller-tools releases
2 parents 834905b + 35d7bbd commit aa9ed14

15 files changed

+1271
-655
lines changed

tools/setup-envtest/README.md

+26-9
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,18 @@ setup-envtest use -i --use-env
4040

4141
# sideload a pre-downloaded tarball as Kubernetes 1.16.2 into our store
4242
setup-envtest sideload 1.16.2 < downloaded-envtest.tar.gz
43+
44+
# If --use-deprecated-gcs is set to false envtest binaries are downloaded from:
45+
# https://raw.githubusercontent.com/kubernetes-sigs/controller-tools/master/envtest-releases.yaml
46+
# To download from a custom index use the following:
47+
# Note: In controller-runtime v0.19.0 --use-deprecated-gcs will default to false.
48+
setup-envtest use --use-deprecated-gcs=false --index https://custom.com/envtest-releases.yaml
49+
50+
# To download from the kubebuilder-tools GCS bucket: (default behavior until v0.18)
51+
# Note: This is a Google-owned bucket and it might be shutdown at any time
52+
# see: https://github.com/kubernetes/k8s.io/issues/2647#event-12439345373
53+
# Note: This flag will also be removed soon.
54+
setup-envtest use --use-deprecated-gcs
4355
```
4456

4557
## Where does it put all those binaries?
@@ -51,16 +63,16 @@ On Linux, this is `$XDG_DATA_HOME`; on Windows, `%LocalAppData`; and on
5163
OSX, `~/Library/Application Support`.
5264

5365
There's an overall folder that holds all files, and inside that is
54-
a folder for each version/platform pair. The exact directory structure is
55-
not guarnateed, except that the leaf directory will contain the names
56-
expected by envtest. You should always use `setup-envtest fetch` or
66+
a folder for each version/platform pair. The exact directory structure is
67+
not guaranteed, except that the leaf directory will contain the names
68+
expected by envtest. You should always use `setup-envtest fetch` or
5769
`setup-envtest switch` (generally with the `-p path` or `-p env` flags) to
5870
get the directory that you should use.
5971

6072
## Why do I have to do that `source <(blah blah blah)` thing
6173

6274
This is a normal binary, not a shell script, so we can't set the parent
63-
process's environment variables. If you use this by hand a lot and want
75+
process's environment variables. If you use this by hand a lot and want
6476
to save the typing, you could put something like the following in your
6577
`~/.zshrc` (or similar for bash/fish/whatever, modified to those):
6678

@@ -79,7 +91,7 @@ setup-envtest() {
7991
There are a few options.
8092

8193
First, you'll probably want to set the `-i/--installed` flag. If you want
82-
to avoid forgetting to set this flag, set the `ENVTEST_INSTALLED_ONLY`
94+
to avoid forgetting to set this flag, set the `ENVTEST_INSTALLED_ONLY`
8395
env variable, which will switch that flag on by default.
8496

8597
Then, you have a few options for managing your binaries:
@@ -98,13 +110,18 @@ Then, you have a few options for managing your binaries:
98110
`--use-env` on by default.
99111

100112
- If you want to use this tool, but download your gziped tarballs
101-
separately, you can use the `sideload` command. You'll need to use the
113+
separately, you can use the `sideload` command. You'll need to use the
102114
`-k/--version` flag to indicate which version you're sideloading.
103115

104116
After that, it'll be as if you'd installed the binaries with `use`.
105117

106-
- If you want to talk to some internal source, you can use the
107-
`--remote-bucket` and `--remote-server` options. The former sets which
118+
- If you want to talk to some internal source via HTTP, you can simply set `--index`
119+
The index must contain references to envtest binary archives in the same format as:
120+
https://raw.githubusercontent.com/kubernetes-sigs/controller-tools/master/envtest-releases.yaml
121+
122+
- If you want to talk to some internal source in a GCS "style", you can use the
123+
`--remote-bucket` and `--remote-server` options together with `--use-deprecated-gcs`.
124+
Note: This is deprecated and will be removed soon. The former sets which
108125
GCS bucket to download from, and the latter sets the host to talk to as
109126
if it were a GCS endpoint. Theoretically, you could use the latter
110127
version to run an internal "mirror" -- the tool expects
@@ -114,7 +131,7 @@ Then, you have a few options for managing your binaries:
114131
```json
115132
{"items": [
116133
{"name": "kubebuilder-tools-X.Y.Z-os-arch.tar.gz", "md5Hash": "<base-64-encoded-md5-hash>"},
117-
{"name": "kubebuilder-tools-X.Y.Z-os-arch.tar.gz", "md5Hash": "<base-64-encoded-md5-hash>"},
134+
{"name": "kubebuilder-tools-X.Y.Z-os-arch.tar.gz", "md5Hash": "<base-64-encoded-md5-hash>"}
118135
]}
119136
```
120137

tools/setup-envtest/env/env.go

+23-19
Original file line numberDiff line numberDiff line change
@@ -26,24 +26,28 @@ import (
2626
// envtest binaries.
2727
//
2828
// In general, the methods will use the Exit{,Cause} functions from this package
29-
// to indicate errors. Catch them with a `defer HandleExitWithCode()`.
29+
// to indicate errors. Catch them with a `defer HandleExitWithCode()`.
3030
type Env struct {
3131
// the following *must* be set on input
3232

3333
// Platform is our current platform
3434
Platform versions.PlatformItem
3535

36-
// VerifiySum indicates whether or not we should run checksums.
36+
// VerifySum indicates whether we should run checksums.
3737
VerifySum bool
38-
// NoDownload forces us to not contact GCS, looking only
39-
// at local files instead.
38+
// NoDownload forces us to not contact remote services,
39+
// looking only at local files instead.
4040
NoDownload bool
4141
// ForceDownload forces us to ignore local files and always
42-
// contact GCS & re-download.
42+
// contact remote services & re-download.
4343
ForceDownload bool
4444

45-
// Client is our remote client for contacting GCS.
46-
Client *remote.Client
45+
// UseDeprecatedGCS signals if the GCS client is used.
46+
// Note: This will be removed together with remote.GCSClient.
47+
UseDeprecatedGCS bool
48+
49+
// Client is our remote client for contacting remote services.
50+
Client remote.Client
4751

4852
// Log allows us to log.
4953
Log logr.Logger
@@ -133,7 +137,7 @@ func (e *Env) ListVersions(ctx context.Context) {
133137
}
134138

135139
// LatestVersion returns the latest version matching our version selector and
136-
// platform from the remote server, with the correspoding checksum for later
140+
// platform from the remote server, with the corresponding checksum for later
137141
// use as well.
138142
func (e *Env) LatestVersion(ctx context.Context) (versions.Concrete, versions.PlatformItem) {
139143
vers, err := e.Client.ListVersions(ctx)
@@ -193,7 +197,7 @@ func (e *Env) ExistsAndValid() bool {
193197
//
194198
// If necessary, it will enumerate on-disk and remote versions to accomplish
195199
// this, finding a version that matches our version selector and platform.
196-
// It will always yield a concrete version, it *may* yield a concrete platorm
200+
// It will always yield a concrete version, it *may* yield a concrete platform
197201
// as well.
198202
func (e *Env) EnsureVersionIsSet(ctx context.Context) {
199203
if e.Version.AsConcrete() != nil {
@@ -247,13 +251,13 @@ func (e *Env) EnsureVersionIsSet(ctx context.Context) {
247251

248252
// if we're not forcing a download, and we have a newer local version, just use that
249253
if !e.ForceDownload && localVer != nil && localVer.NewerThan(serverVer) {
250-
e.Platform.Platform = localPlat // update our data with md5
254+
e.Platform.Platform = localPlat // update our data with hash
251255
e.Version.MakeConcrete(*localVer)
252256
return
253257
}
254258

255259
// otherwise, use the new version from the server
256-
e.Platform = platform // update our data with md5
260+
e.Platform = platform // update our data with hash
257261
e.Version.MakeConcrete(serverVer)
258262
}
259263

@@ -266,13 +270,13 @@ func (e *Env) Fetch(ctx context.Context) {
266270
log := e.Log.WithName("fetch")
267271

268272
// if we didn't just fetch it, grab the sum to verify
269-
if e.VerifySum && e.Platform.MD5 == "" {
273+
if e.VerifySum && e.Platform.Hash == nil {
270274
if err := e.Client.FetchSum(ctx, *e.Version.AsConcrete(), &e.Platform); err != nil {
271-
ExitCause(2, err, "unable to fetch checksum for requested version")
275+
ExitCause(2, err, "unable to fetch hash for requested version")
272276
}
273277
}
274278
if !e.VerifySum {
275-
e.Platform.MD5 = "" // skip verification
279+
e.Platform.Hash = nil // skip verification
276280
}
277281

278282
var packedPath string
@@ -287,7 +291,7 @@ func (e *Env) Fetch(ctx context.Context) {
287291
}
288292
})
289293

290-
archiveOut, err := e.FS.TempFile("", "*-"+e.Platform.ArchiveName(*e.Version.AsConcrete()))
294+
archiveOut, err := e.FS.TempFile("", "*-"+e.Platform.ArchiveName(e.UseDeprecatedGCS, *e.Version.AsConcrete()))
291295
if err != nil {
292296
ExitCause(2, err, "unable to open file to write downloaded archive to")
293297
}
@@ -365,8 +369,8 @@ func (e *Env) PrintInfo(printFmt PrintFormat) {
365369
case PrintOverview:
366370
fmt.Fprintf(e.Out, "Version: %s\n", e.Version)
367371
fmt.Fprintf(e.Out, "OS/Arch: %s\n", e.Platform)
368-
if e.Platform.MD5 != "" {
369-
fmt.Fprintf(e.Out, "md5: %s\n", e.Platform.MD5)
372+
if e.Platform.Hash != nil {
373+
fmt.Fprintf(e.Out, "%s: %s\n", e.Platform.Hash.Type, e.Platform.Hash.Value)
370374
}
371375
fmt.Fprintf(e.Out, "Path: %s\n", path)
372376
case PrintPath:
@@ -409,7 +413,7 @@ func (e *Env) Sideload(ctx context.Context, input io.Reader) {
409413
}
410414

411415
var (
412-
// expectedExectuables are the executables that are checked in PathMatches
416+
// expectedExecutables are the executables that are checked in PathMatches
413417
// for non-store paths.
414418
expectedExecutables = []string{
415419
"kube-apiserver",
@@ -458,7 +462,7 @@ func (e *Env) PathMatches(value string) bool {
458462
}
459463

460464
// versionFromPathName checks if the given path's last component looks like one
461-
// of our versions, and, if so, what version it represents. If succesfull,
465+
// of our versions, and, if so, what version it represents. If successful,
462466
// it'll set version and platform, and return true. Otherwise it returns
463467
// false.
464468
func (e *Env) versionFromPathName(value string) bool {

tools/setup-envtest/go.mod

+14-12
Original file line numberDiff line numberDiff line change
@@ -3,25 +3,27 @@ module sigs.k8s.io/controller-runtime/tools/setup-envtest
33
go 1.22.0
44

55
require (
6-
github.com/go-logr/logr v1.2.4
7-
github.com/go-logr/zapr v1.2.4
8-
github.com/onsi/ginkgo/v2 v2.12.1
9-
github.com/onsi/gomega v1.27.10
6+
github.com/go-logr/logr v1.4.1
7+
github.com/go-logr/zapr v1.3.0
8+
github.com/onsi/ginkgo/v2 v2.17.1
9+
github.com/onsi/gomega v1.32.0
1010
github.com/spf13/afero v1.6.0
1111
github.com/spf13/pflag v1.0.5
1212
go.uber.org/zap v1.26.0
13+
k8s.io/apimachinery v0.0.0-20240424173219-03f2f3350dc5
14+
sigs.k8s.io/yaml v1.3.0
1315
)
1416

1517
require (
1618
github.com/go-task/slim-sprig v0.0.0-20230315185526-52ccab3ef572 // indirect
17-
github.com/golang/protobuf v1.5.3 // indirect
18-
github.com/google/go-cmp v0.5.9 // indirect
19-
github.com/google/pprof v0.0.0-20210407192527-94a9f03dee38 // indirect
19+
github.com/google/go-cmp v0.6.0 // indirect
20+
github.com/google/pprof v0.0.0-20210720184732-4bb14d4b1be1 // indirect
2021
go.uber.org/multierr v1.10.0 // indirect
21-
golang.org/x/net v0.14.0 // indirect
22-
golang.org/x/sys v0.12.0 // indirect
23-
golang.org/x/text v0.12.0 // indirect
24-
golang.org/x/tools v0.12.0 // indirect
25-
google.golang.org/protobuf v1.28.0 // indirect
22+
golang.org/x/net v0.23.0 // indirect
23+
golang.org/x/sys v0.18.0 // indirect
24+
golang.org/x/text v0.14.0 // indirect
25+
golang.org/x/tools v0.18.0 // indirect
26+
google.golang.org/protobuf v1.33.0 // indirect
27+
gopkg.in/yaml.v2 v2.4.0 // indirect
2628
gopkg.in/yaml.v3 v3.0.1 // indirect
2729
)

0 commit comments

Comments
 (0)