Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kustomize build with remote resource may panic on slow networks #3362

Closed
johejo opened this issue Dec 16, 2020 · 3 comments
Closed

kustomize build with remote resource may panic on slow networks #3362

johejo opened this issue Dec 16, 2020 · 3 comments
Labels
area/api issues for api module kind/bug Categorizes issue or PR as related to a bug.

Comments

@johejo
Copy link

johejo commented Dec 16, 2020

Describe the bug

At 6bbc829#diff-867c23b9f5a4ba53f94117fdfeb7c9899f849e5e231fee3d720fab5ceae43b9f
A timeout has been added to the getter.

But, due to improper handling of channels, writing to closed channels occurs at some point.

ch := make(chan bool, 1)
defer close(ch)
d := 21 * time.Second // arbitrary
timer := time.NewTimer(d)
defer timer.Stop()
go func() {
err = client.Get()
ch <- true
}()
select {
case <-ch:
case <-timer.C:
err = types.NewErrTimeOut(d, "go-getter client.Get")
}
return err

If the call to client.Get exceeds 21 seconds (I don't know why this timeout value was chosen), the defer close(ch) closes the channel because the case for the timer finishes and returns.
After that, the background goroutine writes to the channel and panic occurs.

It does not occur every time.

Files that can reproduce the issue

Expected output

$ kustomize build "github.com/kubernetes-sigs/kustomize//examples/multibases"
piVersion: v1
kind: Pod
metadata:
  labels:
    app: myapp
  name: cluster-a-dev-myapp-pod
spec:
  containers:
  - image: nginx:1.7.9
    name: nginx
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: myapp
  name: cluster-a-prod-myapp-pod
spec:
  containers:
  - image: nginx:1.7.9
    name: nginx
---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: myapp
  name: cluster-a-staging-myapp-pod
spec:
  containers:
  - image: nginx:1.7.9
    name: nginx

Actual output

$ kustomize build "github.com/kubernetes-sigs/kustomize//examples/multibases"
panic: send on closed channel

goroutine 9 [running]:
sigs.k8s.io/kustomize/api/loader.getRemoteTarget.func1(0xc00021e160, 0xc00021c610, 0xc000168070)
        /home/heijo/ghq/github.com/kubernetes-sigs/kustomize/api/loader/getter.go:104 +0x65
created by sigs.k8s.io/kustomize/api/loader.getRemoteTarget
        /home/heijo/ghq/github.com/kubernetes-sigs/kustomize/api/loader/getter.go:102 +0x446
exit status 2

Kustomize version

Platform

LInux

Additional context

@johejo
Copy link
Author

johejo commented Dec 16, 2020

Although not directly related to this issue, the variable err may be assigned from multiple goroutines at the same times, so it seems to be a race condition.

johejo added a commit to johejo/kustomize that referenced this issue Dec 16, 2020
johejo added a commit to johejo/kustomize that referenced this issue Dec 16, 2020
@Shell32-Natsu Shell32-Natsu added area/api issues for api module kind/bug Categorizes issue or PR as related to a bug. labels Dec 16, 2020
@monopole
Copy link
Contributor

Thanks for the issue report @johejo , you're correct on all points.

@monopole
Copy link
Contributor

fixed in #3400

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api issues for api module kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants