Manifestival can parse and load yamls with correct format, but unable to go through if there are files in bad format under the target directory

[houshengbo]

I was trying to manually set the env variable

I created my own 0.9.0.yaml, and created a configmap loading it by

kubectl create configmap game-config --from-file=/local_path.../knative.dev/serving-operator/files/knative-serving/0.9.0.yaml

Then, this is the operator deployment yaml I used:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: knative-serving-operator
spec:
  replicas: 1
  selector:
    matchLabels:
      name: knative-serving-operator
  template:
    metadata:
      annotations:
        sidecar.istio.io/inject: "false"
      labels:
        name: knative-serving-operator
    spec:
      serviceAccountName: knative-serving-operator
      containers:
        - name: knative-serving-operator
          image: knative.dev/serving-operator/cmd/manager
          imagePullPolicy: IfNotPresent
          #command:
          #  - ["bin/sh", "-c", "ls /etc/kodata/knative-serving ; echo the path of KO_DATA_PATH: $(KO_DATA_PATH)"]
          env:
            - name: KO_DATA_PATH
              value: "/etc/kodata/"
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: OPERATOR_NAME
              value: "knative-serving-operator"
            - name: SYSTEM_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: METRICS_DOMAIN
              value: knative.dev/serving-operator
          volumeMounts:
            - name: game-config-volume
              mountPath: /etc/kodata/knative-serving
      volumes:
        - name: game-config-volume
          configMap:
            name: game-config

I expected that operator pod will read the env variable KO_DATA_PATH, and load the manifest from the mounted path "/etc/kodata/knative-serving". And it is true that this is something we surely can do.

However, I came across the following log:

{"level":"info","ts":"2019-10-30T17:50:08.438Z","caller":"logging/config.go:103","msg":"Logging level set to info"}
{"level":"info","ts":"2019-10-30T17:50:08.438Z","caller":"logging/config.go:71","msg":"Fetch GitHub commit ID from kodata failed: open /etc/kodata/HEAD: no such file or directory"}
{"level":"info","ts":"2019-10-30T17:50:08.440Z","logger":"serving-operator.knativeserving-controller","caller":"knativeserving/controller.go:75","msg":"The ko_path exists.","knative.dev/controller":"knativeserving-controller"}
{"level":"info","ts":"2019-10-30T17:50:08.440Z","logger":"serving-operator.knativeserving-controller","caller":"knativeserving/controller.go:82","msg":"The ko_path_serving exists.","knative.dev/controller":"knativeserving-controller"}
{"level":"info","ts":"2019-10-30T17:50:08.441Z","logger":"serving-operator.knativeserving-controller","caller":"knativeserving/controller.go:93","msg":"..2019_10_30_17_50_07.090926616","knative.dev/controller":"knativeserving-controller"}
{"level":"info","ts":"2019-10-30T17:50:08.441Z","logger":"serving-operator.knativeserving-controller","caller":"knativeserving/controller.go:93","msg":"..data","knative.dev/controller":"knativeserving-controller"}
{"level":"info","ts":"2019-10-30T17:50:08.441Z","logger":"serving-operator.knativeserving-controller","caller":"knativeserving/controller.go:93","msg":"0.9.0.yaml","knative.dev/controller":"knativeserving-controller"}
{"level":"info","logger":"manifestival","msg":"Reading manifest","name":"/etc/kodata/knative-serving"}

The log has been frozen here. manifestival is trying to Reading manifest, but no further movement.

I added a snippet in the source to find out what is going on. It turned out that there are literally 3 files under the path /etc/kodata/knative-serving: ..data, ..2019_10_30_17_50_07.090926616 and 0.9.0.yaml. It seems that the package manifestival is unable to exclude the files, which are not in the correct yaml format.

I suppose this is a bug for the lib manifestival to improve. For example, we can exclude the files not in correct yaml format.
@jcrossley3 I would love to hear from you. Thx.

[jcrossley3]

Manifestival doesn't currently do any YAML validation. It assumes the YAML you pass it is valid, and it will attempt to load every single file beneath any path that is a directory.

[houshengbo]

I have fixed this issue with a patch in manifestival package: jcrossley3/manifestival@33c10fa