Activator headers getting forwarded prevent scaling from 0

[trisberg]

Expected Behavior

Apps/functions always scale up from 0 when a new request arrives

Actual Behavior

Due to renaming the header used by the Activator in PR #2051 this header now gets forwarded in the eventing message bus and seems to interfere with the scaling up of the next app/function in a message chain.

Steps to Reproduce the Problem

1. create two or more apps
2. create two channels
3. create subscription for one app from the first channel and set reply-to to second channel
4. subscribe the second app to the second channel
5. wait for both apps to scale to 0
6. post message to the first channel and watch the first app get scaled up
7. the second app never scales up even though a message is published to the second channel

Additional Info

The message on the second channel would look something like this:

┌──────┬─────────────────┬───────────────────────────────────────────────────┐
│ DATA │    MESSAGE_ID   │                     ATTRIBUTES                    │
├──────┼─────────────────┼───────────────────────────────────────────────────┤
│ 121  │ 257546630995268 │ Content-Type=text/plain; charset=utf-8            │
│      │                 │ Knative-Activator-Num-Retries=3                   │
│      │                 │ X-B3-Sampled=1                                    │
│      │                 │ X-B3-Spanid=a8cf2769119c0fb4                      │
│      │                 │ X-B3-Traceid=a8cf2769119c0fb4                     │
│      │                 │ X-Request-Id=26a25509-346b-99a2-a458-9cd3340b689b │
└──────┴─────────────────┴───────────────────────────────────────────────────┘

It's easy to verify this by using Cloud Pub/Sub as the bus and viewing the messages for the topics directly.

If this is fixed in serving then that would solve the issue knative/eventing#523 reported in eventing.

[scothis]

/assign @scothis