feat(gcp_stackdriver_logs sink): Support additional top level labels for gcp_stackdriver_logs sink

[stackempty]

Summary

This adds top level label field according to LogEntry docs https://cloud.google.com/logging/docs/reference/v2/rest/v2/LogEntry#FIELDS.labels!
This will allow users add additional labels to the label field in log entries not just the ones in resource object!

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

How did you test this PR?

• Ran the unit tests

  cargo test --package vector --lib -- sinks::gcp::stackdriver::logs::tests --show-output```

• cargo test --all
• Tested against an existing google account and viewed the logs in GCL.

Does this PR include user facing changes?

• Yes. Please add a changelog fragment based on our guidelines.
• No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

• Please read our Vector contributor resources.
  • make check-all is a good command to run locally. This check is
    defined here. Some of these
    checks might not be relevant to your PR. For Rust changes, at the very least you should run:
    • cargo fmt --all
    • cargo clippy --workspace --all-targets -- -D warnings
    • cargo nextest run --workspace (alternatively, you can run cargo test --all)
• If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
  run dd-rust-license-tool write to regenerate the license inventory and commit the changes (if any). More details here.

References

Closes: #8516

[bits-bot]

All committers have signed the CLA.

[pront]

Thank you @stackempty!

[fbs]

I noticed your MR. I just opened a similar one the other day (#22470) with a different approach.

If I understand this change correctly, it means you have to put the labels you want to send into the config file, right? My concern with this is that it removes too much of the flexibility labels should give you. You cannot use arbitrary labels anymore but instead have to define everything you want iin the config. This is probably fine if you ship directly to GCP, but if you want to go through an aggregator that becomes impossible to do.
The 'resource.labels' setup also has that issue, but its not that bad as theres only a few types that make sense, and types dictate the resource.labels options

[pront]

Thank you @stackempty!

[pront]

I noticed your MR. I just opened a similar one the other day (#22470) with a different approach.

If I understand this change correctly, it means you have to put the labels you want to send into the config file, right? My concern with this is that it removes too much of the flexibility labels should give you. You cannot use arbitrary labels anymore but instead have to define everything you want iin the config. This is probably fine if you ship directly to GCP, but if you want to go through an aggregator that becomes impossible to do. The 'resource.labels' setup also has that issue, but its not that bad as theres only a few types that make sense, and types dictate the resource.labels options

Hi @fbs, this PR introduces template-able labels, which means they can be populated from events dynamically (or statically in the config). Does this solve the issues with your use case?

[fbs]

I noticed your MR. I just opened a similar one the other day (#22470) with a different approach.
If I understand this change correctly, it means you have to put the labels you want to send into the config file, right? My concern with this is that it removes too much of the flexibility labels should give you. You cannot use arbitrary labels anymore but instead have to define everything you want iin the config. This is probably fine if you ship directly to GCP, but if you want to go through an aggregator that becomes impossible to do. The 'resource.labels' setup also has that issue, but its not that bad as theres only a few types that make sense, and types dictate the resource.labels options

Hi @fbs, this PR introduces template-able labels, which means they can be populated from events dynamically (or statically in the config). Does this solve the issues with your use case?

In this case only label values are templates, right? The key is still specified by the config. And any key must resolve to a value, even if its just an empty string.

My concern is that there are different label keys that make sense for different log entries. E.g. if the log is processed from a kubernetes pod you'd have labels.pod_name, but if its a vm you could have labels.vm_name.
For a central aggregator that processes both types neither is great. Either you have labels with empty values (but with key), or have N sinks

[stackempty]

I noticed your MR. I just opened a similar one the other day (#22470) with a different approach.
If I understand this change correctly, it means you have to put the labels you want to send into the config file, right? My concern with this is that it removes too much of the flexibility labels should give you. You cannot use arbitrary labels anymore but instead have to define everything you want iin the config. This is probably fine if you ship directly to GCP, but if you want to go through an aggregator that becomes impossible to do. The 'resource.labels' setup also has that issue, but its not that bad as theres only a few types that make sense, and types dictate the resource.labels options

Hi @fbs, this PR introduces template-able labels, which means they can be populated from events dynamically (or statically in the config). Does this solve the issues with your use case?

In this case only label values are templates, right? The key is still specified by the config. And any key must resolve to a value, even if its just an empty string.

My concern is that there are different label keys that make sense for different log entries. E.g. if the log is processed from a kubernetes pod you'd have labels.pod_name, but if its a vm you could have labels.vm_name. For a central aggregator that processes both types neither is great. Either you have labels with empty values (but with key), or have N sinks

Hi @fbs,

Thanks for sharing your concern! I have made updates to the PR I believe should address them.

The labels configuration item would remain. This is the standard even with google's own logging agent documented here. Labels specified here will go into the labels field of the LogEntry object.

labels_key option has been added - This will allow folks specify a key from the structured log from which they will be merged into the labels field of the LogEntry object. The default is logging.googleapis.com/labels. This behavior is similar to the google logging agent and fluentbit's stackdriver output plugin.

[stackempty]

@pront would you please give this another look! Thanks!