kafka: add librdkafka dependency, provide upstream kafka client imple…

[adamkotwasinski]

Commit Message: kafka: add librdkafka dependency, provide upstream kafka client implementation for mesh filter
Additional Description:

• add librdkafka dependency
• use librdkafka's producer to send records received by mesh filter
• keep one librdkafka producer per Envoy-worker-thread * upstream-kafka-cluster pair (so adding more worker threads on Envoy gives us more capacity)

Risk Level: Low
Testing: unit tests
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

[repokitteh-read-only]

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #17887 was opened by adamkotwasinski.

see: more, trace.

[repokitteh-read-only]

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).

🐱

Caused by: #17887 was opened by adamkotwasinski.

see: more, trace.

[moderation]

Any chance you can modify the upstream repo so we don't have to carry a patch? OSSF Scorecard results for the new dep below

scorecard --repo=https://github.com/edenhill/librdkafka --show-details
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
|  SCORE  |          NAME          |             REASON             |                                             DETAILS                                              |                         DOCUMENTATION/REMEDIATION                         |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Binary-Artifacts       | binaries present in source     | Warn: binary detected:                                                                           | github.com/ossf/scorecard/blob/main/docs/checks.md#binary-artifacts       |
|         |                        | code                           | mklove/modules/configure.lib                                                                     |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| ?       | Branch-Protection      | could not find branch name     |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#branch-protection      |
|         |                        | master: branch not found       |                                                                                                  |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | CI-Tests               | 30 out of 30 merged PRs        |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#ci-tests               |
|         |                        | checked by a CI test -- score  |                                                                                                  |                                                                           |
|         |                        | normalized to 10               |                                                                                                  |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | CII-Best-Practices     | no badge found                 |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#cii-best-practices     |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 9 / 10  | Code-Review            | GitHub code reviews found for  | Info: Gerrit code reviews                                                                        | github.com/ossf/scorecard/blob/main/docs/checks.md#code-review            |
|         |                        | 28 commits out of the last 30  | found for 0 commits out of                                                                       |                                                                           |
|         |                        | -- score normalized to 9       | the last 30 Info: Prow code                                                                      |                                                                           |
|         |                        |                                | reviews found for 0 commits                                                                      |                                                                           |
|         |                        |                                | out of the last 30                                                                               |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Contributors           | 16 different companies found   | Info: contributors work for:                                                                     | github.com/ossf/scorecard/blob/main/docs/checks.md#contributors           |
|         |                        | -- score normalized to 10      | HitB2018,collaborne,gentoo,NixOS,Exa-Networks,materializeinc,harvard-crimson,jerikan-network,red |                                                                           |
|         |                        |                                | hat,Collaborne,shadow,MaterializeInc,www.matthowlett.com,tvheadend,gentoo-ev,confluentinc        |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Dependency-Update-Tool | no update tool detected        | Warn: dependabot not detected                                                                    | github.com/ossf/scorecard/blob/main/docs/checks.md#dependency-update-tool |
|         |                        |                                | Warn: renovatebot not detected                                                                   |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Fuzzing                | project is fuzzed in OSS-Fuzz  |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#fuzzing                |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Maintained             | 30 commit(s) found in the last |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#maintained             |
|         |                        | 90 days -- score normalized to |                                                                                                  |                                                                           |
|         |                        | 10                             |                                                                                                  |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| ?       | Packaging              | no published package detected  | Warn: no GitHub publishing                                                                       | github.com/ossf/scorecard/blob/main/docs/checks.md#packaging              |
|         |                        |                                | workflow detected                                                                                |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| ?       | Pinned-Dependencies    | invalid shell code: 19:1:      | Warn: no lock files detected                                                                     | github.com/ossf/scorecard/blob/main/docs/checks.md#pinned-dependencies    |
|         |                        | "done" can only be used to end | for a package manager                                                                            |                                                                           |
|         |                        | a loop                         | Info: GitHub actions are                                                                         |                                                                           |
|         |                        |                                | pinned Info: Dockerfile                                                                          |                                                                           |
|         |                        |                                | dependencies are pinned                                                                          |                                                                           |
|         |                        |                                | Info: no insecure (unpinned)                                                                     |                                                                           |
|         |                        |                                | dependency downloads found                                                                       |                                                                           |
|         |                        |                                | in Dockerfiles Warn: insecure                                                                    |                                                                           |
|         |                        |                                | (unpinned) download detected                                                                     |                                                                           |
|         |                        |                                | in tests/autotest.sh: 'pip3                                                                      |                                                                           |
|         |                        |                                | install -U trivup'                                                                               |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | SAST                   | SAST tool is not run on all    | Warn: 0 commits out of 30 are                                                                    | github.com/ossf/scorecard/blob/main/docs/checks.md#sast                   |
|         |                        | commits -- score normalized to | checked with a SAST tool Warn:                                                                   |                                                                           |
|         |                        | 0                              | CodeQL tool not detected                                                                         |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 0 / 10  | Security-Policy        | security policy file not       |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#security-policy        |
|         |                        | detected                       |                                                                                                  |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| ?       | Signed-Releases        | no releases found              | Warn: no GitHub releases found                                                                   | github.com/ossf/scorecard/blob/main/docs/checks.md#signed-releases        |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Token-Permissions      | tokens are read-only in GitHub |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions      |
|         |                        | workflows                      |                                                                                                  |                                                                           |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|
| 10 / 10 | Vulnerabilities        | no vulnerabilities detected    |                                                                                                  | github.com/ossf/scorecard/blob/main/docs/checks.md#vulnerabilities        |
|---------|------------------------|--------------------------------|--------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------|

[repokitteh-read-only]

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #17887 (comment) was created by @adamkotwasinski.

see: more, trace.

[adamkotwasinski]

Managed to get librdkafka compiled without needing a patch.
/wait-any

[adamkotwasinski]

All green.