Skip to content

Second draft: configmiddleware and extensionlimiter #12633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config/configmiddleware/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
include ../../Makefile.Common
57 changes: 57 additions & 0 deletions config/configmiddleware/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Middleware configuration

**Status: development**

This module defines necessary interfaces to configure middleware
extensions for use in components. Multiple kinds of middleware may be
defined, however middlewares are configured through a single list, and
middleware components are invoked in order. Specific kinds of
middleware are be configured as specific extension types, and more
than one extension type is supported:

- `Limiter`: a weighted protocol-agnostic form of middleware, meant
for use in receivers. To implement a limiter extension, components
should implement the interface in
[`extension/extensionlimiter`](#../../extension/extensionlimiter/README.md).
- `Interceptor`: a protocol-specific form of middleware, may be client
or server, stream or unary. To implement an interceptor extension,
components should implement the interface in
[`extension/extensioninterceptor`](#../../extension/extensioninterceptor/README.md).

Middleware is included in the basic HTTP and gRPC Server Config
structs, making it so that exporters and receivers in push-based
protocols automatically through `confighttp` and `configgrpc`.

The currently known extensions are listed below.

## Limiter implementations

- [Memory Limiter Extension](../../extension/memorylimiterextension/README.md)
- [Admission Limiter Extension](../../extension/admissionlimiterextension/README.md)

Example:

```yaml
extensions:
# Used with gRPC traffic, consults GC statistics.
memory_limiter/cold
request_limit_mib: 100
waiting_limit_mib: 10

# Used with HTTP traffic, counts request bytes in flight.
admission_limiter/warm:
request_limit_mib: 10
waiting_limit_mib: 10

receivers:
otlp:
protocols:
http:
# ...
middleware:
- limiter: admission_limiter/warm
grpc:
# ...
middleware:
- limiter: memory_limiter/cold
```
70 changes: 70 additions & 0 deletions config/configmiddleware/configmiddleware.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

// Package configauth implements the configuration settings to
// ensure authentication on incoming requests, and allows
// exporters to add authentication on outgoing requests.
package configmiddleware // import "go.opentelemetry.io/collector/config/configmiddleware"

import (
"context"
"fmt"

"go.opentelemetry.io/collector/component"
"go.opentelemetry.io/collector/extension/extensionlimiter"
)

var (
errExactlyOneID = fmt.Errorf("middleware must configure a single one component")
errNotALimiter = fmt.Errorf("middleware must configure a single one component")
errMiddlewareNotFound = fmt.Errorf("middleware not found")

emptyID = component.ID{}
)

// Middleware defines the a middleware component. Middleware can be more than
// one type of extension. Each extension uses a single field, and exactly one
// field should be set.
//
// The typical way Middleware configuration is used will be in a list contained
// within another configuration object. For example, the `configgrpc.ServerConfig`
// inside the OTLP receiver lists middleware:
//
// otlp:
// protocols:
// grpc:
// middleware:
// - interceptor: blocker
// - limiter: admission
// - interceptor: decorator
type Middleware struct {
// LimiterID specifies the name of a limiter extension to be used.
LimiterID component.ID `mapstructure:"limiter,omitempty"`

// InterceptorID specifies the name of an interceptor extension to be used.
InterceptorID component.ID `mapstructure:"interceptor,omitempty"`
Comment on lines +41 to +45

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to distinguish between them? Can we have only one ID, and when we check for that extension, if it is a limiter we "convert" it to interceptor?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about the case of a scraper-type receiver, which is able to call a limiter in various ways but cannot be configured with interceptors. Then, I would expect to see

receivers:
  filescraper: # it reads and re-reads a file, so neither gRPC or HTTP interceptors apply
    limiters: # config type is []configmiddleware.Middleware
      - limiter: xyz
      - limiter: abc

However, it would be an error to specify an interceptor-type (e.g., - interceptor: decorator) in this context. That's why I was thinking the distinction would be helpful.

The scraper would be able to request from limiter(s) for request count, item count, and byte-size quantities.

Yes, if we request an interceptor and get a limiter, there can be an automatic conversion. However, if you request a limiter and get an interceptor, it can simply be an error.

}

// Validate ensures that exactly one IDis set.
func (m Middleware) Validate() error {
hasLim := m.LimiterID.String() != ""
hasInt := m.InterceptorID.String() != ""
if hasLim == hasInt {
return errExactlyOneID
}
return nil
}

// GetLimiter requests to locate a named limiter extension.
func (m Middleware) GetLimiter(_ context.Context, extensions map[component.ID]component.Component) (extensionlimiter.Limiter, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you consider creating a middleware implementation that uses a limiter, rather than coupling middleware to the limiter interface? I mean, as an alternative, we could introduce a package like extension/limitermiddlewareextension. Then you would configure the collector like:

extensions:
  # Used with gRPC traffic, consults GC statistics.
  memory_limiter/cold
    request_limit_mib: 100
    waiting_limit_mib: 10

  # Used with HTTP traffic, counts request bytes in flight.
  admission_limiter/warm:
    request_limit_mib: 10
    waiting_limit_mib: 10

  limitermiddleware/memory:
    limiter: memory_limiter_cold

  limitermiddleware/admission:
    limiter: admission_limiter/warm

receivers:
  otlp:
    protocols:
      http:
        # ...
        middleware:
           - limitermiddleware/admission
      grpc:
        # ...
        middleware:
          - limitermiddleware/memory

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@axw I like this idea for the way it separates limiters and middleware.

I've referred to the case of otelarrow receiver, which uses both gRPC unary (for OTLP) and gRPC stream (for OTAP). I would use a limitermiddleware at the gRPC level for the unary case, and for the stream case I would call the configured limiter directly.

if m.LimiterID == emptyID {
return nil, errNotALimiter
}
if ext, found := extensions[m.LimiterID]; found {
if ext, ok := ext.(extensionlimiter.Limiter); ok {
return ext, nil
}
return nil, errNotALimiter
}
return nil, fmt.Errorf("failed to resolve limiter %q: %w", m.LimiterID, errMiddlewareNotFound)
}
35 changes: 35 additions & 0 deletions config/configmiddleware/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
module go.opentelemetry.io/collector/config/configmiddleware

go 1.23.0

require (
go.opentelemetry.io/collector/component v1.27.0
go.opentelemetry.io/collector/extension/extensionlimiter v1.27.0
)

require (
github.com/gogo/protobuf v1.3.2 // indirect
go.opentelemetry.io/collector/extension v1.27.0 // indirect
go.opentelemetry.io/collector/pdata v1.27.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/text v0.21.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f // indirect
google.golang.org/grpc v1.71.0 // indirect
google.golang.org/protobuf v1.36.5 // indirect
)

replace go.opentelemetry.io/collector/pdata => ../../pdata

replace go.opentelemetry.io/collector/component => ../../component

replace go.opentelemetry.io/collector/component/componenttest => ../../component/componenttest

replace go.opentelemetry.io/collector/extension => ../../extension

replace go.opentelemetry.io/collector/extension/extensionlimiter => ../../extension/extensionlimiter
79 changes: 79 additions & 0 deletions config/configmiddleware/go.sum

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

40 changes: 40 additions & 0 deletions extension/extensionlimiter/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# Limiter extension

The limiter extension interface supports components that limit
requests by several measures of weight. Extension components are able
to limit entry to pipeline by the number of requests, the
(signal-specific) number of items, and/or the number of bytes of
network and memory consumed as they transit the pipeline.

The limiter interface supports a single method, `Acquire(ctx,
settings, weight)` accepting context, settings, and weight
parameters.

## Limiter context

The context passed to a Limiter is the one created following the Auth
extension, and it includes the `client.Metadata` of the
request.

## Limiter settings

Secondary information, including the signal kind of the request, is
included in the settings. Other information may be included in this
struct in the future, for example the ID of the calling component.

## Limiter weight

The `Weight` struct includes three fields for three different ways to
limit requests, including by request count, by item count, and by
bytes.

The weight `Bytes` field is particularly important for memory-based
limiters. For measuring bytes, components are encouraged to be
approximate. The goal is to estimate the amount of actual memory that
will be used by pipeline data as the request is in transit, so
components may use the uncompressed size of the data as a proxy for
the amount of memory that will be used. The corresponding pipeline
data `Sizer` class may be used.

In cases where components allocate memory in multiple steps, they can
make multiple calls to Acquire to report additional bytes used.
33 changes: 33 additions & 0 deletions extension/extensionlimiter/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
module go.opentelemetry.io/collector/extension/extensionlimiter

go 1.23.0

require (
go.opentelemetry.io/collector/component v1.27.0
go.opentelemetry.io/collector/extension v1.27.0
)

require (
github.com/gogo/protobuf v1.3.2 // indirect
go.opentelemetry.io/collector/pdata v1.27.0 // indirect
go.opentelemetry.io/otel v1.35.0 // indirect
go.opentelemetry.io/otel/metric v1.35.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.35.0 // indirect
go.opentelemetry.io/otel/trace v1.35.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/net v0.34.0 // indirect
golang.org/x/sys v0.30.0 // indirect
golang.org/x/text v0.21.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20250115164207-1a7da9e5054f // indirect
google.golang.org/grpc v1.71.0 // indirect
google.golang.org/protobuf v1.36.5 // indirect
)

replace go.opentelemetry.io/collector/component => ../../component

replace go.opentelemetry.io/collector/component/componenttest => ../../component/componenttest

replace go.opentelemetry.io/collector/extension => ../

replace go.opentelemetry.io/collector/pdata => ../../pdata
Loading
Loading