Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy protocol: new feature auto-detect proxy protocol #18951

Merged
merged 61 commits into from
May 6, 2022
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
278ce7e
Auto-detect proxy protocol
Nov 9, 2021
121e87d
Fix format
Nov 10, 2021
2d43ae3
Merge branch 'auto_detect_proxy_protocol' of github.com:solo-io/envoy…
Nov 10, 2021
de6b7f2
Fix format
Nov 10, 2021
3a36695
Merge branch 'main' into auto_detect_proxy_protocol
Nov 10, 2021
409f94c
Fix merge
Nov 10, 2021
ae19e1f
Clearer error message
Nov 10, 2021
ed4fb54
Add unit test for partial v1 protocol
Nov 10, 2021
bcc7aef
Rename new field to
Nov 10, 2021
8c87bcd
Fix format
Nov 10, 2021
4aa520e
Minor tweak to version history note
Nov 10, 2021
91c9a1b
Use attention clause in API comment
Nov 10, 2021
c45ecc7
Set in ctor initialization list
Nov 10, 2021
4259f1a
Add comment to clarify test
Nov 10, 2021
2b4a89a
Return value was only used in one of two return paths, tiny refactor
Nov 10, 2021
f105c3d
Fix format
Nov 10, 2021
6a0d49b
Test case with partial signature, just char
Nov 10, 2021
f47232f
Use standard library for min
Nov 10, 2021
fa9746a
PR comments; fix bug with large requests being allowed
Nov 10, 2021
a37ad43
Assert msg size in tests in addition to noting in comment
Nov 10, 2021
6f19883
Do not allow any false positives
Nov 10, 2021
0927000
Only check if proxy protocol on initial MSG_PEEK
Nov 11, 2021
b5cbfc1
Don't check too many bytes on v1
Nov 11, 2021
a7ef68f
Revert "Don't check too many bytes on v1"
Nov 11, 2021
9495853
Merge branch 'main' into auto_detect_proxy_protocol
Dec 16, 2021
ffb56d1
Merge branch 'main' into auto_detect_proxy_protocol
Apr 28, 2022
ac2982f
Fix bad merge on current.rst
Apr 28, 2022
2a66a01
Formatting
Apr 28, 2022
c9e086f
Wait for more bytes if we can't tell
Apr 29, 2022
af6f88d
Update API comment to reflect waiting behavior
Apr 29, 2022
7f17660
Formatting
Apr 29, 2022
59206da
ReadOrParseState is not always an error if not Done
Apr 29, 2022
b416242
Improve comments
Apr 29, 2022
4e4638a
Formatting
Apr 29, 2022
354a365
Better test for waiting for more bytes
Apr 29, 2022
86ad288
More accurate comment
Apr 29, 2022
899f199
We can wait for less bytes for v1 proxy protocol
Apr 29, 2022
887d0bc
Fix spelling error
Apr 30, 2022
a23ac5f
Reuse previous parsing logic; split checks on allowRequestsWithoutPro…
Apr 30, 2022
6075b67
Revert "Reuse previous parsing logic; split checks on allowRequestsWi…
Apr 30, 2022
4d6f75f
Add v2 partial tests
Apr 30, 2022
2cb4526
Reuse existing parsing logic
May 2, 2022
1667461
Remove whitespace in test
May 2, 2022
a0c977f
Move check to else statement
May 2, 2022
85706e6
Clarify what tests are covering
May 2, 2022
c86edef
Add another test and fix for v2 followed by v1 signature bytes
May 2, 2022
6031151
Add tests to ensure we disconnect invalid requests after valid signature
May 2, 2022
a09bb1b
Use capital letters
May 2, 2022
cfeda5a
Use capital letters
May 2, 2022
5a660a0
Use const
May 2, 2022
e92a8a4
Do all checking in one place up front
May 2, 2022
84cdc32
Do all checking in one place up front; remove unneeded check
May 2, 2022
f3d65e9
Merge branch 'main' into auto_detect_proxy_protocol
May 2, 2022
b2d467e
Merge branch 'main' into auto_detect_proxy_protocol
May 3, 2022
45e9c24
Merge branch 'main' into auto_detect_proxy_protocol
May 3, 2022
056f74f
Add changelog
May 3, 2022
39e7b50
Remove current.rst file that was added in bad merge
May 3, 2022
392124e
PR comments
May 4, 2022
307df1c
Merge branch 'main' into auto_detect_proxy_protocol
May 4, 2022
5c50e23
Ensure every case is handled in switch statement
May 5, 2022
36bbfa2
Merge branch 'main' into auto_detect_proxy_protocol
May 5, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -40,4 +40,9 @@ message ProxyProtocol {

// The list of rules to apply to requests.
repeated Rule rules = 1;

// NOTICE: only enable if ALL traffic to the listener comes from a trusted source.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably in an .. attention:: clause.
See example.

I also suggest to rephrase the comment, starting with with it does, its default value, and then the notice part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the example, addressed in 91c9a1b

// Defaults to false. If true, attempt to detect proxy protocol if present, and allow
// requests through if proxy protocol is not used on the connection.
bool detect_proxy_protocol = 2;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel the major goal is allow the requests through if proxy protocol is not used. And actually the whole proxy protocol filter is about detect the proxy protocol, so this option name feels confuse. Should we call it something like allow_requests_with_proxy_protocol?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 please clarify the name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that better as well, addressed in bcc7aef

}
1 change: 1 addition & 0 deletions docs/root/version_history/current.rst
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ New Features
* listener: added support for :ref:`MPTCP <envoy_v3_api_field_config.listener.v3.Listener.enable_mptcp>` (multipath TCP).
* oauth filter: added :ref:`cookie_names <envoy_v3_api_field_extensions.filters.http.oauth2.v3.OAuth2Credentials.cookie_names>` to allow overriding (default) cookie names (``BearerToken``, ``OauthHMAC``, and ``OauthExpires``) set by the filter.
* oauth filter: setting IdToken and RefreshToken cookies if they are provided by Identity provider along with AccessToken.
* proxy protocol: added support for auto-detecting proxy protocol on the listener from trusted downstreams as an opt-in flag.
* tcp: added a :ref:`FilterState <envoy_v3_api_msg_type.v3.HashPolicy.FilterState>` :ref:`hash policy <envoy_v3_api_msg_type.v3.HashPolicy>`, used by :ref:`TCP proxy <envoy_v3_api_field_extensions.filters.network.tcp_proxy.v3.TcpProxy.hash_policy>` to allow hashing load balancer algorithms to hash on objects in filter state.
* tcp_proxy: added support to populate upstream http connect header values from stream info.
* thrift_proxy: add header to metadata filter for turning headers into dynamic metadata.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Config::Config(
Stats::Scope& scope,
const envoy::extensions::filters::listener::proxy_protocol::v3::ProxyProtocol& proto_config)
: stats_{ALL_PROXY_PROTOCOL_STATS(POOL_COUNTER(scope))} {
detect_proxy_protocol_ = proto_config.detect_proxy_protocol();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be set in the c'tor's member initialization list above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in c45ecc7

for (const auto& rule : proto_config.rules()) {
tlv_types_[0xFF & rule.tlv_type()] = rule.on_tlv_present();
}
Expand All @@ -63,6 +64,8 @@ const KeyValuePair* Config::isTlvTypeNeeded(uint8_t type) const {

size_t Config::numberOfNeededTlvTypes() const { return tlv_types_.size(); }

bool Config::detectProxyProtocol() const { return detect_proxy_protocol_; }

Network::FilterStatus Filter::onAccept(Network::ListenerFilterCallbacks& cb) {
ENVOY_LOG(debug, "proxy_protocol: new connection accepted");
Network::ConnectionSocket& socket = cb.socket();
Expand All @@ -77,11 +80,20 @@ Network::FilterStatus Filter::onAccept(Network::ListenerFilterCallbacks& cb) {
return Network::FilterStatus::StopIteration;
}

ReadOrParseState Filter::resetAndContinue(Network::IoHandle& io_handle) {
// Release the file event so that we do not interfere with the connection read events.
io_handle.resetFileEvents();
cb_->continueFilterChain(true);
return ReadOrParseState::Done;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method doesn't need to return anything, and the returned status should probably be only at the end of Filter::onReadWorker()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree, addressed in 2b4a89a

}

void Filter::onRead() {
const ReadOrParseState read_state = onReadWorker();
if (read_state == ReadOrParseState::Error) {
config_->stats_.downstream_cx_proxy_proto_error_.inc();
cb_->continueFilterChain(false);
} else if (read_state == ReadOrParseState::SkipFilterError) {
resetAndContinue(cb_->socket().ioHandle());
}
}

Expand Down Expand Up @@ -135,10 +147,7 @@ ReadOrParseState Filter::onReadWorker() {
proxy_protocol_header_.value().remote_address_);
}

// Release the file event so that we do not interfere with the connection read events.
socket.ioHandle().resetFileEvents();
cb_->continueFilterChain(true);
return ReadOrParseState::Done;
return resetAndContinue(socket.ioHandle());
}

absl::optional<size_t> Filter::lenV2Address(char* buf) {
Expand Down Expand Up @@ -309,7 +318,7 @@ ReadOrParseState Filter::parseExtensions(Network::IoHandle& io_handle, uint8_t*
const auto recv_result = io_handle.recv(buf, to_read, 0);
if (!recv_result.ok()) {
if (recv_result.err_->getErrorCode() == Api::IoError::IoErrorCode::Again) {
return ReadOrParseState::TryAgainLater;
return ReadOrParseState::TryAgainLaterError;
}
ENVOY_LOG(debug, "failed to read proxy protocol (no bytes avail)");
return ReadOrParseState::Error;
Expand Down Expand Up @@ -424,7 +433,7 @@ ReadOrParseState Filter::readProxyHeader(Network::IoHandle& io_handle) {

if (!result.ok()) {
if (result.err_->getErrorCode() == Api::IoError::IoErrorCode::Again) {
return ReadOrParseState::TryAgainLater;
return ReadOrParseState::TryAgainLaterError;
}
ENVOY_LOG(debug, "failed to read proxy protocol (no bytes read)");
return ReadOrParseState::Error;
Expand All @@ -434,6 +443,9 @@ ReadOrParseState Filter::readProxyHeader(Network::IoHandle& io_handle) {
if (nread < 1) {
ENVOY_LOG(debug, "failed to read proxy protocol (no bytes read)");
return ReadOrParseState::Error;
} else if (nread < PROXY_PROTO_V2_HEADER_LEN && config_.get()->detectProxyProtocol()) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the client send a partial v1 protocol header, I feel this will skip the process the v1 protocol header. it would be great to add an unitest for this case.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the feedback, addressed in ed4fb54

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ENVOY_LOG(debug, "need more bytes to detect proxy protocol header");
return ReadOrParseState::SkipFilterError;
}

if (buf_off_ + nread >= PROXY_PROTO_V2_HEADER_LEN) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,22 @@ class Config : public Logger::Loggable<Logger::Id::filter> {
*/
size_t numberOfNeededTlvTypes() const;

/**
* Filter configuration that determines if we should pass-through failed proxy protocol
* requests. Should only be configured to true for trusted downstreams.
*/
bool detectProxyProtocol() const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name here should also be clear (following the api field name change).


private:
absl::flat_hash_map<uint8_t, KeyValuePair> tlv_types_;
bool detect_proxy_protocol_{};
};

using ConfigSharedPtr = std::shared_ptr<Config>;

enum ProxyProtocolVersion { Unknown = -1, InProgress = -2, V1 = 1, V2 = 2 };

enum class ReadOrParseState { Done, TryAgainLater, Error };
enum class ReadOrParseState { Done, TryAgainLaterError, Error, SkipFilterError };
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did you rename TryAgainLater?

I think SkipFilterError should be renamed to SkipFilter. It isn't an error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addressed in fa9746a


/**
* Implementation the PROXY Protocol listener filter
Expand Down Expand Up @@ -98,7 +105,7 @@ class Filter : public Network::ListenerFilter, Logger::Loggable<Logger::Id::filt
/**
* Helper function that attempts to read the proxy header
* (delimited by \r\n if V1 format, or with length if V2)
* @return bool true valid header, false if more data is needed or socket errors occurred.
* @return ReadOrParseState Done if successfully parsed, or the error type.
*/
ReadOrParseState readProxyHeader(Network::IoHandle& io_handle);

Expand All @@ -118,6 +125,8 @@ class Filter : public Network::ListenerFilter, Logger::Loggable<Logger::Id::filt
bool parseV2Header(char* buf);
absl::optional<size_t> lenV2Address(char* buf);

ReadOrParseState resetAndContinue(Network::IoHandle& io_handle);

Network::ListenerFilterCallbacks* cb_{};

// The offset in buf_ that has been fully read
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,19 @@ TEST_P(ProxyProtocolTest, V1Basic) {
disconnect();
}

TEST_P(ProxyProtocolTest, DetectNoProxyProtocol) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment on what the test is validating.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed in 4259f1a, please let me know if this doesn't clarify what might have been confusing


envoy::extensions::filters::listener::proxy_protocol::v3::ProxyProtocol proto_config;
proto_config.set_detect_proxy_protocol(true);
connect(true, &proto_config);

write("more data");

expectData("more data");

disconnect();
}

TEST_P(ProxyProtocolTest, V1Minimal) {
connect();
write("PROXY UNKNOWN\r\nmore data");
Expand Down