Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dremio 25.0 12.0 project id merge #64

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
204 commits
Select commit Hold shift + click to select a range
1779601
Add Session management messages, Location URI path accessors
jduo Dec 2, 2022
3594d38
Address PR feedback
Apr 3, 2023
9cc353f
Remove non-const references
Apr 16, 2023
4ecad66
Add partial (some gaps) ServerSessionMiddleware impl
May 17, 2023
647631d
Initial StartCall impl, etc.
Jun 14, 2023
cff7886
Prelim impl ServerSessionMWFactory locking
Jun 16, 2023
4ae40cd
Untested initial middleware impl
Jun 22, 2023
f43ee71
Rename lock_ and remove duplicitous comments
Jun 23, 2023
c0000d1
Make code build
Jun 23, 2023
f0ff5ed
Fix server set-cookie header name
Jun 23, 2023
9b2fda9
Remove/revert abandoned Location changes
Jun 28, 2023
1402210
[BUILDS] Move Session messages to Flight PB
Jul 13, 2023
772a759
Change Proto to use map of Session options
Jul 14, 2023
3249107
FSQL Client: Use SessionOptions PB Map
Jul 14, 2023
a22f3f8
FSQL Server: Use SessionOptions PB Map (incomplete?)
Jul 14, 2023
1f6b725
[Builds] Fix build errors
Jul 18, 2023
4fe1c01
[BUILDS] Use visitors to convert SessionOptionValue->Proto
Jul 24, 2023
0b23f8e
Change to string_view for interfaces
Aug 10, 2023
1eedcdc
WIP: High-level design review ONLY
Oct 22, 2023
003d931
WIP: impl *::Deserialize()
Oct 30, 2023
67acc78
WIP: ~finish flight/types.cc & serialization_internal.h
Oct 31, 2023
ba83605
WIP: Update some temporary comments
Oct 31, 2023
fb9ff41
Impl FromProto(SessionOptionValue...)
Nov 1, 2023
0ca0845
Random WIP checkpoint
Nov 1, 2023
ffc539b
WIP: Finished serialization_internal.cc ?
Nov 1, 2023
f5caacc
WIP: Remove 'Action' prefix from Proto messages
Nov 1, 2023
51798e7
Reduce long Proto nested enum names
Nov 1, 2023
ac1790d
Indentation etc.
Nov 1, 2023
805a269
Update to 'Status' enums in Proto
Nov 1, 2023
72f6e8d
WIP: More replacement of Result with Status for enums
Nov 1, 2023
10b9296
WIP: Even more replacement of Result with Status for enums
Nov 1, 2023
d5c8e6c
WIP: Even MORE replacement of Result with Status for enums
Nov 1, 2023
a7df3bb
Formatting & remove erroneous 'static'
Nov 1, 2023
01b2723
Typos
Nov 1, 2023
3eede7f
Add new SetSessionOption statuses to Proto
Nov 1, 2023
d28327b
Fixup flight/client.*
Nov 1, 2023
2dcc16e
Fix name change + remove temporary comments
Nov 1, 2023
3b6c7f2
Move lambda overload helpers around & other cleanup
Nov 1, 2023
a85b69b
Fixed all but 1 build error
Nov 2, 2023
67be21a
Fix last build issue
Nov 2, 2023
26f37e9
[builds] Remote extraneous comments
Nov 2, 2023
5862c66
[builds] Some PR fixups
Nov 2, 2023
938d124
[builds] More PR feedback
Nov 2, 2023
b4c542c
[builds] More PR feedback
Nov 2, 2023
fad6511
[builds] Fix linter errors
Nov 2, 2023
e3d701a
[builds] clang-format
Nov 2, 2023
07f48f8
[builds] Add missing include
Nov 2, 2023
c47a515
[builds] Fix clang-format{14,17} differences
Nov 3, 2023
1b086ad
[builds] Add missed change from previous
Nov 3, 2023
a348231
[builds] C++ docs fixup
Nov 3, 2023
dfcc785
Add Protobuf docstrings & rename Flight session cookie
Nov 3, 2023
9876288
[builds] Remove unused Proto import
Nov 3, 2023
6da407c
Fixes for @stevelorddremio's feedback
Nov 9, 2023
d939742
Revert extraneous linter changes
Nov 9, 2023
6333801
More PR feedback resolution
Nov 9, 2023
a11c264
More PR feedback
Nov 10, 2023
b8764d0
More PR feedback
Nov 10, 2023
6d392af
[builds] More PR fixups
Nov 10, 2023
ee210b7
PR feedback
Nov 10, 2023
b726aec
PR feedback
Nov 10, 2023
88317b8
PR feedback
Nov 10, 2023
2057c52
Refactor to facilitate proper make_shared
Nov 10, 2023
05b88ea
[builds] Finish refactor to use map.find()
Nov 10, 2023
5ff1da6
Remove extraneous using namespace
Nov 10, 2023
261e3f6
[builds] Factor out session ID generation & remove Boost
Nov 10, 2023
e2bcdac
[builds] make GetNewSession return std::pair
Nov 10, 2023
302728e
[builds] Fix argument forwarding for FlightSqlSession::SetSessionOption
Nov 10, 2023
1622e15
[builds] Docs fixup
Nov 10, 2023
a3ebd8b
[builds] move SetSessionOptionsResult status to estensible nested msg
Nov 15, 2023
70e2ba4
[builds] Remove Result<> wrapper from GetSessionOption
Nov 15, 2023
29b063c
[builds] PR feedback fixups
Nov 15, 2023
3337f28
[builds] PR feedback fixups
Nov 15, 2023
922d148
[builds] Add virtual default destructor to ServerSessionMiddleware
Nov 15, 2023
6286f66
[builds] Fix CI linter issues.
Nov 15, 2023
8964cec
[builds] Fix abstract subclass ServerSessionMiddleware.
Nov 15, 2023
192b768
[builds] WHOOPS: Fix tangled/redundant Session ActionTypes
Nov 15, 2023
2776624
[builds] CI fixups
Nov 16, 2023
e403712
[builds] Remove redundant branch
Nov 16, 2023
2011f51
[builds] Fix reader/writer locks
Nov 17, 2023
909f41e
[builds] Remove extraneous include
Nov 17, 2023
7ba0f12
Apply @kou suggestions from code review
indigophox Nov 17, 2023
dd067f0
[builds] More @kou feedback integrated
Nov 17, 2023
021803e
[builds] Fix docs
Nov 20, 2023
62367e0
[builds] PR feedback
Nov 21, 2023
cbba7a8
[builds] Return {"",NULLPTR} on id generation collision
Nov 21, 2023
36e3d00
[builds] linter
Nov 21, 2023
b6cfdf2
[builds] copy to proto repeated string value using iterator
Nov 21, 2023
3f8e0bf
[builds] (partial?) fix for ParseCookieString, pending testing
Nov 21, 2023
587a4ca
[builds] fixup
Nov 21, 2023
1b9e097
[builds] fixup
Nov 21, 2023
2e2e46c
[builds] grrrr CI grrrr
Nov 21, 2023
bc2a9a5
[builds] Revert iterator-copy not supported by CI PB libs
Nov 21, 2023
de43e5c
Add some testing
Nov 22, 2023
e225b3b
oops
Nov 22, 2023
063959d
grrr linter
Nov 22, 2023
3db666e
grrr linter
Nov 22, 2023
9c62bcd
grrrr linter
Nov 22, 2023
18131fe
grrrr linter
Nov 23, 2023
20df8ac
[builds] Remove extraneous SessionOptions type tests
Nov 23, 2023
20c055c
Typo fix
Nov 29, 2023
503b727
[Java WIP] Add ActionTypes & some SessionOptionValue types
Nov 29, 2023
401cf00
+some docs
Nov 29, 2023
60e9af6
+some docs
Nov 29, 2023
d93670b
+some docs
Nov 29, 2023
98b1bce
Add some partially/fully implemented POJO message types
Dec 1, 2023
6500504
Add method skeleton
Dec 2, 2023
1842779
WIP: Java: add serde + POJO classes
Dec 6, 2023
24e7feb
Fixup
Dec 6, 2023
809a0ce
Change SetSessionOptionResult (Proto) to error-only behaviour
Dec 8, 2023
cfdaae0
[builds] Update C++ to error-only SetSessionOptionsResult
Dec 8, 2023
ac5cbd6
WIP: pseudo-complete Java impl
Dec 9, 2023
677c2d7
Fix some checkstyle errors
Dec 9, 2023
954b395
Fix some more checkstyle errors
Dec 9, 2023
e17ac4f
checkpoint
Dec 10, 2023
1e81385
checkpoint
Dec 10, 2023
3cd4d0e
checkpoint - should be working but...
Dec 10, 2023
6d53d62
[builds] Prelim Java implementation
Dec 10, 2023
00f60b1
[builds] Prelim Java implementation
Dec 10, 2023
e8b9bf7
[Java WIP]: Add nullable SessionOptionValue
Dec 12, 2023
3fe59e7
fixup
Dec 12, 2023
d501f89
[C++][builds] Add unset SessionOptionValue functionality
Dec 12, 2023
2e8a5e5
[C++ builds][Java builds] typos/etc
Dec 12, 2023
1eb10a8
Add Proto comment re: nullability
Dec 12, 2023
2e5bc08
[C++][builds] Add braces
Dec 13, 2023
9a48343
[C++][builds] Tiny refactor
Dec 13, 2023
f47ed55
Update Proto comments re: HTTP-cookies-or-similar
Dec 13, 2023
ac23fd4
checkpoint
Dec 14, 2023
7eec98f
[Java builds] fixups
Dec 18, 2023
7b5feaf
[Java builds] Add ServerSessionMiddleware.java
Dec 18, 2023
f5a4141
Fixup
Dec 23, 2023
ebe2bc0
checkpoint
Jan 2, 2024
915ad4e
checkpoint
Jan 2, 2024
f6307f2
checkpoint
Jan 3, 2024
018e5a2
checkpoint
Jan 3, 2024
eea08f4
[Java builds] Add initial Java integration tests
Jan 3, 2024
84743d4
[C++] clang-format fixups
Jan 3, 2024
5d2a3f2
[C++] clang-format fixup
Jan 3, 2024
9c63d6b
[Java builds] Add and fix initial working Java integration test
Jan 5, 2024
295f245
[Java builds] Expand SessionOptionsProducer to facilitate more tests
Jan 10, 2024
4c74ba7
WIP
Jan 11, 2024
180d612
checkpoint
Jan 11, 2024
130150e
[Java builds & passes] fix up integration tests & related code
Jan 11, 2024
42cba18
[Java builds & passes] Fix & add Unicode integration test coverage
Jan 11, 2024
84c16d8
[Java builds & passes] ServerSessionMiddleware.CloseSession: expire c…
Jan 11, 2024
830977a
[Java builds & passes] Finish Java integration tests
Jan 12, 2024
ea25c4a
[C++ builds] Factor ServerSessionMiddleware to separate header for te…
Jan 12, 2024
3ad33b8
[temp] Add dummy C++ unit test
Jan 15, 2024
3be7652
[temp][fixup] Add dummy C++ unit test
Jan 15, 2024
f2e3663
[temp][linter fixup] Add dummy C++ unit test
Jan 15, 2024
edbbae8
Fix Java build errors & dependencies
Jan 15, 2024
c2b4dc6
[Java builds][C++ builds] Extricate int32 & float
Jan 16, 2024
6bcee0b
[Java builds][C++ builds] Add NoOpSessionOptionValueVisitor
Jan 16, 2024
05f9564
Test adding Flight C++ unit test
Jan 16, 2024
2da5666
Test #2 adding Flight C++ unit test
Jan 16, 2024
d172187
Test #3 adding Flight C++ unit test
Jan 16, 2024
5dd4c47
Test #4 adding Flight C++ unit test
Jan 16, 2024
d8546c1
Test #5 adding Flight C++ unit test
Jan 16, 2024
8ea8d05
[C++ builds] added tautological failure to arrow-flight-sql-test
Jan 16, 2024
788bb30
[C++ builds] Add C++ ServerSessionMiddleware cookie parsing test
Jan 17, 2024
0799cda
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Jan 18, 2024
803a679
GH-37795: [Java][FlightSQL] Add mock FlightSqlProducer and tests (#37…
jduo Sep 25, 2023
31cf166
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Jan 19, 2024
2b478fe
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Jan 22, 2024
9059298
[C++ builds] Probably fix Win64 C++ CI build linker error
Jan 19, 2024
619f0ea
Revert "[C++ builds] Probably fix Win64 C++ CI build linker error"
Jan 19, 2024
2bb6e0b
[C++ builds] CI linter fixups
Jan 20, 2024
5a188ad
[C++ builds] Linter + linker fixups
Jan 20, 2024
43e02c9
[Docs] Doc fixup
Jan 20, 2024
6878570
[C++ builds][Java builds] PR feedback etc
Jan 21, 2024
1f95c2c
Guessed workaround for opaque CMake linter error
Jan 21, 2024
1fe109b
Remove extraneous whitespace in CMakeLists.txt
Jan 21, 2024
5b85167
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Jan 24, 2024
abae23b
[C++ builds] checkpoint
Jan 22, 2024
4004d0a
[C++ builds] checkpoint
Jan 22, 2024
e595616
[C++ builds] checkpoint
Jan 22, 2024
da0b841
[C++ builds] checkpoint
Jan 22, 2024
8e523a1
[C++ builds] BUG CHECKPOINT
Jan 22, 2024
b4cdf29
[C++ builds] Add session cookie expiry
Jan 22, 2024
81a4254
[Java builds] Fix NoOpSessionOptionValueVisitor iface
Jan 23, 2024
5d4162a
[C++ builds & passes] Bodge around middleware gap
Jan 24, 2024
c4fe980
[C++ builds] remove debug spew
Jan 24, 2024
489a29d
[C++ builds & passes] Update middleware & tests
Jan 24, 2024
bdd6046
[C++ builds & passes] CI linter fixups
Jan 24, 2024
59c0fad
Docstring
Jan 24, 2024
bb96fe3
Fix CMake lint error
Jan 24, 2024
0d59ab1
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Jan 26, 2024
b63641f
[Java builds & passes] Remove Java integration tests that won't work …
Jan 29, 2024
c0bf9ce
Add docs
Jan 29, 2024
f8d0d68
Add session_options to Archery integration testing
Jan 29, 2024
5ceb14d
Fix imports
Jan 29, 2024
07a581e
Exclude session_options go test
Jan 29, 2024
457f403
Docs touchup
Jan 29, 2024
35cb85e
[Builds & passes] naming consistency fix
Jan 30, 2024
3b622e1
[Builds & passes] Docstring fixups
Jan 30, 2024
f9221bc
Address PR feedback
Feb 5, 2024
d23127a
Update Protobuf enum comments
Feb 5, 2024
1c39b78
Linter fixups & rebase
Feb 5, 2024
b1e15f9
Factor out addMiddlewareHeaders
Feb 10, 2024
d084f60
Refactor GrpcServiceHandler::GetFlightInfo() to use addMiddlewareHead…
Feb 10, 2024
dc269f5
PR feedback
Feb 15, 2024
dfba6ee
PR feedback
Feb 15, 2024
14c0660
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Feb 20, 2024
ae24499
DX-61034: Arrow changes from GH-34865 PR
stevelorddremio Feb 21, 2024
1598587
temporarily Fix arrow-build issue with newest pygit2.
lriggs Jan 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,11 @@ python/doc/
# Egg metadata
*.egg-info

CMakeFiles
cpp-jni
java-jni
**/debug

.vscode
.idea/
.pytest_cache/
Expand Down
74 changes: 74 additions & 0 deletions cpp/src/arrow/flight/client.cc
Original file line number Diff line number Diff line change
Expand Up @@ -703,6 +703,80 @@ Status FlightClient::DoExchange(const FlightCallOptions& options,
return Status::OK();
}

::arrow::Result<SetSessionOptionsResult> FlightClient::SetSessionOptions(
const FlightCallOptions& options, const SetSessionOptionsRequest& request) {
RETURN_NOT_OK(CheckOpen());
ARROW_ASSIGN_OR_RAISE(auto body, request.SerializeToString());
Action action{ActionType::kSetSessionOptions.type, Buffer::FromString(body)};
ARROW_ASSIGN_OR_RAISE(auto stream, DoAction(options, action));
ARROW_ASSIGN_OR_RAISE(auto result, stream->Next());
ARROW_ASSIGN_OR_RAISE(
auto set_session_options_result,
SetSessionOptionsResult::Deserialize(std::string_view(*result->body)));
ARROW_RETURN_NOT_OK(stream->Drain());
return set_session_options_result;
}

::arrow::Result<GetSessionOptionsResult> FlightClient::GetSessionOptions(
const FlightCallOptions& options, const GetSessionOptionsRequest& request) {
RETURN_NOT_OK(CheckOpen());
ARROW_ASSIGN_OR_RAISE(auto body, request.SerializeToString());
Action action{ActionType::kGetSessionOptions.type, Buffer::FromString(body)};
ARROW_ASSIGN_OR_RAISE(auto stream, DoAction(options, action));
ARROW_ASSIGN_OR_RAISE(auto result, stream->Next());
ARROW_ASSIGN_OR_RAISE(
auto get_session_options_result,
GetSessionOptionsResult::Deserialize(std::string_view(*result->body)));
ARROW_RETURN_NOT_OK(stream->Drain());
return get_session_options_result;
}

::arrow::Result<CloseSessionResult> FlightClient::CloseSession(
const FlightCallOptions& options, const CloseSessionRequest& request) {
RETURN_NOT_OK(CheckOpen());
ARROW_ASSIGN_OR_RAISE(auto body, request.SerializeToString());
Action action{ActionType::kCloseSession.type, Buffer::FromString(body)};
ARROW_ASSIGN_OR_RAISE(auto stream, DoAction(options, action));
ARROW_ASSIGN_OR_RAISE(auto result, stream->Next());
ARROW_ASSIGN_OR_RAISE(auto close_session_result,
CloseSessionResult::Deserialize(std::string_view(*result->body)));
ARROW_RETURN_NOT_OK(stream->Drain());
return close_session_result;
}


/* PHOXME: take anything from here then delete this code...
pb::CloseSessionRequest request;

std::unique_ptr<ResultStream> results;
ARROW_ASSIGN_OR_RAISE(auto action, PackAction("CloseSession", request));
ARROW_RETURN_NOT_OK(DoAction(options, action, &results));

pb::ActionCloseSessionResult result;
ARROW_RETURN_NOT_OK(ReadResult(results.get(), &result));
ARROW_RETURN_NOT_OK(DrainResultStream(results.get()));
switch (result.result()) {
case pb::ActionCloseSessionResult::CLOSE_RESULT_UNSPECIFIED:
return CloseSessionResult::kUnspecified;
case pb::ActionCloseSessionResult::CLOSE_RESULT_CLOSED:
return CloseSessionResult::kClosed;
case pb::ActionCloseSessionResult::CLOSE_RESULT_CLOSING:
return CloseSessionResult::kClosing;
case pb::ActionCloseSessionResult::CLOSE_RESULT_NOT_CLOSEABLE:
return CloseSessionResult::kNotClosable;
default:
break;
}

return Status::IOError("Server returned unknown result ", result.result()); */

// /PHOXME






Status FlightClient::Close() {
if (!closed_) {
closed_ = true;
Expand Down
21 changes: 21 additions & 0 deletions cpp/src/arrow/flight/client.h
Original file line number Diff line number Diff line change
Expand Up @@ -412,6 +412,27 @@ class ARROW_FLIGHT_EXPORT FlightClient {
return Status::OK();
}

/// \brief Set server session option(s) by name/value. Sessions are generally
/// persisted via HTTP cookies.
/// \param[in] options Per-RPC options
/// \param[in] request The server session options to set
::arrow::Result<SetSessionOptionsResult> SetSessionOptions(
const FlightCallOptions& options, const SetSessionOptionsRequest& request);

/// \brief Get the current server session options. The session is generally
/// accessed via an HTTP cookie.
/// \param[in] options Per-RPC options
/// \param[in] request The (empty) GetSessionOptions request object.
::arrow::Result<GetSessionOptionsResult> GetSessionOptions(
const FlightCallOptions& options, const GetSessionOptionsRequest& request);

/// \brief Close/invalidate the current server session. The session is generally
/// accessed via an HTTP cookie.
/// \param[in] options Per-RPC options
/// \param[in] request The (empty) CloseSession request object.
::arrow::Result<CloseSessionResult> CloseSession(const FlightCallOptions& options,
const CloseSessionRequest& request);

/// \brief Explicitly shut down and clean up the client.
///
/// For backwards compatibility, this will be implicitly called by
Expand Down
19 changes: 19 additions & 0 deletions cpp/src/arrow/flight/integration_tests/flight_integration_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,25 @@ TEST(FlightIntegration, AuthBasicProto) { ASSERT_OK(RunScenario("auth:basic_prot

TEST(FlightIntegration, Middleware) { ASSERT_OK(RunScenario("middleware")); }

TEST(FlightIntegration, Ordered) { ASSERT_OK(RunScenario("ordered")); }

TEST(FlightIntegration, ExpirationTimeDoGet) {
ASSERT_OK(RunScenario("expiration_time:do_get"));
}

TEST(FlightIntegration, ExpirationTimeListActions) {
ASSERT_OK(RunScenario("expiration_time:list_actions"));
}

TEST(FlightIntegration, SessionOptions) {
ASSERT_OK(RunScenario("session_options"));
}

TEST(FlightIntegration, SessionOptions) { ASSERT_OK(RunScenario("session_options")); }

TEST(FlightIntegration, PollFlightInfo) { ASSERT_OK(RunScenario("poll_flight_info")); }
}

TEST(FlightIntegration, FlightSql) { ASSERT_OK(RunScenario("flight_sql")); }

TEST(FlightIntegration, FlightSqlExtension) {
Expand Down
Loading
Loading