Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Change from protobuf to cbor for gRPC #3061

Conversation

fredcarle
Copy link
Collaborator

Relevant issue(s)

Resolves #3057

Description

This PR makes the net gRPC server use cbor instead of protobuf. By doing this we remove the protobuf overhead and the related dependencies which will make compilation to wasm simpler and simplify browser support.

Tasks

  • I made sure the code is well commented, particularly hard-to-understand areas.
  • I made sure the repository-held documentation is changed accordingly.
  • I made sure the pull request title adheres to the conventional commit style (the subset used in the project can be found in tools/configs/chglog/config.yml).
  • I made sure to discuss its limitations such as threats to validity, vulnerability to mistake and misuse, robustness to invalidation of assumptions, resource requirements, ...

How has this been tested?

make test

Specify the platform(s) on which this was tested:

  • MacOS

@fredcarle fredcarle added refactor This issue specific to or requires *notable* refactoring of existing codebases and components area/p2p Related to the p2p networking system labels Sep 24, 2024
@fredcarle fredcarle added this to the DefraDB v0.14 milestone Sep 24, 2024
@fredcarle fredcarle requested a review from a team September 24, 2024 02:57
@fredcarle fredcarle self-assigned this Sep 24, 2024
@fredcarle fredcarle force-pushed the fredcarle/refactor/3057-protobuf-to-cbor branch from 8f0274c to b5a27ed Compare September 24, 2024 02:58
Copy link

codecov bot commented Sep 24, 2024

Codecov Report

Attention: Patch coverage is 87.20930% with 11 lines in your changes missing coverage. Please review.

Project coverage is 79.41%. Comparing base (98befd6) to head (212d1aa).
Report is 1 commits behind head on develop.

Files with missing lines Patch % Lines
net/grpc.go 66.67% 7 Missing and 1 partial ⚠️
internal/kms/pubsub.go 81.82% 0 Missing and 2 partials ⚠️
net/server.go 94.12% 0 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #3061      +/-   ##
===========================================
- Coverage    79.47%   79.41%   -0.06%     
===========================================
  Files          343      345       +2     
  Lines        26511    26534      +23     
===========================================
+ Hits         21067    21070       +3     
- Misses        3934     3945      +11     
- Partials      1510     1519       +9     
Flag Coverage Δ
all-tests 79.41% <87.21%> (-0.06%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
net/client.go 100.00% <100.00%> (ø)
net/codec.go 100.00% <100.00%> (ø)
net/dialer.go 92.59% <100.00%> (ø)
net/peer.go 79.49% <100.00%> (-0.81%) ⬇️
net/server.go 75.68% <94.12%> (+0.11%) ⬆️
internal/kms/pubsub.go 66.67% <81.82%> (ø)
net/grpc.go 66.67% <66.67%> (ø)

... and 15 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 98befd6...212d1aa. Read the comment docs.

Copy link
Member

@shahzadlone shahzadlone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow this is good work lots of proto code removed. Is nicer and cleaner! Left some minor questions/suggestions.

// handleEncryptionMessage handles incoming FetchEncryptionKeyRequest messages from the pubsub network.
func (s *pubSubService) handleRequestFromPeer(peerID libpeer.ID, topic string, msg []byte) ([]byte, error) {
req := new(pb.FetchEncryptionKeyRequest)
if err := proto.Unmarshal(msg, req); err != nil {
req := new(fetchEncryptionKeyRequest)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: not very common way of creating a value on heap.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You know that is your code right 😉

Copy link
Member

@nasdf nasdf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fredcarle fredcarle force-pushed the fredcarle/refactor/3057-protobuf-to-cbor branch from dc7d230 to 212d1aa Compare September 24, 2024 18:01
@fredcarle fredcarle merged commit 8dae109 into sourcenetwork:develop Sep 24, 2024
42 of 43 checks passed
@fredcarle fredcarle deleted the fredcarle/refactor/3057-protobuf-to-cbor branch September 24, 2024 18:42
ChrisBQu pushed a commit to ChrisBQu/defradb that referenced this pull request Feb 21, 2025
## Relevant issue(s)

Resolves sourcenetwork#3057

## Description

This PR makes the net gRPC server use cbor instead of protobuf. By doing
this we remove the protobuf overhead and the related dependencies which
will make compilation to wasm simpler and simplify browser support.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/p2p Related to the p2p networking system refactor This issue specific to or requires *notable* refactoring of existing codebases and components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Change net package data protocol from protobuf to cbor.
4 participants