Can protobuf.js be used together with grpc-web?

[gunters63]

I am currently using grpc-web in a React SPA-project.

The backend is implemented with gRPC for .NET, which already implements a middleware to translate gRPC-Web to gRPC HTTP/2 (like a built-in Envoy).

On the client side I use protoc.exe together with the grpc-web plugin (protoc-gen-grpc-web.exe) to generate code for my services and for the proto serialization.

protoc.exe is currently not able to generate ES6 modules which is a bit problematic in my setup (I use Vite). Protobufjs seems to be better in this regard. Furthermore protobufjs claims to be several times faster with serialization/deserialization than the protoc.exe generated code. I would like to switch, but I would still have to keep grpc-web for the transport channel (multiplexing/framing) I think. My client code running in the browser has dozens of simultaneous server streaming subscriptions which is no problem with grpc-web and the protoc-generated code.

So my question: Is it possible to use protobuf.js together with grpc-web?

[alexander-fenster]

I think it's not possible, but it's better to ask grpc-web folks :)

[gunters63]

I solved my problem in the meanwhile by hacking protoc.exe to generate ES6 modules

[achingbrain]

Not really as this module doesn't support streaming rpc methods properly - there's a PR to fix that here but it appears to have stalled: #1115

[forrestli74]

This might help: grpc/grpc-web#80 (comment)
It implements rpcImpl for grpc web, which is a little hacky and only handles non streaming.

[arontsang]

I solved my problem in the meanwhile by hacking protoc.exe to generate ES6 modules

You might want to take a look at my nuget package https://www.nuget.org/packages/ArtsTech.Grpc.TypescriptClientGenerator/

[phi0411141]

@gunters63 can you help to share how to make protoc.exe generates es6 module?

[gunters63]

I have forked commit a1d8f8f9 of https://github.com/protocolbuffers/protobuf and patched ES6 support. It works for me and sorry, I never bothered to pull master again.

Here is a patch:

https://gist.github.com/gunters63/3c8798de46fa469e1ce06966df017ca8

If you want I can send you a Nextcloud link to Windows binaries

I create my (Typescript) code with a command line like this:

Executing bin\protobuf\windows-x86_64\protoc.exe -I=src\proto -I=src\proto\gen src\proto\alarm.proto --js_out=import_style=es6:src\proto\gen
Executing bin\protobuf\windows-x86_64\protoc.exe -I=src\proto -I=src\proto\gen src\proto\alarm.proto --plugin="protoc-gen-grpc-web=bin\protobuf\windows-x86_64\protoc-gen-grpc-web-1.3.0-windows-x86_64.exe" --grpc-web_out=import_style=typescript,mode=grpcwebtext:src\proto\gen

[gunters63]

Here a link: https://nextcloud.spranz.info/s/A7xHwmrbM6ByTnf

The zip-Archiv also contains Linux binaries, but I never tested them.

[nathansegers]

@gunters63 Thank you very much for this patch! This solved my search for a working Typescript version of my Stubs!

I have added your Linux binaries into a Docker container from namely/protoc-all which I use to generate a bunch of stubs for other programming languages as well.
It does not generate the exact same results as on Windows.

A few things I have noticed:
Windows generates this (addition.proto):

import * as jspb from 'google-protobuf';
var goog = jspb;
var proto = {};

goog.exportSymbol('addition.AddRequest', null, proto);
...
export const {
  AddRequest,
} = proto.addition;

whereas on Linux I get:

goog.exportSymbol('proto.addition.AddRequest', null, global);

Without any exports...

Currently I have 'hacked' my way around embedding your plugin into a custom Docker Entrypoint bash script to fetch all proto files and execute protoc on top, but this executes the commands on Linux and thus generates an 'invalid' JS file.

Eventually, I should try to add it to their entrypoint instead.

[gunters63]

@nathansegers: Yeah, looks like the Linux binaries I uploaded where built from unpatched sources.

I really don't understand why the protobuf team didn't implement ES6 support yet. It should be quite easy.. There is some initial support (header constants ...) in the sources already for years now.

I suggest you clone the specific commit of protobuf, apply the patch and build the linux binaries yourself.

[nathansegers]

Currently I'll just leave to work on Windows, which is fine for my situation ;-)

[yinzara]

This is an interesting alternative now:
https://buf.build/blog/protobuf-es-the-protocol-buffers-typescript-javascript-runtime-we-all-deserve

It does work with protoc-gen-grpc-web and supports ESM and is tree shakeable.