Server streaming not implemented

[rhlsthrm]

Error: NOT IMPLEMENTED
      at GrpcTransport.serverStreaming (/root/node_modules/@protobuf-ts/grpc-transport/build/commonjs/grpc-transport.js:86:15)
      at tail (/root/node_modules/@protobuf-ts/runtime-rpc/build/commonjs/rpc-interceptor.js:21:49)
      at Object.stackIntercept (/root/node_modules/@protobuf-ts/runtime-rpc/build/commonjs/rpc-interceptor.js:26:16)

Is there a way to make a server streaming client that works in NodeJS? I was trying to use the promise-client.

[timostamm]

Is it possible that you are using an older version of @protobuf-ts/grpc-transport?

I've had server streaming running for some time now. See here for example:

protobuf-ts/packages/example-chat-system/client.ts

Line 27 in 10eb14f

const call = client.join({

It's a node CLI client using grpc-transport and server streaming.

[rhlsthrm]

Ah wow, I didn't realize I had an older version in my package.json, sorry for that. I see that you're not using the promise-client in any of your node examples. Do you not recommend using that?

[timostamm]

I don't know about promise-client 🤷 What is it?

The goal for protobuf-ts is to be able to generate:

• generic clients (can use "Transports" to use Twirp, Grpc-Web, gRPC, etc.) (implemented, but gRPC is not well tested yet)
• generic servers (can use "Backends" to serve Grpc-Web, gRPC, etc.) (in development, see Generic server support #56)

In addition to that, because it is trivial to implement and because users know them:

• generate clients specifically for @grpc/grpc-js (planned in Generate clients for @grpc/grpc-js #57)
• generate server interfaces specifically for @grpc/grpc-js (implemented in gRPC server support for @grpc/grpc-js #52)

For gRPC client and server on node, I recommend --ts_opt server_grpc1 for the server (see example) and --ts_opt client_call with @protobuf-ts/grpc-transport for the client (see example).

I can't recommend using gRPC for production yet. That'll have to wait for the 2.0 release.

[rhlsthrm]

promise-client was one of the possible opts in the command, I see some docs for it here: https://github.com/timostamm/protobuf-ts/blob/master/MANUAL.md#rpc-client-styles.

I will go ahead and use the client_call as you recommend! Thank you.

[rhlsthrm]

Hi @timostamm, I know this is off-topic, but since you are being so responsive and helpful, I thought I would ask you another question. I have a server and client I am testing together now. I am getting an error:

Uncaught RpcError: 13 INTERNAL: Cannot read property 'length' of undefined
      at ClientReadableStreamImpl.<anonymous> (/root/node_modules/@protobuf-ts/grpc-transport/build/commonjs/grpc-transport.js:79:52)
      at Object.onReceiveStatus (/root/node_modules/@grpc/grpc-js/src/client.ts:570:18)
      at Object.onReceiveStatus (/root/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48)
      at /root/node_modules/@grpc/grpc-js/src/call-stream.ts:249:24
      at processTicksAndRejections (internal/process/task_queues.js:79:

I'm thinking this is due to the data I am sending vs the schema? Do you have any tips on how to debug this type of issue or how to get more information on where the error is occurring?

[timostamm]

promise-client was one of the possible opts in the command, I see some docs for it here: https://github.com/timostamm/protobuf-ts/blob/master/MANUAL.md#rpc-client-styles.

Ah. Yes, client_call is the best choice.

[timostamm]

I am getting an error:

Uncaught RpcError: 13 INTERNAL: Cannot read property 'length' of undefined
      at ClientReadableStreamImpl.<anonymous> (/root/node_modules/@protobuf-ts/grpc-transport/build/commonjs/grpc-transport.js:79:52)
      at Object.onReceiveStatus (/root/node_modules/@grpc/grpc-js/src/client.ts:570:18)
      at Object.onReceiveStatus (/root/node_modules/@grpc/grpc-js/src/client-interceptors.ts:389:48)
      at /root/node_modules/@grpc/grpc-js/src/call-stream.ts:249:24
      at processTicksAndRejections (internal/process/task_queues.js:79:

Looks like the error originates somewhere in @grpc/grpc-js.
Unfortunately the stack trace is gone because of new RpcError in grpc-transport.ts.

I would just patch in a console.log statement in grpc-transport.js:79

        gCall.addListener('error', err => {
            console.log(err) // or err.stack?

Then you can follow real the stack trace.

[rhlsthrm]

@timostamm one more thing. I'm using google.protobuf.Any in my protobuf and trying to pass in an object into the client which is causing the above error? How are you supposed to use the google.protobuf.Any type?

[timostamm]

Usage is pretty similar to other languages

// Example 2: Pack and unpack a message in Java.
//
//     Foo foo = ...;
//     Any any = Any.pack(foo);
//     ...
//     if (any.is(Foo.class)) {
//       foo = any.unpack(Foo.class);
//     }

let foo: Foo = Foo.create();

let any: Any = Any.pack(foo, Foo);

if (Any.contains(any, Foo)) {
  foo = Any.unpack(any, Foo);
}

If you want to send a message that contains a field with an any:

message MyRequest {
   google.protobuf.Any xxx = 1;
}
message Foo {}

let foo: Foo = {};
let r: MyRequest = {
   xxx : Any.pack(foo, Foo)
};

Any is just binary data of a message + a "type url" that contains the type name. The following is equivalent to Any.pack():

let any: Any = {
   value: Foo.toBinary(foo),
   typeUrl: "type.googleapis.com/Foo"
};

[rhlsthrm]

Thanks so much! I was having a lot of trouble finding TS usage examples. This would be helpful info for the readme, seems like everything online is outdated.

[rhlsthrm]

Hey @timostamm still having some trouble here. In your example, the Any type maps to another message type. I thought the point of Any is that you don't know the message type upfront and it could be anything.

For example, this does not work, obviously because you can't have a type as a value:

type User = {
  firstName: string;
  lastName: string;
};
GrpcTypes.Any.pack(data, User);

Is there any way to use Any to represent types that are not part of the protobuf schema?

[timostamm]

This is an example for a good use case for any:

message Status {
  // The status code, which should be an enum value of [google.rpc.Code][google.rpc.Code].
  int32 code = 1;

  // A developer-facing error message, which should be in English. Any
  // user-facing error message should be localized and sent in the
  // [google.rpc.Status.details][google.rpc.Status.details] field, or localized by the client.
  string message = 2;

  // A list of messages that carry the error details.  There is a common set of
  // message types for APIs to use.
  repeated google.protobuf.Any details = 3;
}

https://github.com/googleapis/googleapis/blob/master/google/rpc/status.proto

When you create a status message, you put one of the messages defined in error_details.proto in the "details" field.

But if your API has some special requirements where the messages defined in error_details.proto is not sufficients, it is totally fine to put a different message in there.
You just have to take care when reading the message (but that should be obvious when handling an any).

[timostamm]

If you really want to pack a "User" into the any, you should have some kind of protobuf declaration for this type.

Or you could just create the type at runtime:

let userType = new MessageType<UnknownMessage>("User", [
    {no: 1, name: "firstName", kind: "scalar", T: ScalarType.STRING},
    {no: 2, name: "lastName", kind: "scalar", T: ScalarType.STRING}
]);

let user = {
    firstName: "a",
    lastName: "b"
};

let any = Any.pack(user, userType);

Of course you have to be pretty careful with that. There are no typescript interfaces preventing you from passing an invalid object.

[timostamm]

Or you could assemble the binary data yourself:

let bytes = new BinaryWriter()
    .fork()
    .tag(1, WireType.LengthDelimited)
    .string("a")
    .tag(2, WireType.LengthDelimited)
    .string("b")
    .join()
    .finish();

let any: Any = {
      value: bytes,
      typeUrl: "type.googleapis.com/User"
};

Terrible idea for maintenance, but possible :)

[timostamm]

Maybe google.protobuf.Struct is a better match for your use case?

Structs allow arbitrary data. Same types as JSON.

message Foo {
   google.protobuf.Struct user = 1;
}

let foo: Foo = {
   user: Struct.fromJson({
    firstName: "a",
    lastName: "b"
  })
}

[rhlsthrm]

Ah, I think a Struct is exactly what I need! Thank you so much for this! You have been amazingly helpful. Please let me know if you have a donation link, I'm happy to support.