Port to Go for 1.0 Release

[pseudomuto]

Working on porting this project to go. More details to follow, these lists are works in progress as well.

New Things

• Support for proto3 (without losing anything from proto2) 🎉
• Fairly comprehensive test coverage
• Bug fixes (see below)

Breaking Changes (might address some of these):

• Invocation changed from parsing --doc_out to parsing --doc_opt.
• Templates are now golang templates, rather than mustache
• JSON output now includes top-level files object, and all names are camelCased to be more idiomatic
• No direct support for PDF, though fop would be easy to add back in

Fixed Issues:

• Fixes Add support for protobuf 3.0 #17
• Fixes Message enums are included in file_enums #288
• Fixes Include the "files" key name in generated json #289
• Fixes markdown.template lists are not correctly formatted #293
• Fixes Slashes end up in the html output when using triple-slash comments #298
• Fixes Field information not rendered with custom html.mustache templates #300
• Fixes Message headers in markdown broken due missing newline #303
• Fixes Enum values seem to be enumerated, rather than taking the value defined #311

[estan]

Very cool! I'm excited to see this happen, and humbled by being added as a potential reviewer.

However, I'm a complete Go rookie. Both Go and Rust has long been on my TODO of things to familiarize myself with, but I've been so preoccupied with work (where it's all C++/Python) that it's never happened.

I'm all for this port, but I think you better find someone else if you want a proper review. Or well, just say to hell with it and push, you're the maintainer now after all :)

[pseudomuto]

@estan thanks so much for all your reviews! Can I ask you one more favour?

I was wondering if you could try running it, just for my own sanity to make sure it's not one of those "works for me" situations 😄

I've pushed 1.0.0-alpha, so the Docker image is available in DockerHub now.

https://github.com/pseudomuto/protoc-gen-doc/tree/3d7ea549ea28da4fd929fb6b20f24dd763355c11#using-the-docker-image-recommended

[pseudomuto]

@aschrijver I saw that you had opened a few issues that should be resolved with this PR. Any chance you'd be willing to give it a shot?

[aschrijver]

@pseudomuto Sure! Really nice work you've done here!
I'll update my issues, SO questions based on findings..

[UPDATE The docker image is not yet available, so now figuring out how the go build (currently finding out how to build from go_port branch :)

[aschrijver]

I need some help in using / building. Both the docker and go package are not yet there, only in go_port branch. Building locally go build or install doesn't do it as well.

plugin.go:5:2: cannot find package "github.com/golang/protobuf/proto" in any of:
	/usr/lib/go-1.6/src/github.com/golang/protobuf/proto (from $GOROOT)
	/home/arnold/go/src/github.com/golang/protobuf/proto (from $GOPATH)
plugin.go:6:2: cannot find package "github.com/golang/protobuf/protoc-gen-go/plugin" in any of:
	/usr/lib/go-1.6/src/github.com/golang/protobuf/protoc-gen-go/plugin (from $GOROOT)
	/home/arnold/go/src/github.com/golang/protobuf/protoc-gen-go/plugin (from $GOPATH)
plugin.go:7:2: cannot find package "github.com/pseudomuto/protoc-gen-doc/parser" in any of:
	/usr/lib/go-1.6/src/github.com/pseudomuto/protoc-gen-doc/parser (from $GOROOT)
	/home/arnold/go/src/github.com/pseudomuto/protoc-gen-doc/parser (from $GOPATH)

[aschrijver]

Or the docker image..

[pseudomuto]

Hmm...I did push an alpha build with the docker image.

Are you able to get it using docker pull pseudomuto/protoc-gen-doc?

To test with a local build, take a look at CONTRIBUTING.md on this branch. Once you've got the dependencies you can run make build to build the binary.

Thanks so much for taking a look man, I really appreciate it!

[aschrijver]

Yep, I can pull the docker image now. I'll commence my tests later today! Thanks!

[aschrijver]

@pseudomuto there seems to be no way to specify a single .proto instead of a whole directory, am I right? I'm trying to do something like:

docker run --rm -v $(pwd)/build:/out -v $(pwd)/schemas:/protos pseudomuto/protoc-gen-doc 
     --doc_opt=markdown,hypercore-protocol.md schemas/HypercoreSpecV1_md.proto

(I always get 'No such file or directory' even when using $(pwd), but the file exists at that location)

[UPDATE 😵 😵 Should use the docker volume, stupid 😆 ]

[aschrijver]

Alright, I adjusted my TravisCI test project to the new version, and now have 2 branches to compare:

• https://github.com/aschrijver/hypercore-protocol/tree/feature/document-generator-new
• https://github.com/aschrijver/hypercore-protocol/tree/feature/document-generator

I didn't update the pro's/con's yet, and custom templates are same as standard still, but some first observations:

• it's not possible to specify individual .proto, so I used different directories instead
• there are still a number of things to fix in the generated output (compare old/new)

[pseudomuto]

Pushed a new build (and container) that addresses the issue of generating docs for individual files (can be multiple). You can get the latest by running docker pull pseudomuto/protoc-gen-doc again.

[aschrijver]

Great, I will test tomorrow. (I'm in CET, Netherlands)

[aschrijver]

This is a bit of a feature request, but putting here, since this is not yet live (I can create issues later).
As you can see in my example .proto there is a sequence to the messages which corresponds more or less to how the protocol works.

Having the message in the same sequence as defined in the .proto would be much more intuitive than having it alphabetically ordered.

So maybe the ordering could be set with a flag or something, or even an @order annotation.

[aschrijver]

@pseudomuto don't hesitate to give me a note when I should regenerate/retest my doc example project

[pseudomuto]

Hey @aschrijver. I've updated the examples with your PR applied. I put out a RC release (1.0.0-rc) which you can get with docker pull pseudomuto/protoc-gen-doc. Can you give it one more shot before launch?

As for your feature request, it seems like a pretty good idea. I'd be happy to work on that (or review a PR) for 1.1.0, but I'd really like to cut this release soon as this PR is already massive.

[aschrijver]

Hi @pseudomuto

Thanks! I created a dedicated repository for the testing: https://github.com/aschrijver/protoc-gen-doc-example. There are a number of issues still in the template output.

HTML:

• takes line breaks literally from the comments, converts to <p> yielding too much line space
• code snippets have no indentation spaces
• protocol file description is not added to the output

Markdown:

• file name ToC title has line breaks in front of it
• code snippets have no indentation spaces
• protocol file description is not added to the output

[aschrijver]

I also updated my 2 stackoverflow answers that were about protoc-gen-doc:

• https://stackoverflow.com/a/45254099/8295283
• https://stackoverflow.com/a/45253665/8295283

[glasser]

This looks awesome. However, since you can't point go get at a branch, it's hard to install this directly until you merge it. Are you planning to merge it? Or maybe move it to a different GitHub repo? (I could fork it to my own but would have to update all the internal package references...)

[glasser]

The nobr filter used in markdown descriptions seems pretty intense: it doesn't add spaces, so words at the end and beginning of lines get put directly together without a space.

[pseudomuto]

This looks awesome. However, since you can't point go get at a branch, it's hard to install this directly until you merge it

Thanks man! I definitely am planning on merging this. I want to take a look at a few of the issues mentioned here first.

In the meantime, the docker image is available, so you should be able to run it that way. https://github.com/pseudomuto/protoc-gen-doc/tree/go_port#using-the-docker-image-recommended