Athens airgapped mode

[LukeMauldin]

I would like to run Athens in a completely airgapped mode. I have it setup with disk storage. I use a separate Athens server on a machine with internet access to download modules and then manually transfer the storage directory to the Athens server. This works as long as I specify the exact version in go get. Example:
‘go get -v github.com/foo/bar@v1.0.1’ however if I run ‘go get -v github.com/foo/bar’ then it fails because Athens tries to do a ‘go list -m’ internally which fails due to lack of internet access.

I would prefer for that ‘go list’ command just to return the versions that I have uploaded into Athens for a specific module. If that is not possible, I would at least like to configure it not to make the external calls which will always fail.

I have tried the filtering modes and download mode options but have not been able to get the configuration correct. Is this even possible with the current version?

Using version v0.6.0

[arschles]

Hi @LukeMauldin - at the moment, Athens tries to fetch the list of versions from storage and from the upstream, and then merge them (here is where it does that)

When you do a go get without the version number, iirc it's doing a request to the @latest endpoint. We could add a config flag to Athens to make it just look at the latest version in storage instead of going upstream. @marwan-at-work what do you think?

[marwan-at-work]

I think it makes sense to add a new configuration option where a "more advanced" Athens admin would "disable using go list, or go mod download"

However, I do want to understand your use case a little more...couldn't you just solve this on the client side by doing GOPROXY=athens1,athens2,direct,etc go get ...??

[LukeMauldin]

I like the idea of a configuration option to tell Athens to use the packages in storage and not check upstream for completely airgapped scenarios.
As far as I know, I cannot resolve this client side. Use case: I import github.com/foo/bar v1.0.0 and v1.0.1 manually into Athens storage. I then have a module that depends on v1.0.0. I would like to be able to do ‘go get -u github.com/foo/bar’ and for Athens to return v1.0.1 and not try to check upstream in github.com because that will fail.

[arschles]

Sorry for the delay @LukeMauldin. In your use-case, when you do the go get -u github.com/foo/bar, are you looking to prevent Athens from lookup upstream, even if it can access the public internet?

That said, I'm ok with adding an "airgapped" configuration. It would be fairly easy to prevent Athens from looking upstream on the /@v/list codepath -- we could prevent it from launching the "check upstream" goroutine. As for the @latest handler, we can basically do a storage-only list as with the list handler and get the first element in the sorted list.

What do you think @marwan-at-work ?

[marwan-at-work]

It should be as simple as just avoiding the UpstreamLister and keep whatever in storage. We can do it in two ways:

1. Yet another config. Something like "Offline=true"
2. Pass the DownloadFile to the Lister and if the DownloadMode == None, then don't use UpstreamLister.

I prefer 2 for now since we already have a pretty big config file

[LukeMauldin]

Using the new config options in Athens, I was able to set the GoProxy and GoPrivate variables to get the behavior that I needed. I was able to get it to work for both Go 1.12 and Go 1.13 clients.

[arschles]

@LukeMauldin that's awesome. Can you share that part of your config? We can add that to the docs