Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go mod tidy always prints "finding" log messages #27123

Closed
rogpeppe opened this issue Aug 21, 2018 · 21 comments
Closed

cmd/go: go mod tidy always prints "finding" log messages #27123

rogpeppe opened this issue Aug 21, 2018 · 21 comments
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@rogpeppe
Copy link
Contributor

rogpeppe commented Aug 21, 2018

go version devel +4e1b11e2c9 Tue Aug 21 14:08:55 2018 +0000 linux/amd64

While I expect log messages the first time running a command, I don't expect them
to be printed the second time, after all information has already been found.
However in some cases, go mod tidy prints some finding ... latest messages
every time it's run.

For example:

% cd /tmp
% git clone git@github.com:juju/charmstore-client
fatal: repository 'github.com/juju/charmstore-client' does not exist
% git clone git@github.com:juju/charmstore-client
Cloning into 'charmstore-client'...
remote: Counting objects: 1092, done.        
% cd charmstore-client
% go mod init
go: creating new go.mod: module github.com/juju/charmstore-client
go: copying requirements from dependencies.tsv
% go mod tidy
go: finding launchpad.net/gocheck latest
go: finding github.com/docker/go-connections/sockets latest
go: finding github.com/docker/go-connections/nat latest
go: finding github.com/docker/go-connections/tlsconfig latest
go: finding github.com/juju/proxy latest
go: finding github.com/gotestyourself/gotestyourself/assert/cmp latest
go: finding github.com/gotestyourself/gotestyourself/env latest
go: finding github.com/opencontainers/image-spec/specs-go latest
go: finding github.com/gotestyourself/gotestyourself/assert latest
go: finding github.com/docker/libtrust latest
go: finding github.com/gogo/protobuf/proto latest
go: finding github.com/juju/rfc/rfc5424/sdelements latest
go: finding github.com/juju/rfc/rfc5424 latest
go: finding github.com/Nvveen/Gotty latest
go: finding github.com/juju/rfc latest
go: finding github.com/Azure/go-ansiterm/winterm latest
go: finding gopkg.in/juju/blobstore.v2 latest
go: finding github.com/Azure/go-ansiterm latest
go: finding gopkg.in/juju/charmrepo.v2 latest
go: finding gopkg.in/juju/charmstore.v5-unstable latest
go: finding golang.org/x/sync/errgroup latest
go: finding golang.org/x/sync latest
go: finding github.com/gotestyourself/gotestyourself/skip latest
go: finding github.com/docker/go-metrics latest
go: finding github.com/juju/txn latest
go: finding github.com/juju/romulus/wireformat/metrics latest
go: finding github.com/dustin/go-humanize latest
go: finding github.com/juju/packaging/config latest
go: finding github.com/juju/romulus/wireformat latest
go: finding github.com/juju/romulus latest
go: finding github.com/juju/jsonschema latest
go: finding github.com/juju/packaging latest
go: finding github.com/juju/naturalsort latest
go: finding github.com/CanonicalLtd/omniutils/testing/pgtest latest
go: finding github.com/coreos/go-systemd/unit latest
go: finding github.com/CanonicalLtd/omniutils/testing latest
go: finding github.com/CanonicalLtd/omniutils latest
go: finding github.com/juju/description latest
go: finding github.com/coreos/go-systemd latest
go: finding github.com/stretchr/testify/assert latest
go: finding github.com/juju/packaging/commands latest
go: finding github.com/juju/pubsub latest
go: finding github.com/coreos/go-systemd/dbus latest
go: finding golang.org/x/text/encoding/unicode latest
go: finding github.com/bmizerany/pat latest
go: finding gopkg.in/amz.v3 latest
go: finding github.com/masterzen/winrm latest
go: finding github.com/juju/txn/testing latest
go: finding github.com/juju/packaging/manager latest
go: finding github.com/juju/romulus/api/plan latest
go: finding github.com/juju/romulus/api latest
go: finding github.com/golang/mock/gomock latest
go: finding github.com/juju/replicaset latest
go: finding github.com/juju/romulus/wireformat/sla latest
go: finding github.com/juju/bundlechanges latest
go: finding golang.org/x/text/encoding latest
go: finding github.com/juju/gomaasapi latest
go: finding gopkg.in/natefinch/npipe.v2 latest
go: finding github.com/juju/romulus/wireformat/plan latest
go: finding github.com/joyent/gocommon/errors latest
go: finding github.com/joyent/gocommon/client latest
go: finding github.com/joyent/gocommon latest
go: finding github.com/joyent/gosdc/cloudapi latest
go: finding github.com/joyent/gosdc/localservices/cloudapi latest
go: finding github.com/joyent/gosdc latest
go: finding github.com/joyent/gosdc/localservices latest
go: finding github.com/google/go-querystring/query latest
go: finding github.com/google/go-querystring latest
go: finding github.com/joyent/gosign/auth latest
go: finding github.com/joyent/gosign latest
go: finding github.com/Azure/azure-sdk-for-go/arm/disk latest
go: finding github.com/Azure/azure-sdk-for-go/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/compute latest
go: finding github.com/Azure/azure-sdk-for-go/arm/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/subscriptions latest
go: finding github.com/Azure/azure-sdk-for-go/arm latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources latest
go: finding github.com/Azure/go-autorest/autorest/to latest
go: finding github.com/Azure/go-autorest/autorest/mocks latest
go: finding github.com/Azure/go-autorest/autorest/azure latest
go: finding github.com/Azure/go-autorest/autorest latest
go: finding github.com/Azure/go-autorest/autorest/adal latest
go: finding github.com/juju/romulus/wireformat/budget latest
go: finding github.com/juju/go-oracle-cloud/response latest
go: finding github.com/juju/go-oracle-cloud/api latest
go: finding github.com/juju/go-oracle-cloud/common latest
go: finding github.com/juju/go-oracle-cloud latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/resources latest
go: finding github.com/juju/romulus/api/sla latest
go: finding google.golang.org/api/googleapi latest
go: finding google.golang.org/api latest
go: finding github.com/lxc/lxd/shared/api latest
go: finding github.com/lxc/lxd/shared latest
go: finding github.com/lxc/lxd latest
go: finding k8s.io/client-go/tools/clientcmd latest
go: finding k8s.io/client-go/tools/clientcmd/api latest
go: finding k8s.io/client-go/tools latest
go: finding github.com/altoros/gosigma latest
go: finding github.com/altoros/gosigma/mock latest
go: finding github.com/vmware/govmomi/vim25/types latest
go: finding github.com/vmware/govmomi/vim25/progress latest
go: finding github.com/vmware/govmomi/property latest
go: finding github.com/vmware/govmomi/vim25/soap latest
go: finding github.com/vmware/govmomi/object latest
go: finding github.com/vmware/govmomi/list latest
go: finding github.com/vmware/govmomi/vim25/methods latest
go: finding github.com/vmware/govmomi/vim25/mo latest
go: finding github.com/vmware/govmomi/vim25 latest
go: finding github.com/vmware/govmomi/ovf latest
go: finding github.com/juju/romulus/api/budget latest
go: finding github.com/Azure/azure-sdk-for-go/arm/network latest
go: finding github.com/vmware/govmomi/find latest
go: finding github.com/lxc/lxd/shared/osarch latest
go: finding github.com/vmware/govmomi/session latest
go: finding github.com/Azure/azure-sdk-for-go/arm/authorization latest
go: finding github.com/altoros/gosigma/data latest
go: finding github.com/vmware/govmomi/vim25/xml latest
go: finding github.com/Azure/go-autorest/autorest/validation latest
go: finding google.golang.org/api/compute latest
go: finding golang.org/x/oauth2/google latest
go: finding golang.org/x/oauth2 latest
go: finding github.com/lib/pq latest
go: finding gotest.tools/x/subtest latest
go: finding gotest.tools/internal/format latest
go: finding gotest.tools/assert latest
go: finding gotest.tools/internal/source latest
go: finding gotest.tools/assert/cmp latest
go: finding gotest.tools/skip latest
go: finding gotest.tools/x latest
go: finding gotest.tools/internal latest
go: finding github.com/pmezard/go-difflib/difflib latest
go: finding gopkg.in/juju/jujusvg.v2 latest
go: finding github.com/lestrrat/go-jsschema/validator latest
go: finding github.com/lestrrat/go-jsschema latest
go: finding github.com/davecgh/go-spew/spew latest
go: finding github.com/dnaeon/go-vcr/recorder latest
go: finding github.com/dnaeon/go-vcr/cassette latest
go: finding github.com/dnaeon/go-vcr latest
go: finding github.com/smartystreets/goconvey/convey latest
go: finding github.com/smartystreets/goconvey latest
go: finding github.com/ChrisTrenkamp/goxpath/tree/xmltree latest
go: finding github.com/ChrisTrenkamp/goxpath latest
go: finding github.com/ChrisTrenkamp/goxpath/tree latest
go: finding github.com/masterzen/azure-sdk-for-go/core/tls latest
go: finding github.com/masterzen/azure-sdk-for-go/core/http latest
go: finding github.com/masterzen/azure-sdk-for-go/core latest
go: finding github.com/masterzen/simplexml/dom latest
go: finding github.com/masterzen/simplexml latest
go: finding github.com/Azure/go-ntlmssp latest
go: finding github.com/nu7hatch/gouuid latest
go: finding google.golang.org/appengine/urlfetch latest
go: finding k8s.io/apimachinery/pkg/runtime latest
go: finding k8s.io/apimachinery/pkg/types latest
go: finding k8s.io/apimachinery/pkg/runtime/schema latest
go: finding k8s.io/apimachinery/pkg/runtime/serializer/streaming latest
go: finding k8s.io/apimachinery/pkg/watch latest
go: finding k8s.io/apimachinery/pkg/runtime/serializer/json latest
go: finding k8s.io/apimachinery/pkg latest
go: finding k8s.io/apimachinery/pkg/runtime/serializer latest
go: finding github.com/golang/glog latest
go: finding k8s.io/apimachinery latest
go: finding k8s.io/apimachinery/pkg/api/errors latest
go: finding cloud.google.com/go/compute/metadata latest
go: finding k8s.io/apimachinery/pkg/api latest
go: finding cloud.google.com/go/compute latest
go: finding k8s.io/api/batch/v2alpha1 latest
go: finding k8s.io/api/rbac latest
go: finding k8s.io/api/apps/v1beta1 latest
go: finding k8s.io/api/scheduling/v1alpha1 latest
go: finding k8s.io/api/storage/v1beta1 latest
go: finding k8s.io/api/networking latest
go: finding k8s.io/api/authentication latest
go: finding k8s.io/api/core latest
go: finding k8s.io/api latest
go: finding k8s.io/api/batch latest
go: finding k8s.io/api/apps latest
go: finding k8s.io/api/scheduling latest
go: finding k8s.io/api/storage latest
go: finding k8s.io/api/admissionregistration/v1alpha1 latest
go: finding k8s.io/api/admissionregistration/v1beta1 latest
go: finding k8s.io/api/events/v1beta1 latest
go: finding k8s.io/api/policy/v1beta1 latest
go: finding k8s.io/apimachinery/pkg/util/sets latest
go: finding k8s.io/apimachinery/pkg/conversion latest
go: finding k8s.io/api/authentication/v1beta1 latest
go: finding k8s.io/api/admissionregistration latest
go: finding k8s.io/apimachinery/pkg/util/intstr latest
go: finding k8s.io/api/events latest
go: finding k8s.io/api/policy latest
go: finding k8s.io/apimachinery/pkg/util latest
go: finding k8s.io/api/settings/v1alpha1 latest
go: finding k8s.io/apimachinery/pkg/util/validation latest
go: finding k8s.io/api/scheduling/v1beta1 latest
go: finding k8s.io/apimachinery/pkg/util/clock latest
go: finding k8s.io/apimachinery/pkg/util/errors latest
go: finding k8s.io/api/batch/v1beta1 latest
go: finding k8s.io/api/settings latest
go: finding k8s.io/api/rbac/v1beta1 latest
go: finding k8s.io/apimachinery/pkg/util/httpstream latest
go: finding k8s.io/api/autoscaling latest
go: finding k8s.io/apimachinery/pkg/util/diff latest
go: finding k8s.io/apimachinery/pkg/apis/meta latest
go: finding k8s.io/apimachinery/pkg/util/net latest
go: finding k8s.io/apimachinery/pkg/runtime/serializer/versioning latest
go: finding k8s.io/api/apps/v1beta2 latest
go: finding k8s.io/api/storage/v1alpha1 latest
go: finding k8s.io/apimachinery/pkg/apis latest
go: finding k8s.io/api/rbac/v1alpha1 latest
go: finding k8s.io/api/authorization latest
go: finding k8s.io/apimachinery/pkg/util/wait latest
go: finding k8s.io/apimachinery/pkg/api/equality latest
go: finding k8s.io/api/extensions/v1beta1 latest
go: finding k8s.io/api/certificates/v1beta1 latest
go: finding k8s.io/api/autoscaling/v2beta1 latest
go: finding k8s.io/apimachinery/pkg/version latest
go: finding k8s.io/api/authorization/v1beta1 latest
go: finding k8s.io/api/extensions latest
go: finding k8s.io/api/certificates latest
go: finding github.com/google/gofuzz latest
go: finding golang.org/x/time/rate latest
go: finding golang.org/x/time latest
go: finding github.com/lestrrat/go-jsval/builder latest
go: finding github.com/lestrrat/go-jsval latest
go: finding github.com/lestrrat/go-jsref latest
go: finding github.com/lestrrat/go-jsref/provider latest
go: finding github.com/lestrrat/go-pdebug latest
go: finding github.com/modern-go/reflect2 latest
go: finding github.com/json-iterator/go latest
go: finding github.com/xeipuuv/gojsonschema latest
go: finding github.com/smartystreets/assertions latest
go: finding github.com/lestrrat/go-structinfo latest
go: finding github.com/lestrrat/go-jspointer latest
go: finding github.com/xeipuuv/gojsonreference latest
go: finding github.com/modern-go/concurrent latest
go: finding github.com/gopherjs/gopherjs/js latest
go: finding github.com/gopherjs/gopherjs latest
go: finding github.com/xeipuuv/gojsonpointer latest
% 
% go mod tidy
go: finding github.com/masterzen/azure-sdk-for-go/core/tls latest
go: finding github.com/masterzen/azure-sdk-for-go/core/http latest
go: finding github.com/masterzen/azure-sdk-for-go/core latest
go: finding github.com/Azure/azure-sdk-for-go/arm/disk latest
go: finding github.com/Azure/azure-sdk-for-go/arm/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/subscriptions latest
go: finding github.com/Azure/azure-sdk-for-go/arm/network latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/resources latest
go: finding github.com/Azure/azure-sdk-for-go/arm/authorization latest
go: finding github.com/Azure/azure-sdk-for-go/arm/compute latest
go: finding github.com/Azure/azure-sdk-for-go/arm latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources latest
go: finding github.com/lxc/lxd latest
% go mod tidy
go: finding github.com/masterzen/azure-sdk-for-go/core/http latest
go: finding github.com/masterzen/azure-sdk-for-go/core/tls latest
go: finding github.com/masterzen/azure-sdk-for-go/core latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/subscriptions latest
go: finding github.com/Azure/azure-sdk-for-go/arm/disk latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/resources latest
go: finding github.com/Azure/azure-sdk-for-go/arm/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/network latest
go: finding github.com/Azure/azure-sdk-for-go/arm/authorization latest
go: finding github.com/Azure/azure-sdk-for-go/arm/compute latest
go: finding github.com/lxc/lxd latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources latest
go: finding github.com/Azure/azure-sdk-for-go/arm latest
@hdonnay
Copy link

hdonnay commented Aug 29, 2018

I had this issue with a message about finding github.com/google/martian (an indirect dependency via cloud.google.com/go/storage.test) every time, I assume because of some weirdness with its tag situation. Forcing a newer revision via go get github.com/google/martian@master fixed it.

I think there's something to do with indirect dependencies not providing packages at the chosen module version, but that's just a wild guess.

@bcmills
Copy link
Contributor

bcmills commented Sep 25, 2018

github.com/google/martian is weird: it has two valid prerelease version tags (v2.0.0-beta.2 and v2.0.0-beta) from three years ago, one invalid tag (v1.0), and no valid release-version tags, and in the three years since the prerelease tag it has gotten a lot of new packages.

Azure is weird too, but I don't understand why yet. (See #27627.)

@bcmills bcmills added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Sep 27, 2018
@bcmills bcmills self-assigned this Sep 27, 2018
@bcmills bcmills added this to the Go1.12 milestone Sep 27, 2018
@myitcv
Copy link
Member

myitcv commented Oct 17, 2018

As another example of an "unstable" go mod tidy (by "unstable" I mean that for two consecutive calls to go mod tidy, all things being equal, the second should produce no output, else it is "unstable"):

https://gist.github.com/myitcv/65994e8083770e0a518c30cf815db47d

The first gist file shows that go mod tidy is "unstable" on a v0.13.0 checkout of Buffalo.

The second gist file shows that go mod tidy is "stable" where that same version of Buffalo is a dependency of another module.

What's interesting however is the contents of the go.mod in that second gist:

module example.com/blah

require (
	github.com/cockroachdb/apd v1.1.0 // indirect
	github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c // indirect
	github.com/gobuffalo/buffalo v0.13.0
	github.com/gobuffalo/fizz v1.0.12 // indirect
	github.com/jackc/fake v0.0.0-20150926172116-812a484cc733 // indirect
	github.com/jackc/pgx v3.2.0+incompatible // indirect
	github.com/jmoiron/sqlx v1.2.0 // indirect
	github.com/satori/go.uuid v1.2.0 // indirect
	github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24 // indirect
)

Given Buffalo is already a module, I would only have expected a single require directive, namely:

github.com/gobuffalo/buffalo v0.13.0

(that said, the initial go mod tidy in the first gist suggests that go.mod has not been fully tidied, so this expectation doesn't hold 100%. And that's before any issues that might be influencing what's going on here)

Perhaps the other contents of the go.mod give some sort of clue as to what's going wrong here?

@rogpeppe
Copy link
Contributor Author

Here's some more oddness demonstrated with the buffalo example. If I create a dumb wrapper package as follows:

package main

import _ "github.com/gobuffalo/buffalo"

func main() {
}

Then:

go mod init example.com/foo
go mod tidy
go get github.com/gobuffalo/buffalo@v0.13.0

This created a go.mod file with more entries than expected, as mentioned by @myitcv above:

module example.com/foo

require (
	github.com/cockroachdb/apd v1.1.0 // indirect
	github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c // indirect
	github.com/gobuffalo/buffalo v0.13.0
	github.com/gobuffalo/fizz v1.0.12 // indirect
	github.com/jackc/fake v0.0.0-20150926172116-812a484cc733 // indirect
	github.com/jackc/pgx v3.2.0+incompatible // indirect
	github.com/satori/go.uuid v1.2.0 // indirect
	github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24 // indirect
)

Then I checked out github.com/gobuffalo/buffalo@v0.13.0 and went into that directory.
After running go mod tidy there, one extra dependency was added (github.com/jmoiron/sqlx@v1.2.0) but even after that, some of the dependencies mentioned in the dumb wrapper weren't mentioned.
The results of go list -m all that I'd expect to be very similar, were quite different. Here's the diff:

1c1
< example.com/foo
---
> github.com/gobuffalo/buffalo
5,6d4
< github.com/cockroachdb/apd v1.1.0
< github.com/cockroachdb/cockroach-go v0.0.0-20181001143604-e0a95dfd547c
14d11
< github.com/gobuffalo/buffalo v0.13.0
19d15
< github.com/gobuffalo/fizz v1.0.12
51,53c47
< github.com/jackc/fake v0.0.0-20150926172116-812a484cc733
< github.com/jackc/pgx v3.2.0+incompatible
< github.com/jmoiron/sqlx v0.0.0-20180614180643-0dae4fefe7c0
---
> github.com/jmoiron/sqlx v1.2.0
55d48
< github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
82d74
< github.com/satori/go.uuid v1.2.0
85d76
< github.com/shopspring/decimal v0.0.0-20180709203117-cd690d0c9e24
109d99
< google.golang.org/appengine v1.2.0

To take one example, the github.com/gobuffalo/fizz package seems to be included in the dumb wrapper package but not in the github.com/gobuffalo/buffalo module, which is quite odd.

Further anomalies ensue:

% go list -m
github.com/gobuffalo/buffalo
% go mod why -m github.com/gobuffalo/fizz
# github.com/gobuffalo/fizz
(main module does not need module github.com/gobuffalo/fizz)
% go mod why github.com/gobuffalo/fizz
# github.com/gobuffalo/fizz
github.com/gobuffalo/buffalo
github.com/gobuffalo/pop
github.com/gobuffalo/fizz

go mod graph includes this entry:

github.com/gobuffalo/buffalo-pop@v1.0.5 github.com/gobuffalo/pop@v4.8.3+incompatible

but then no further entries for the dependencies of github.com/gobuffalo/pop@v4.8.3+incompatible, which does have several dependencies, including on github.com/gobuffalo/fizz.

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

when we looked at something similar today it seems tidy is doing something similar to go get -u which means if you're using go get -u=patch go mod tidy will break your repo.

@myitcv
Copy link
Member

myitcv commented Oct 19, 2018

@stevenh

it seems tidy is doing something similar to go get -u

Do you have a repro that shows go mod tidy behaving like go get -u?


Just linking to another go mod tidy issue for reference: #27868

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

Our reproduction case was quite simple:

  1. Create a repo with a single dependency
  2. Set the dependency version to not the latest minor or patch revision e.g. latest: 1.6.4, set: 1.4.0 exists: 1.4.3 our dependency was github.com/influxdata/influxdb
  3. Run go get -u=patch you'll see it updates from 1.4.0 to 1.4.3
  4. Run go mod tidy you'll see it updates to 1.6.4

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

I'm trying to create a simple repo case as there may have been something else a play.

@myitcv
Copy link
Member

myitcv commented Oct 19, 2018

@stevenh I'm not seeing that:

$ export GOPATH=$(mktemp -d)
$ export PATH=$GOPATH/bin:$PATH
$ cd $(mktemp -d)
$ go mod init example.com/hello
go: creating new go.mod: module example.com/hello
$ cat <<EOD >main.go
package main
import _ "github.com/influxdata/influxdb"
func main() {}
EOD
$ go get github.com/influxdata/influxdb@v1.4.0
go: finding github.com/influxdata/influxdb v1.4.0
go: downloading github.com/influxdata/influxdb v1.4.0
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.0
$ go get -u=patch
go: finding github.com/influxdata/influxdb v1.4.3
go: downloading github.com/influxdata/influxdb v1.4.3
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.3
$ go mod tidy
$ go list -m all
example.com/hello
github.com/influxdata/influxdb v1.4.3

Do you have another dependency here which requires v1.6.3? As a bit of a long shot, there's a chance you hadn't done anything that required that dependency (the one that requires v1.6.3) and that go mod tidy was the first time MVS actually ran across all dependencies. What does:

$ go mod graph | grep influx

give you?

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

Ok here's a set of repo steps from a blank directory:

mkdir wibble
cat <<EOT >wibble/test.go
package main

import (
        "fmt"
        "github.com/influxdata/influxdb/query"
)

func main() {
        fmt.Println("go mod test")
}
EOT
go mod init
go get -u github.com/influxdata/influxdb@v1.3.9
grep influxdb go.mod
go get -u=patch
grep influxdb go.mod
go mod tidy
grep influxdb go.mod

The final grep here will show:

        github.com/influxdata/influxdb v1.6.4

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

go mod graph | grep influx
github.com/multiplay/gomod github.com/influxdata/influxdb@v1.6.4
github.com/multiplay/gomod github.com/influxdata/influxql@v0.0.0-20180925231337-1cbfca8e56b6```

@myitcv
Copy link
Member

myitcv commented Oct 19, 2018

@stevenh that's happening because github.com/influxdata/influxdb/query only came into existence in >= v1.4.0. So the go tool resorts to getting the latest

@stevenh
Copy link
Contributor

stevenh commented Oct 19, 2018

Interesting, so why doesn't the go get -u=patch detect that and update the dependency?

I'll keep playing as the original issue we had on our main repo which it at 1.4.0, go get -u=patch updated it to 1.4.3 but then something else forced it to 1.6.4

Thanks for investigating that, most appreciated.

@bcmills
Copy link
Contributor

bcmills commented Nov 15, 2018

@rogpeppe, can you still reproduce this? I notice that github.com/google/martian added a well-formed tag (v2.1.0), although I still haven't been able to dig into the Azure issue.

@rogpeppe
Copy link
Contributor Author

@bcmills Yes, I just tried with the latest release and it still happens.
Example:

% go mod tidy
go: finding github.com/masterzen/azure-sdk-for-go/core/http latest
go: finding github.com/masterzen/azure-sdk-for-go/core/tls latest
go: finding github.com/masterzen/azure-sdk-for-go/core latest
go: finding github.com/Azure/azure-sdk-for-go/arm/disk latest
go: finding github.com/Azure/azure-sdk-for-go/arm/network latest
go: finding github.com/Azure/azure-sdk-for-go/arm/storage latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/resources latest
go: finding github.com/Azure/azure-sdk-for-go/arm/compute latest
go: finding github.com/Azure/azure-sdk-for-go/arm/authorization latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources/subscriptions latest
go: finding github.com/lxc/lxd latest
go: finding github.com/Azure/azure-sdk-for-go/arm latest
go: finding github.com/Azure/azure-sdk-for-go/arm/resources latest

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

I can't reproduce this from the HEAD revision of github.com/juju/charmstore-client because it seems to depend on a private repo, but I do see it at the v2.3.0 revision.

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

The Azure thing is just some combination of #27063 and #27102. github.com/Azure/azure-sdk-for-go removed the arm subdirectory starting at v14.0.0.

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

Same for github.com/lxc/lxd: the top-level package in that repo was removed at lxd-2.17, and for some reason they didn't consider that a big enough change to warrant a major version...

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

Same for go.opencensus.io/resource. The resource package was added on the master branch, but is not present in the latest tagged version (v0.18.0). (See also census-instrumentation/opencensus-go#998.)

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

After running

go get -m go.opencensus.io@master github.com/lxc/lxd@lxd-2.16 github.com/Azure/azure-sdk-for-go@v12.5.0-beta

go mod tidy is stable in github.com/juju/charmstore-client at v2.3.0.

$ git clone https://github.com/juju/charmstore-client.git
[…]
$ cd charmstore-client/
charmstore-client$ git checkout v2.3.0
[…]
charmstore-client$ go mod init
go: creating new go.mod: module github.com/juju/charmstore-client
go: copying requirements from dependencies.tsv

charmstore-client$ go get -m go.opencensus.io@master github.com/lxc/lxd@lxd-2.16 github.com/Azure/azure-sdk-for-go@v12.5.0-beta
go: finding github.com/mattn/go-colorable v0.0.6
[…]

charmstore-client$ go mod tidy
go: downloading github.com/juju/juju v0.0.0-20180522155248-f51f7f4c2327
[…]

charmstore-client$ go mod tidy

charmstore-client$

@bcmills
Copy link
Contributor

bcmills commented Dec 12, 2018

Closing as a dup of #27063 and/or #27102. Thanks for the report!

@rsc rsc unassigned bcmills Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge modules NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants