On 2018-11-08, Jeff Kowalczyk
Aleksa Sarai writes:
We are going to be facing quite a few issues in the near future because of Go modules. Go 1.11 added this concept of modules, which is supposed to eventually replace vendor/ and all of the other awful systems Go developers have had to deal with for the many years.
Indeed, I'm very glad to have modules in Go. The transition period will be a challenge, as upstream Go projects migrate at their own pace from using 'dep' et al. and/or complex targets in their Makefile to gather dependency sources. My hope is that in the not too distant future, most Go projects will be buildable offline, without GOPATH setup, following a 'go mod vendor' call from an OBS source service, as you mention below.
In order to stop "go build" from downloading stuff from the internet (which is disallowed in OBS obviously) you need to pass "-mod=vendor".
The kicker is that only Go 1.11 supports "-mod=vendor", while "go build" supports Go modules as far back as Go 1.9.
Did you mean Go modules or 'finds dependencies in vendor/' (Go 1.6)?
I meant Go modules, but see below (they backported some module features but I actually just misremembered what the problem was).
So when building currently we have to delete go.mod and go.sum.
Can you expand a bit on the cases where go.{mod,sum} need to be deleted? With go1.11 I've been using env var GO111MODULE={auto|on|off}, does that cover those cases?
I just realised that I misrepresented what the issue was. The problem is that I need to build restic for Leap which has an older Go compiler (1.9 is the latest I think -- with 1.10 coming out once they get the new Docker from SLES). The issue was that I needed to find a way to build with both Go 1.11 and Go 1.9 (where there was already a pre-populated vendor/ tree) and Go 1.11 would not use the vendor/ tree by default. -mod=vendor fixes this problem but obviously that doesn't work on Go 1.9. So I think you're right that GO111MODULE=off should work since older Go versions will ignore it. I will test this (the go.mod thing was mostly a hack to get the build to work so I could get my backups working again).
I proposed about a year ago that we should have some OBS service to deal with the vendoring problem by automating the pulling and setting up of vendor directories. Personally I think it's about time I dust off that idea and actually get to work on it -- since we now are going to need it for Go since (as I've discovered) some projects don't commit a vendor/ anymore.
I've been thinking about this as well and intend to contribute. What implementation are you envisioning for an OBS source service: calling the 'go mod vendor' CLI from one of bash, python, or a Go executable?
I would aim for it to be more generic so that we can fix problems with languages like Node. Rust was one other pitch I had at the time -- but I've heard that the rust2rpm story is effectively finished and so there's nothing to fix there (I don't really like the *2rpm model, but maybe I just like whinging for no good reason :P). It would have to extract the source archive, figure out what vendoring system it's using and then split out the vendor tree (either by generating a new one in the case of "go mod vendor" or other vendoring tools or by taking the existing one). Then it should hopefully generate some .spec metadata so that legal and maintenance know what on earth is inside "vendor.tar.xz". I am quite worried how nested vendors with modules will work. This in particular will be quite a nightmare because I don't think that "go mod vendor" will recursively handle vendor/s that use go modules (but I'd have to check). -- Aleksa Sarai Senior Software Engineer (Containers) SUSE Linux GmbH https://www.cyphar.com/