switch to using go modules

[nirnanaaa]

as the title already hints, this PR aims to remove gopkg in favor of go modules.

Since the vendor folder get's messy pretty easily, I've split up the PR into two commits.

[rhysemmas]

Thanks for contributing @nirnanaaa - personally, I would rather that we drop the vendor directory completely and just allow go mod to fetch the dependencies when required, let me know your thoughts.

[nirnanaaa]

yeah interesing point. I think this discussion could get philosophical if we discuss vendor vs. no-vendor. I have no real opinion on it. really. I would say, if there is no need because of some private dependencies we can as well remove the vendor folder. WDYT?

[nirnanaaa]

can someone tell my why my drone build is failing? apparently I have no permission to access the build details page.

[waiariki-koia]

Hey @nirnanaaa, the errors were because Drone couldn't find any packages. Adding

    environment:
      - GO111MODULE=on

to .drone.yml should fix it!

[nirnanaaa]

Is there a way to enable contributors to actually see the CI? The error ofc makes sense. I forgot that.

[waiariki-koia]

Unfortunately we can't give contributors access as they have to be apart of our Github organisation in order to get access to Drone.

The builds are failing because the GO111MODULE environment variable also needs to be included in the "benchmarks" step.

[nirnanaaa]

well, this somehow turns off any external contributor. Anyways, seems fixed now.

Maybe in the future this project could consider moving to an open source CI.

[surajnarwade]

I tried PR locally,

$ make docker
docker image build -t "quay.io/uswitch/kiam:pr_315" .
Sending build context to Docker daemon  361.5kB
Step 1/11 : FROM golang:1.12-alpine3.10 as build
 ---> 841429c8613b
Step 2/11 : ENV GO111MODULE=on
 ---> Using cache
 ---> 862b57b1f9e4
Step 3/11 : RUN apk add --no-cache make
 ---> Using cache
 ---> 4d45f6be7f10
Step 4/11 : WORKDIR /go/src/github.com/uswitch/kiam
 ---> Using cache
 ---> 5c319efcf68c
Step 5/11 : ADD . .
 ---> Using cache
 ---> b20ea6b618cb
Step 6/11 : RUN go mod download
 ---> Running in 2afd36d3cb2c
go: github.com/uswitch/k8sc@v0.0.0-20170525133932-475c8175b340: git init --bare in /go/pkg/mod/cache/vcs/923be03619cf0792a4198b54d923116b9110b5d6ce69480904dae5aef231b11c: exec: "git": executable file not found in $PATH
go: github.com/gogo/protobuf@v1.0.0: git init --bare in /go/pkg/mod/cache/vcs/008745bab24c72f9836ba820e1058a38f5f3bf297e848d8fddb723e5eaefb30a: exec: "git": executable file not found in $PATH
go: github.com/beorn7/perks@v0.0.0-20180321164747-3a771d992973: git init --bare in /go/pkg/mod/cache/vcs/1a437ed9cd95ef0b3850ad46f66cacade14a3fca4584dad97c8e5299ff7b4735: exec: "git": executable file not found in $PATH
go: github.com/cenkalti/backoff@v2.0.0+incompatible: git init --bare in /go/pkg/mod/cache/vcs/79e8d1963969a9df3425d256a195fff829ee975571acfe13306e2ad79ea04255: exec: "git": executable file not found in $PATH
go: github.com/spf13/pflag@v1.0.1: git init --bare in /go/pkg/mod/cache/vcs/389cbbf79b0218a16e6f902e349b1cabca23e0203c06f228d24031e72b6cf480: exec: "git": executable file not found in $PATH
go: github.com/ghodss/yaml@v1.0.0: git init --bare in /go/pkg/mod/cache/vcs/6feb2f2826a02fc310b017108da5b5af632374eb86245293fd4d7f2bbcd4866e: exec: "git": executable file not found in $PATH
go: github.com/howeyc/gopass@v0.0.0-20170109162249-bf9dde6d0d2c: git init --bare in /go/pkg/mod/cache/vcs/3d8176d886c7e89665ec5970cc3772bae7692e916fbfc1de38ae5b6443b35e96: exec: "git": executable file not found in $PATH
go: github.com/vmg/backoff@v1.0.0: git init --bare in /go/pkg/mod/cache/vcs/4a9a3020a2e9e7a52d41ab41d8206f95aed63103106a0a129da85829140c2166: exec: "git": executable file not found in $PATH
go: github.com/alecthomas/template@v0.0.0-20160405071501-a0175ee3bccc: git init --bare in /go/pkg/mod/cache/vcs/832659f94734cc5b04953d078fbd1c7184d3f0b86cb88e1d609b37112368d2c7: exec: "git": executable file not found in $PATH
go: github.com/go-ini/ini@v1.37.0: git init --bare in /go/pkg/mod/cache/vcs/f63afdd23e3f4a0ba51aa6e624a654abefeae191beca1b19ebd2c902c09ef9d0: exec: "git": executable file not found in $PATH
go: github.com/prometheus/procfs@v0.0.0-20180601124529-94663424ae5a: git init --bare in /go/pkg/mod/cache/vcs/98c949bd9adf825ba7df08da3b4256328548aa917c603094bfc20a4042505d6d: exec: "git": executable file not found in $PATH
go: github.com/prometheus/client_model@v0.0.0-20171117100541-99fa1f4be8e5: git init --bare in /go/pkg/mod/cache/vcs/2a98e665081184f4ca01f0af8738c882495d1fb131b7ed20ad844d3ba1bb6393: exec: "git": executable file not found in $PATH
go: github.com/hashicorp/golang-lru@v0.0.0-20180201235237-0fb14efe8c47: git init --bare in /go/pkg/mod/cache/vcs/1eaaa73a95627cc6e520c43f6cce496386f24ff1b7fa8af090724d7660d929e9: exec: "git": executable file not found in $PATH
go: github.com/sirupsen/logrus@v1.0.5: git init --bare in /go/pkg/mod/cache/vcs/ce05746539f15caa8470a1cb206cefcfc18421bcd2c6e35153546df051d6a96e: exec: "git": executable file not found in $PATH
go: github.com/golang/protobuf@v1.2.0: git init --bare in /go/pkg/mod/cache/vcs/a20e27c072e7660590311c08d0311b908b66675cb634b0caf7cbb860e5c3c705: exec: "git": executable file not found in $PATH
go: github.com/json-iterator/go@v0.0.0-20180315132816-ca39e5af3ece: git init --bare in /go/pkg/mod/cache/vcs/5de4f584a32ae7229f646f407cc2c0ef18113d510ff832a496ebdc9bec6cb9e7: exec: "git": executable file not found in $PATH
go: github.com/grpc-ecosystem/go-grpc-middleware@v1.0.0: git init --bare in /go/pkg/mod/cache/vcs/ba4252397c51dfcbb8f1bbf8ae7d341c3cc72dc75d6c6c307a6847d62170009d: exec: "git": executable file not found in $PATH
go: github.com/googleapis/gnostic@v0.2.0: git init --bare in /go/pkg/mod/cache/vcs/a92061fdd8a1f889227ce02d465967bfd665a741b6ccb9066d078e07202fef10: exec: "git": executable file not found in $PATH
go: github.com/coreos/go-iptables@v0.3.0: git init --bare in /go/pkg/mod/cache/vcs/2adf1e2a3fdebdf3f9ca1e708cf134be91e300b835199ce5c6a6c0ebe62e26a1: exec: "git": executable file not found in $PATH
go: github.com/gorilla/context@v1.1.1: git init --bare in /go/pkg/mod/cache/vcs/97ea9650f028793bd42d555dba7196a11b404ffbdbebffc0084e752976008464: exec: "git": executable file not found in $PATH
go: github.com/google/gofuzz@v0.0.0-20170612174753-24818f796faf: git init --bare in /go/pkg/mod/cache/vcs/494b36eea28d4b5317329e131e5193b575157a2ef53fa71b1b1175513e222398: exec: "git": executable file not found in $PATH
go: github.com/imdario/mergo@v0.3.4: git init --bare in /go/pkg/mod/cache/vcs/8f4a21d5581de7e927389c1f577216bbb221653904fa4465baf5a8d072175f94: exec: "git": executable file not found in $PATH
go: github.com/prometheus/client_golang@v0.9.0-pre1: git init --bare in /go/pkg/mod/cache/vcs/3785b9359e0a97a9dc849585696cf17189b86b713b7e59c2842fb3c44d3ff610: exec: "git": executable file not found in $PATH
go: github.com/fortytw2/leaktest@v1.3.0: git init --bare in /go/pkg/mod/cache/vcs/c8b9ddff9acab6cca57383df47daedba1c34a0e8f1f68f4f8f10a25253a43fef: exec: "git": executable file not found in $PATH
go: github.com/patrickmn/go-cache@v2.1.0+incompatible: git init --bare in /go/pkg/mod/cache/vcs/b05cbb7d90cdcbcfa5ac7177241e1094d83905685f8d4e38824e0a1c9dba126d: exec: "git": executable file not found in $PATH
go: github.com/davecgh/go-spew@v1.1.0: git init --bare in /go/pkg/mod/cache/vcs/b9a4b9bbdb4a59723f2348415ad7ffda91568455a1cfd92e97976132bdfbaf57: exec: "git": executable file not found in $PATH
go: github.com/jmespath/go-jmespath@v0.0.0-20160202185014-0b12d6b521d8: git init --bare in /go/pkg/mod/cache/vcs/7b1106ecb177564b0bc9784f963c6c785e31d09dcd9f08114684d32af620443f: exec: "git": executable file not found in $PATH
go: github.com/alecthomas/units@v0.0.0-20151022065526-2efee857e7cf: git init --bare in /go/pkg/mod/cache/vcs/6466505548ed0f0b2b825c1ba6b822fa76675aa03269697c2b442d0c1c44678d: exec: "git": executable file not found in $PATH
go: github.com/modern-go/reflect2@v0.0.0-20180228065516-1df9eeb2bb81: git init --bare in /go/pkg/mod/cache/vcs/e3da2a565352fc053fb7730886bf07c23545d2a92d4bc6bc166097f9c01f7a88: exec: "git": executable file not found in $PATH
go: github.com/gorilla/mux@v1.6.2: git init --bare in /go/pkg/mod/cache/vcs/6a3c85c1fa560af3c11df6bfabdc0a25c69c3127a5822875d586b38142c71189: exec: "git": executable file not found in $PATH
go: k8s.io/kube-openapi@v0.0.0-20180629012420-d83b052f768a: git init --bare in /go/pkg/mod/cache/vcs/22fe76eef9b94b295c27ec12355677e1028001031745e263eca2f37c7d340067: exec: "git": executable file not found in $PATH
go: k8s.io/apimachinery@v0.0.0-20180515182440-31dade610c05: git init --bare in /go/pkg/mod/cache/vcs/e465b8697dd53d8e0cb9707e58cf9355b7d943c17205f2689000db716d6cb142: exec: "git": executable file not found in $PATH
go: k8s.io/api@v0.0.0-20180521142803-feb48db456a5: git init --bare in /go/pkg/mod/cache/vcs/e5dcdfe061d044413d88cabafb5d13ca80abedb39b7761f921bb5f9d23587372: exec: "git": executable file not found in $PATH
go: k8s.io/client-go@v7.0.0+incompatible: git init --bare in /go/pkg/mod/cache/vcs/70a2f246b82ec44ed4585aa2fce0516a01c564496f1ed2f026b01d18086f5282: exec: "git": executable file not found in $PATH
go: golang.org/x/time@v0.0.0-20180412165947-fbb02b2291d2: git init --bare in /go/pkg/mod/cache/vcs/7b57f807c86645f3c00fd0a072a22a82f6c2bc9a0a9659b07ebacd86742e1a8b: exec: "git": executable file not found in $PATH
go: golang.org/x/net@v0.0.0-20190311183353-d8887717615a: git init --bare in /go/pkg/mod/cache/vcs/4a22365141bc4eea5d5ac4a1395e653f2669485db75ef119e7bbec8e19b12a21: exec: "git": executable file not found in $PATH
go: github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0: git init --bare in /go/pkg/mod/cache/vcs/e865532dcd0e75430d75350d2d92035b589313e311d39d395252ff922102d9f9: exec: "git": executable file not found in $PATH
go: github.com/golang/groupcache@v0.0.0-20180513044358-24b0969c4cb7: git init --bare in /go/pkg/mod/cache/vcs/f7a002ff08195b00c4fb5bb7261149151f6e428ed0b5528aaeb02b6c48b497cf: exec: "git": executable file not found in $PATH
go: github.com/matttproud/golang_protobuf_extensions@v1.0.1: git init --bare in /go/pkg/mod/cache/vcs/c9f9bbb8cc68e928d83601198e57e7c65ab57e6945016296f48e56d8e0f8d013: exec: "git": executable file not found in $PATH
go: github.com/prometheus/common@v0.0.0-20180518154759-7600349dcfe1: git init --bare in /go/pkg/mod/cache/vcs/78eeb4629eb558fa221be26a69bd8019d99c56c9d5e61a056019c7d4845bf714: exec: "git": executable file not found in $PATH
go: github.com/aws/aws-sdk-go@v1.14.2: git init --bare in /go/pkg/mod/cache/vcs/cb1953cbdfd14fc2ffce4dfd06487e8d5a0c96da00d00bdef65874ff644eaa26: exec: "git": executable file not found in $PATH
go: github.com/modern-go/concurrent@v0.0.0-20180306012644-bacd9c7ef1dd: git init --bare in /go/pkg/mod/cache/vcs/f01d3ec57cbfcf96147ea0a8e0eab452d1cb5497383acd5dfe2e3cd6276e112a: exec: "git": executable file not found in $PATH
go: google.golang.org/grpc@v1.23.0: git init --bare in /go/pkg/mod/cache/vcs/53ab5f2f034ba42de32f909aa45670cf730847987f38664c4052b329152ad727: exec: "git": executable file not found in $PATH
go: gopkg.in/alexcesaro/statsd.v2@v2.0.0: git init --bare in /go/pkg/mod/cache/vcs/cfc5bad1bc03655e27766cc3329315b1da1d2d2e7647bf9ff81416b60d9d17e7: exec: "git": executable file not found in $PATH
go: gopkg.in/alecthomas/kingpin.v2@v2.2.6: git init --bare in /go/pkg/mod/cache/vcs/e2b9a9d574a0eccce75e3bfb3c72a92825e717eb27ee84f95275b5ad6af841c4: exec: "git": executable file not found in $PATH
go: gopkg.in/yaml.v2@v2.2.1: git init --bare in /go/pkg/mod/cache/vcs/4f939cf8921e8662ef2d8cf78b0db3bcacccda0d79c9e1b4cef06c640b4a8cef: exec: "git": executable file not found in $PATH
go: gopkg.in/inf.v0@v0.9.1: git init --bare in /go/pkg/mod/cache/vcs/c138926de552f5684690f08e3cdce10e4256ca625f06436262831416339bbc4c: exec: "git": executable file not found in $PATH
go: error loading module requirements
The command '/bin/sh -c go mod download' returned a non-zero code: 1
Makefile:40: recipe for target 'docker' failed
make: *** [docker] Error 1

@nirnanaaa , are you able to build the image?

[nirnanaaa]

most probably not. i'll change the build to a multistaged one.

[surajnarwade]

@nirnanaaa , I think reason is golang:1.12-alpine3.10 image doesn't contains git package which is needed for go mod

[nirnanaaa]

yeah probably. I missed that. I'll create another PR after this one to refactor the dockerfile to more modern build dependencies

[surajnarwade]

go build takes care of downloading the dependencies, we don't need go mod download

[nirnanaaa]

I was referring mostly to what kubebuilder does in caching docker layers: https://github.com/kubernetes-sigs/kubebuilder/blob/master/testdata/project-v2/Dockerfile#L8

[surajnarwade]

cool, TFS

[surajnarwade]

@nirnanaaa as mentioned here, use golang:1.12.5 as base image and we are good 😸 to merge this PR as if we merge this PR, image build will fail

[surajnarwade]

@nirnanaaa ignore my previous comment as you have added git as a dependency 👍

[nirnanaaa]

but you are still right about the Dockerfile in a way that go mod download only makes sense if we copy go.mod and go.sum before the code changes. Changing that right now. let's see

[nirnanaaa]

now it looks much better @surajnarwade

[surajnarwade]

LGTM 🎉

[nirnanaaa]

do we need another approval, or something else before a merge?

[surajnarwade]

cc @rhysemmas

[rhysemmas]

LGTM, thanks for this @nirnanaaa!