Breaking out of GOPATH with Go Modules

About a year ago, I (finally) started learning Go. I started working on a small CLI tool for creating Terraform plans from templates. Nothing too fancy, just a small tool to automate manual and error-prone work. It meant getting to know some of the basics, like CLI arguments and flags, structs, interaction, handling files, rendering templates and working with YAML. Everyone on our team was new to Go, but we actually made quite a bit of progress pretty quickly, and the CLI tools works rather nicely. But it’s safe to say I’m by no means a Go veteran.

One of the things that annoyed me about Go was that it forced my hand with regard to where to keep my code. Usually I organize my git repos around companies or projects, like this:

/Users/benny/git
├── a-redacted-company
├── a-redacted-project
├── dotfiles
├── personal
└── stove

Since the CLI tool I mentioned was part of a bigger project, I would usually keep the code in $HOME/git/someproject/clitool, but Go wouldn’t allow me to do that. So instead I had to keep it in $GOPATH/src/blah.tld/someproject/clitool. Not a very big deal, but annoying enough that I would create a symlink so the code would be where all the other code was, which was fine until I wanted to run any Go command (e.g. go build); all commands would fail because they were being run outside of $GOPATH. I did my fair share of yelling at clouds this year for this very reason.

To make matters worse, we would occasionally run into build failures that weren’t consistent. Who doesn’t love some ‘it works on my machine’ on a Monday morning? Well, I certainly don’t, and it gets worse when your local build fails and your only option is to spam your CI to see if your code works. As it turns out, we were being sloppy with regard to managing dependencies on our local systems, but Go didn’t really make our lives easy here.

So as much as I liked writing Go, I didn’t quite enjoy how it forced a workflow that was unlike any other language I’ve worked with, and didn’t have ‘proper’ dependency management. Until recently, that is.

Go 1.11 fixes everything!

OK, maybe that’s a tad dramatic, but it sure makes life a lot easier. Thanks to Go Modules I can now store my code wherever I want, instead of being forced into $GOPATH. It also offers usable dependency management, including the option to specify versions. Yay!

You can read about all the nitty-gritty details by following the link above, but here’s what you’ll really want to know. How do I keep my code outside of $GOPATH and still manage to do everything? Well, let’s assume I’m going to write a new CLI tool called terraform-monkey and I’ll be releasing it on Github.

# the key to this all is using Go 1.11 or newer 
$ go version
go version go1.11.1 darwin/amd64

# let's assume I already have an existing repo with code
$ cd ~/git/personal/terraform-monkey

# initialize the new module
$ go mod init github.com/bennycornelissen/terraform-monkey
go: creating new go.mod: module github.com/bennycornelissen/terraform-monkey

# let's build it!
$ go build
$ ./terraform-monkey

Awesome! Now, what about those dependencies?

When we initialized our gm module, a new file was created: go.mod. Not only does it contain the name of our new module, it will also keep tabs on our dependencies, even implicit/indirect dependencies. So since I’m using YAML files as my ‘data source’ when rendering Terraform plans, I’ve added a dependency:

$ cat go.mod
module github.com/bennycornelissen/terraform-monkey

require (
	gopkg.in/yaml.v2 v2.2.1
)

By keeping this file next to the code, it’s easy to commit it to Git as well, and this way everyone can easily use the exact same versions of the exact same dependencies, have reproducible builds, and keep the repo wherever they want in the process.

I’ll call that a win :-)