Shahidh K Muhammed
6 Mar 2019
•
8 min read
You probably want to use Docker with Go, because:
GOPATH
variables).
Well, you’ve come to the right place.We’ll incrementally build a basic Dockerfile for Go, with live reloading and package management, and then extend the same to create a highly optimized production ready image with ~100x reduction in size. If you use a CI/CD system, image size might not matter, but when docker push
and docker pulls
are involved, a leaner image will definitely help.
If you’d like to jump right ahead to the code, check out the GitHub repo:
go-docker - Sample code and dockerfiles accompanying the blog post The Ultimate Guide to Writing Dockerfiles for Go…github.com
Let’s assume a simple directory structure. The application is called go-docker
and the directory structure is as shown below. All source code is inside src
directory and there is a Dockerfile
at the same level. main.go
defines a web-app listening on port 8080.
go-docker
├── Dockerfile
└── src
└── main.go
```## 1. The Simplest One ##
FROM golang:1.8.5-jessie
WORKDIR /go/src/app
ADD src src
CMD ["go", "run", "src/main.go"]
We are using `debian jessie` here since some commands like `go get` require `git` etc. to be present. Also, all Debian packages are available in case we need them. For production version we’ll use a smaller image like `alpine`.
Build and run this image:
$ cd go-docker $ docker build -t go-docker-dev . $ docker run --rm -it -p 8080:8080 go-docker-dev
The app will be available at http://localhost:8080. Use `Ctrl+C` to quit.
But this doesn’t make much sense because we’ll have to build and run the docker image every time any change is made to the code.
A better version would be to mount the source code into a docker container so that the environment is contained and using a shell inside the container to stop and start `go run` as we wish.
$ cd go-docker $ docker build -t go-docker-dev . $ docker run --rm -it -p 8080:8080 -v $(pwd):/go/src/app \ go-docker-dev bash root@id:/go/src/app# go run src/main.go
These commands will give us a shell, where we can execute `go run src/main.go` and run the server. We can edit `main.go` from host machine and run the code again to see changes, as the the files are mounted directly into the container.
But, what about packages?
## 2. Package Management & Layering ##
[Package management in Go](https://github.com/golang/go/wiki/PackageManagementTools) is still in an experimental stage. There are a couple of tools around, but my favourite is [Glide](https://glide.sh/). We’ll install Glide inside the container and use it from within.
Create two files called `glide.yaml` and `glide.lock` inside `go-docker` directory:
$ cd go-docker $ touch glide.yaml $ touch glide.lock
Change the Dockerfile to the one below and build a new image.
FROM golang:1.8.5-jessie
RUN go get github.com/Masterminds/glide
WORKDIR /go/src/app
ADD glide.yaml glide.yaml ADD glide.lock glide.lock
RUN glide install
ADD src src
CMD ["go", "run", "src/main.go"]
If you look closely, you can see that `glide.yaml` and `glide.lock` are being added separately (instead of doing a `ADD . .`), resulting in separate layers. By separating out package management to a separate layer, Docker will cache the layer and will only rebuild it if the corresponding files change, i.e. when a new package is added or an existing one is removed. Hence, `glide install` won’t be executed for every source code change.
Let’s install a package by getting into the container’s shell:
$ cd go-docker $ docker build -t go-docker-dev . $ docker run --rm -it -v $(pwd):/go/src/app go-docker-dev bash root@id:/go/src/app# glide get github.com/golang/glog
Glide will install all packages into a `vendor` directory, which can be gitignore-d and dockerignore-d. It uses `glide.lock` to lock packages to specific versions. To (re-)install all packages mentioned in `glide.yaml`, execute:
$ cd go-docker $ docker run --rm -it -p 8080:8080 -v $(pwd):/go/src/app \ go-docker-dev bash
root@id:/go/src/app# glide install
The `go-docker` directory has grown a little bit now:
├── Dockerfile ├── glide.lock ├── glide.yaml ├── src │ └── main.go └── vendor/
Don’t forget to add `vendor` to `.gitignore` and `.dockerignore`.
## 3. Live Reloading ##
[codegangsta/gin]( https://github.com/codegangsta/gin) is my favourite among all the live-reloading tools. It is specifically built for Go web servers. We’ll install gin using `go get:`
FROM golang:1.8.5-jessie
RUN go get github.com/Masterminds/glide
RUN go get github.com/codegangsta/gin
WORKDIR /go/src/app
ADD glide.yaml glide.yaml ADD glide.lock glide.lock
RUN glide install
ADD src src
CMD ["go", "run", "src/main.go"]
We’ll build the image and run gin so that the code is rebuilt whenever there is any change inside `src` directory.
$ cd go-docker $ docker build -t go-docker-dev . $ docker run --rm -it -p 8080:8080 -v $(pwd):/go/src/app \ go-docker-dev bash
root@id:/go/src/app# gin --path src --port 8080 run main.go
Note that the web-server should take a `PORT` environment variable to listen to since gin will set a random `PORT` variable and proxy connections to it.
All edits in `src` directory will trigger a rebuild and changes will be available live at `http://localhost:8080`.
Once we are done with development, we can build the binary and run it, instead of using the `go run` command. The binary can be built and served using the same image or we can make use of Docker multi-stage builds to build using a `golang` image and serve using a bare minimum linux container like `alpine`.
## 4. Single Stage Production Build ##
FROM golang:1.8.5-jessie
RUN go get github.com/Masterminds/glide
WORKDIR /go/src/app
ADD glide.yaml glide.yaml ADD glide.lock glide.lock
RUN glide install
ADD src src
RUN go build src/main.go
CMD ["./main"]
Build and run the all-in-one image:
$ cd go-docker $ docker build -t go-docker-prod . $ docker run --rm -it -p 8080:8080 go-docker-prod
The image built will be ~750MB (depending on your source code), due to the underlying Debian layer. Let’s see how we can cut this down.
## 5. Multi Stage Production Build ##
Multi stage builds lets you build programs in a full-fledged OS environment, but the final binary can be run from a very slim image which is only slightly larger than the binary itself.
FROM golang:1.8.5-jessie as builder
RUN go get github.com/Masterminds/glide
WORKDIR /go/src/app ADD glide.yaml glide.yaml ADD glide.lock glide.lock
RUN glide install
ADD src src
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main src/main.go
FROM alpine:3.7
RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/*
WORKDIR /root
COPY --from=builder /go/src/app/main .
CMD ["./main"]
The binary here is ~14MB and the docker image is ~18MB. Thanks to `alpine` awesomeness.
Want to cut down the binary size itself? Read ahead.
## 6. Bonus: Binary Compression using UPX ##
At [Hasura](https://hasura.io/), we have been using [UPX](https://upx.github.io/) everywhere, our CLI tool binary which is ~50MB comes down to ~8MB after compression, making it easy to download. UPX can do extremely fast in-place decompression, without any extra tools since it injects the decompressor into the binary itself.
FROM golang:1.8.5-jessie as builder
RUN apt-get update && apt-get install -y \ xz-utils \ && rm -rf /var/lib/apt/lists/*
ADD https://github.com/upx/upx/releases/download/v3.94/upx-3.94-amd64_linux.tar.xz /usr/local RUN xz -d -c /usr/local/upx-3.94-amd64_linux.tar.xz | \ tar -xOf - upx-3.94-amd64_linux/upx > /bin/upx && \ chmod a+x /bin/upx
RUN go get github.com/Masterminds/glide
WORKDIR /go/src/app ADD glide.yaml glide.yaml ADD glide.lock glide.lock
RUN glide install
ADD src src
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main src/main.go
RUN strip --strip-unneeded main RUN upx main
FROM alpine:3.7
RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/*
WORKDIR /root
COPY --from=builder /go/src/app/main .
CMD ["./main"]
The UPX compressed binary is ~3MB and the docker image is ~6MB.
**~100x reduction in size from where we started from.**
## 7. Dep instead of Glide ##
`dep` is a prototype dependency management tool for Go. Glide is considered to be in a state of support rather than active feature development, in favour of `dep`. Executing `dep init` in a directory with `glide.yaml` and glide.lock will create `Gopkg.toml` and `Gopkg.lock` by reading the glide files.
Adding a new package using dep is similar to glide:
$ dep ensure -add github.com/sirupsen/logrus
`glide install` equivalent is `dep ensure`.
FROM golang:1.8.5-jessie
RUN go get github.com/golang/dep/cmd/dep
WORKDIR /go/src/app
ADD Gopkg.toml Gopkg.toml ADD Gopkg.lock Gopkg.lock
RUN dep ensure --vendor-only
ADD src src
CMD ["go", "run", "src/main.go"]
## 8. Scratch instead of `Alpine` ##
Alpine is useful when you have to quickly access the shell inside the container and do some debugging. For example, shell comes to the rescue while debugging DNS issues in a Kubernetes cluster. We can run `ping/wget` etc. Also, if your application makes API calls to external services over HTTPS, `ca-certificates` need to be present.
But, if you don’t need a shell or ca-certs, but just want to run the binary, you can use `scratch` as the base for the image in multi-stage build.
FROM golang:1.8.5-jessie as builder
RUN apt-get update && apt-get install -y \ xz-utils \ && rm -rf /var/lib/apt/lists/*
ADD https://github.com/upx/upx/releases/download/v3.94/upx-3.94-amd64_linux.tar.xz /usr/local RUN xz -d -c /usr/local/upx-3.94-amd64_linux.tar.xz | \ tar -xOf - upx-3.94-amd64_linux/upx > /bin/upx && \ chmod a+x /bin/upx
RUN go get github.com/golang/dep/cmd/dep
WORKDIR /go/src/app
ADD Gopkg.toml Gopkg.toml ADD Gopkg.lock Gopkg.lock
RUN dep ensure --vendor-only
ADD src src
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main src/main.go
RUN strip --strip-unneeded main RUN upx main
FROM scratch
WORKDIR /root
COPY --from=builder /go/src/app/main .
CMD ["./main"]
The resulting image is just 1.3 MB, compared to the 6MB apline image.
Ground Floor, Verse Building, 18 Brunswick Place, London, N1 6DZ
108 E 16th Street, New York, NY 10003
Join over 111,000 others and get access to exclusive content, job opportunities and more!