You may have encountered situations where you have to push and pull container images frequently when developing locally using minikube. We did too in imager200. Imagine that your container images are a few hundreds of megabytes, and your internet connection is not that fast, how would your dev lifecycle be like if you are pushing your image to a remote registry like Github or AWS ? Well, you will have to wait and wait every time you want to test the new changes, because pushing the image from your host to the registry can take some time (the upload bandwidth is usually not that great), and once you deploy the K8 manifests to minikube, you will have to wait again simply because the minikube machine has a distinct docker daemon and is not aware of your host container image cache. Another factor that makes things clearly inefficient is the fact that minikube machines/hosts have usually small disk sizes that can quickly reach full capacity, so one would have to run docker system prune every now and then to free some space. Relying on the minikube image cache does not help much as well. We encountered those issues too when developing features for imager200, until we found out about the minikube registry addon. In this post, we would like to share how we setup the minikube registry addon and how it helped us.

Introducing the minikube registry addon:

The registry addon launches a local container registry inside the minikube master. It can be enabled during the first launch by adding the argumemt --addons=registry to the minikube command. For example: minikube start --driver=virtualbox --disk-size=10g --memory=4096m --addons=registry. This is only needed during the first launch. For subsequent starts, minikube start is enough. If you have an already existing minikube cluster, you can delete it using minikube delete and create a new one. Once minikube is launched you can test that the registry is running by hitting the minikube host ip at port 5000, for example: curl $(minikube ip):5000. The response shoud be 200.

It is also important to note that the registry is exposed over http, and not https, and since docker expects the registries endpoints to be secured using TLS, the push and pull commands will likely fail if you do not configure docker to accept insecure connections. To do so, the insecure-registries config option need to be set in the docker daemon.

If you run dockerd --help in your local machine, you will find the following among the options:

--insecure-registry list                  Enable insecure registry communication

The best way to set permanent options for the docker daemon is by creating a daemon.json file under /etc/docker (for linux, for other OSes, checkout this SO thread)

for example, assuming that our minikube address is 192.168.59.101, our daemon.json can look like:

{
"insecure-registries" : [ "192.168.59.101:5000" ]
//other options here...
}

After adding the insecure-registries, the docker daemon needs to be restarted (e.g systemctl restart dockerd).

Working with containers in action:

Now that our local registry is set, we are ready to start pushing images to it. To do so, we can use the minikube ip with the port as the name of the registry, for example:

docker push $(minikube ip):5000/my-app-image:latest

Compared to a remote registry, the push time is drastically reduced.

Once the imaged is pushed, It can be referred to from your K8 manifests by replacing the minikube ip address by localhost, for example:

spec:
  containers:
    - name: my-app-image
      image: localhost:5000/my-app-image:latest

The pulling from the K8 side is also considerably fast (expected since the registry is on the same host).

pretty neat, no?