Docker healthchecks in distroless Node.js

Docker has a built-in way of running health checks to make sure that, well, the container is healthy.

You can define these in your Dockerfile using the HEALTHCHECK directive, override them on the command line using various docker run --health-* flags, or in a Docker Compose file using the healthcheck property.

A pretty common configuration for a web server would look something like:

HEALTHCHECK --interval=5m --timeout=3s \
  CMD curl -f http://localhost/ || exit 1

Essentially it's just a command to run (along with various intervals, grace periods and timeouts) which will return exit code 0 if everything is fine, or 1 if there's an error.

In a normal image based on Alpine, Debian or some other regular OS, this works fine, but when it comes to distroless images it doesn't work.

Distroless images

What exactly are distroless images?

Distroless images, such as those distributed by Google, contain only the bare minimum runtime dependencies to run your application. They provide images for Java, Node & Python as well several others. It should be noted that strictly speaking they are based on a Linux distro, Debian to be precise - but they're very trimmed down.

These images do not contain anything that isn't absolutely essential. They don't contain package managers, typical Linux utilities or even a shell. Forget running docker exec -it <my_container> /bin/sh - it won't work with a distroless image.

And critically, there's no curl or wget either, so the normal way of running a healthcheck won't work.

Despite these inconveniences, the rationale behind using distroless images is that they are potentially more secure, and the images are small - the base Debian image is just 2MB, half the size of Alpine Linux (~5MB) and much smaller than the regular Debian Docker image (~120MB):

$ docker images --format "{{.Repository}}:{{.Tag}} {{.Size}}"
gcr.io/distroless/static-debian11:latest 2.34MB
alpine:latest 7.05MB
debian:latest 124MB

Healthchecks with curl or wget

I mentioned earlier that often you might use curl or wget for your healthchecks, but obviously these don't work in distroless where those binaries don't exist.

Your first thought might be: "OK, so just add the binaries to your image". Sure, you can do that:

FROM busybox:uclibc AS builder
FROM gcr.io/distroless/static-debian11

COPY --from=builder /bin/wget /usr/bin/wget

But now you've begun to erode the value of using the distroless image in the first place - you've added another binary, and increased the size of your image:

$ docker history distroless-with-wget
IMAGE          CREATED         CREATED BY                                SIZE      COMMENT
48fdcd4b0638   2 minutes ago   COPY /bin/wget /usr/bin/wget # buildkit   1.17MB    buildkit.dockerfile.v0
<missing>      47 hours ago    bazel build ...                           2.34MB

While this will work, my preference is to use the runtime that is already baked into the distroless image I'm using - in my case, Node.js.

Native Node.js healthcheck

My language of choice for building web applications is Javascript, running on the server using Node.js. It's very easy to build a simple healthcheck script using Node.js:

# healthcheck.js

const http = require('node:http');

const options = { hostname: 'localhost', port: process.env.PORT, path: '/api/health', method: 'GET' };

http
  .request(options, (res) => {
    let body = '';

    res.on('data', (chunk) => {
      body += chunk;
    });

    res.on('end', () => {
      try {
        const response = JSON.parse(body);
        if (response.healthy === true) {
          process.exit(0);
        }

        console.log('Unhealthy response received: ', body);
        process.exit(1);
      } catch (err) {
        console.log('Error parsing JSON response body: ', err);
        process.exit(1);
      }
    });
  })
  .on('error', (err) => {
    console.log('Error: ', err);
    process.exit(1);
  })
  .end();

In my example above, the healthcheck script will attempt to make a simple HTTP request to localhost, on the port specified in the PORT environment variable, and with a path of /api/health.

At that endpoint, I return a simple JSON response such as:

$ curl http://localhost/api/health
{"healthy":true,"version":"1.2.3"}

The healthcheck script will return exit code 0 if it sees that healthy is true and exit code 1 in all other situations.

Adding this to the Dockerfile looks something like this:

# Dockerfile

COPY healthcheck.js .

HEALTHCHECK --interval=30s --timeout=30s --start-period=5s --retries=3 CMD ["/nodejs/bin/node", "healthcheck.js"]

Note that we have to use the exec form of the CMD instead of the shell form (i.e. the array of strings), because there's no shell!

Does it work?

Assuming that your application is running inside the container correctly, yes! If you run docker ps you should see the status listed as healthy:

$ docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED         STATUS                   PORTS                  NAMES
edd732787da1   my-app    "/nodejs/bin/node se…"   4 minutes ago   Up 4 minutes (healthy)   0.0.0.0:3000->80/tcp   my-app

It's also worth mentioning that any text output sent to stdout or stderr by your healthcheck can be found using docker inspect which can be useful when debugging. But if all looks good, you should see something like this:

$ docker inspect my-app | jq ".[0].State.Health"
{
  "Status": "healthy",
  "FailingStreak": 0,
  "Log": [
    {
      "Start": "2023-02-12T20:15:12.550645484Z",
      "End": "2023-02-12T20:15:12.713112121Z",
      "ExitCode": 0,
      "Output": ""
    }
  ]
}

Wrapping up

I like to use distroless images for my production apps wherever possible to minimize the image sizes and hopefully improve security. Healthchecks are an important part of any Docker deployment, and they can work just as well in distroless containers.

In fact, using the runtime that your application uses instead of curl or wget works just as well in non-distroless (distroful?) images too. Using the same runtime that your application uses can often give you a lot of useful capability - such as being able to parse a JSON response as in my example.