From the Couch of Ben Johnson

Father, Principal Engineer at Prodigy Education, serial hyperbolist.

Scale to Zero with Fly.io Machines

I’ve written a bunch of hobby apps over time that I use to keep my life moving forward. I’ve got some small side projects, like http.ist and Lemur that need web hosting. I also have utility apps that filter podcasts from podcast feeds, truncate my RSS news, or show me the swimming schedule for my local pool (their website was terrible).

I’ve been managing a Kubernetes cluster on DigitalOcean for this, but it feels like massive overkill in cost and resources. As an alternative, I’ve been looking at Fly.io, a Heroku-like PasS. Fly has been great for an easy single-command deployment.

In the past few months, Fly has started releasing the next generation of their platform: Fly Machines. They’re lightweight Firecracker VMs that boot quickly and are cheap to run. Fly Machines will eventually be used to provide all new features for Fly’s central app hosting service, but at the moment they don’t have all the bells-and-whistles niceties of Fly’s general development platform. I’m sure this will change over time. But, we can take advantage of them now with a few simple commands.

What’s powerful about Fly Machines today is that we can scale them down to zero and then save on cost while they’re idle. When a request comes in, they can scale up in roughly 300ms, serve traffic and then scale back down again. This makes them an excellent fit for hobby apps.

Let’s look at a sample app and how we’d do this with Fly.

First, you need to write your app. I’ve written cookiemonster that just yells “Cookie!” when you visit it. For Fly to scale the service down when idle, it looks for cases where the app has exited with a status code of 0. If the exit code is anything other than 0, Fly will assume that the app has crashed and will restart it. This is great, because the application developer has total control of when to mark the app as idle and shut it down.

To scale to zero in Node, it might look like this:

const express = require("express");
const app = express();

let lastRequest = Date.now();

app.get("/", (req, res) => {
  lastRequest = Date.now();
  res.send("Cookie!");
});

app.listen(process.env.PORT);

setInterval(() => {
  const oneMinute = 1000 * 60;
  if (Date.now() - lastRequest > oneMinute) {
    console.log("No more cookies.");
    process.exit(0);
  }
}, 1000 * 60 * 2);

If no one has made a request in the last two minutes, our setInterval check will detect it and exit the app. This can be improved with stuff like middleware, but that’s the general idea.

Deploying on Fly

To deploy this on Fly Machines, we can’t use the normal Fly.io steps (at least not yet, I’m sure it’s coming soon).

Instead we’ll do the following:

flyctl apps create cookiemonster --machines

This step creates an app “container” that holds our Fly Machine. It’s what you’ll see in the Fly.io dashboard.

By default, if you create an app with the above commands, it won’t have an IP address (and since these are web applications, we need them to have a public ip). So, we’ll run the following:

flyctl ips allocate-v4 -a cookiemonster
flyctl ips allocate-v6 -a cookiemonster

Then to deploy the app, we’ll run the following:

fly machine run . --app cookiemonster --port 443:3000/tcp:tls

This command does the heavy lifting. It will:

When you run this command, it’ll give you back an id for the Fly Machine that it creates. You’ll need this to make changes (like pushing updates). If you lose it, it’s in the Fly.io dashboard inside your app under “Machines”.

When you want to update the app image, you can run:

fly machine update e148e453be6e89 --dockerfile Dockerfile

This will build the docker image again, push it, and then update the Fly Machine. This step is also really easy to add to CI pipelines.

If you want to view logs to watch the app boot or shut down, you can run:

flyctl logs --app cookiemonster

It will show the steps of the machine reserving resources, booting, and the stout of the app. When the app exits, it’ll helpfully report: “machine exited with exit code 0, not restarting”.

Cron Tasks

For cron tasks, you can also schedule Fly machines to wake up and run (hourly, daily, or monthly):

flyctl machine update e148e453be6e89 --schedule daily

Cron tasks don’t need the IP steps above. They’ll wake themselves up on the schedule, run their code, and then suspend.

Happy cheap hosting!