Health notifications for all

08 Apr 2020

Disk space is a finicky thing in many applications. It’s easy for an application to use an unexpected amount of data, either on disk or in the database, especially if it’s using disk space for caching. And if the disk fills up, well, few applications handle that gracefully.

That’s why Platform.sh has offered low-disk health notifications as an option on all projects for years. Users can set up email, Slack, or PagerDuty notifications, which will notify users when any service’s available disk space drops below 20% and then again when it drops below 10%. We recommend it for all customers.

This week, we’re taking it a step further. All new projects now have email notifications configured by default for all admin users on a project. No extra steps necessary; create a project and you’ll get health notifications. You’re still free to add more notification channels if you wish.

Moreover, we’ve automatically enabled email notifications to admin users for all projects that didn’t have any notifications set up already. As of today, every project on Platform.sh has health notifications turned on.

What should I do if I get a low-disk warning?

If you’ve recently gotten an email from us saying you’re low on disk space, it means your application, database, or some other service is below a warning threshold and you should probably look into it.

The first thing to look for is if you’re able to clean up any unused files. If your application writes a lot of files to disk, especially generated files, it’s quite possible that there are many just taking up space that you don’t need. If so, you can safely delete those. Old logs in either the database or on disk, if you’re doing your own logging, are another common source of surprise disk usage.

The next step to check is your caching. Many applications default to using an SQL database for cache data, which is inefficient and frequently a big waste of disk space. It’s not unusual for cache-heavy applications to be using half their entire database just for cache data. We strongly recommend using a Redis service for your application’s cache data, or if your application doesn’t support Redis we also offer Memcached. Both of those are memory-resident cache services, which means they’re both faster than the database and won’t use up disk space.

For a disk-based cache, another option is to use the /tmp directory on your application. The /tmp directory is a ramdisk, which makes it faster and avoids using up disk space. Be aware, however, that because it’s volatile storage it may get wiped during a new deploy. Don’t store files there long-term, but it works well for regeneratable cache data.

If you simply need more disk space, that’s fine too. Go to your project billing page and increase the amount of disk space on your plan; disk space can be reserved in increments of 5 GB and is per-environment.

Once you’ve saved the plan with the higher disk budget, edit your .platform.app.yaml or services.yaml files as appropriate to increase the disk value. For example, to add 2 GB of storage to an existing MariaDB server, locate this section in services.yaml:

db:
    type: "mariadb:10.4"
    disk: 2048

And change the disk value to 4096. (The number is in megabytes.) Commit your changes and push to activate more space.

But I don’t want to get emails!

Totally understandable. (We get enough email as is, too.) If you’d rather get Slack or PagerDuty notifications, you can enable those using the Platform.sh CLI. See the notifications documentation for details. You can also add or remove email notifications if you’d rather send email notifications to someone other than “all admins.”

However, we strongly, strongly recommend having at least one health notification enabled on every project at all times. Not doing so is how out-of-disk errors sneak up on you and take a site down at the least convenient moment (because errors never happen when it’s convenient), and no one wants that.