Deleting Mastodon Cache to Reduce Storage Usage, and a Discussion on MinIO and LocalStack

5 min

mastodon自宅サーバーオブジェクトストレージ
language: ja bn en es hi pt ru zh-cn zh-tw

Hello, this is Munou.
A persistent problem when running a Mastodon instance is storage.
Since it retrieves all information from other servers, its growth is unavoidable.
So, I found a promising script.

Scripting with Bash

Apparently, it's possible to delete them all at once using tootctl, which became available from Mastodon v4.1 onwards.
Improving Mastodon’s disk usage
So, I changed the days to 1 and copied the following exactly:

#!/bin/bash

# Prune remote accounts that never interacted with a local user
RAILS_ENV=production /home/sns/mastodon/bin/tootctl accounts prune;

# Remove remote statuses that local users never interacted with older than 1 days
RAILS_ENV=production /home/sns/mastodon/bin/tootctl statuses remove --days 1;

# Remove media attachments older than 1 days
RAILS_ENV=production /home/sns/mastodon/bin/tootctl media remove --days 1;

# Remove all headers (including people I follow)
RAILS_ENV=production /home/sns/mastodon/bin/tootctl media remove --remove-headers --include-follows --days 0;

# Remove link previews older than 1 days
RAILS_ENV=production /home/sns/mastodon/bin/tootctl preview_cards remove --days 1;

# Remove files not linked to any post
RAILS_ENV=production /home/sns/mastodon/bin/tootctl media remove-orphans;

It takes too much time, so I can't yet determine how much storage will be freed up ^^;

Ideal Home Object Storage

Recently, I've been thinking about setting up a separate storage server from the one that's currently in production.
The reasons are as follows:

  • When large files were placed on a separate disk, the Linux kernel would stack with Disk I/O errors upon disk failure.
  • Breaking away from filesystem dependency.
  • It's clearer to separate it from the executable binaries.

There are other reasons, but in short, it's as described above.
In particular, I learned that the I/O error mentioned in the first point can cause the system to stack even during reboot, sometimes requiring the disk to be physically removed (though it can be avoided by changing kernel parameters and rebuilding), so it seems necessary to implement this.

Regarding breaking away from filesystem dependency, while I'd like to use robust ZFS, the ZFS available in a Linux environment is a pseudo-copy and not a complete ZFS.
If you want to use ZFS, you need to set up and use a FreeBSD environment.
RAID is even more complex, and I won't use it because there's a possibility of not being able to recover if the RAID card dies.
Therefore, I want to use FreeBSD as an object storage server.

Regarding the clarity of separating it from executable binaries, it's simply that I want to build a separate environment as a general-purpose storage area, like the old Plan9, with distributed systems in mind.
If you're just running programs, you don't need hundreds of GBs or several TBs.

MinIO and LocalStack

LocalStack seems to be able to emulate AWS S3, but the free version is not persistent and has limitations.
As a more OSS object storage, MinIO seems good if you're actually deploying it on a server.
It seems they should be used differently: LocalStack for temporary use of an AWS S3-like environment, such as in development, and MinIO for when you want to set up a server as object storage.
Therefore, I will set up a MinIO server.

That's all for now. Best regards.

Related Posts