Performing WebP Conversion with LumeCMS
Hello, I'm Incompetent.
Since I've been using WordPress until now, WebP conversion was easily done by installing a plugin and internally executing ImageMagick, but such a feature does not exist in LumeCMS.
It might be possible with a Lume plugin, but ultimately, for efficiency in image processing, it's faster and more efficient to execute something written in C.
It's not impossible with an Nginx-side script, but since I'm also running a reverse proxy, there were other things I needed to implement, and I didn't feel the need to force it within the very limited resources of the reverse proxy server (1vCPU Mem 1GB Disk 25GB).
Also, it seems some people are making it possible by internally executing a Lua script. Achieving automatic WebP conversion of server images with Nginx+Lua+libwebp
The drawback is that this script is executed at the time of a request, so I feel it has the potential to concentrate resources.
There seems to be a wrapper for npm called cwebp-bin, but what's actually needed is image conversion and rewriting the image paths in the HTML files of the built static site.
Therefore, as a backend process, I decided to execute a shell script that converts to WebP within my own custom init daemon script, which monitors Lume changes and automatically builds.
Also, I'm not using libvips. Libvips's WebP conversion merely uses libwebp internally, and it's the same as cwebp which can be easily installed via apt, so I will perform the conversion with cwebp.
Shell Script for Conversion
As shown below, image files are converted to WebP, excluding duplicate patterns.
#!/bin/bash
# Directory to monitor
SOURCE_DIR="/var/www/html/soulmining/src/uploads"
# WebP output directory
DEST_DIR="/var/www/html/soulmining/src/uploads"
# Extensions to convert
EXTENSIONS=("png" "jpg" "jpeg")
# WebP quality setting (0-100)
QUALITY=80
# Check for required command
command -v cwebp >/dev/null 2>&1 || { echo >&2 "cwebp command not found. Please install it."; exit 1; }
# Create output directory if it doesn't exist
mkdir -p "$DEST_DIR"
# Function to process files
process_file() {
local file="$1"
local filename=$(basename "$file")
local name="${filename%.*}"
local dest_file="$DEST_DIR/${name}.webp"
# Skip already converted files
if [ -f "$dest_file" ]; then
echo "Skipping: $filename (already converted)"
return
fi
# Convert to WebP
cwebp -q $QUALITY "$file" -o "$dest_file"
if [ $? -eq 0 ]; then
echo "Conversion successful: $filename -> ${name}.webp"
else
echo "Conversion failed: $filename"
fi
}
# Main processing
for ext in "${EXTENSIONS[@]}"; do
find "$SOURCE_DIR" -type f -name "*.$ext" | while read file; do
process_file "$file"
done
done
echo "Processing complete"
Add this to the init daemon's c/initme-watcher`](httpithub.cturame-watcher).
#!/bin/bash
### BEGIN INIT INFO
# Provides: lume-watcher
# Required-Start: $remote_fs $syslog
# Required-Stop: $remote_fs $syslog
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Watches directory and triggers Lume task
# Description: Watches the specified directory and triggers the Deno Lume task when changes are detected.
### END INIT INFO
WATCHED_DIR="/var/www/html/soulmining/src/"
COMMAND="/home/haturatu/.deno/bin/deno task lume --dest=site"
CONV_WEBP="/opt/sh/webp.sh"
~~~
Omitted
~~~
# Start monitoring
monitor_directory() {
inotifywait -m -r -e modify,create,delete "$WATCHED_DIR" | while read -r directory events filename; do
echo "$(date): Change detected" >> "$LOG_FILE"
last_run_time=$(get_last_run_time)
now=$(current_time)
if [ $((now - last_run_time)) -ge $COOLDOWN_TIME ]; then
check_and_rotate_log
echo "$(date): Executing command" >> "$LOG_FILE"
sleep 0
cd $OUTPUT_ROOT_DIR || exit
$CONV_WEBP >> "$LOG_FILE" 2>&1
$COMMAND >> "$LOG_FILE" 2>&1
cd ~ || exit
set_last_run_time
else
echo "$(date): Command not executed due to cooldown" >> "$LOG_FILE"
fi
done
}
~~~
End omitted
Strictly speaking, in the case of this init daemon, it detects all changes within the target directory, so a rebuild will run just by uploading an image, but I guess that's unavoidable.
If you were to do it, you should specify the inotifywait execution option as --exclude "dth".
Add to _config.ts
Well, there's still work to be done.
As it is, the paths in the HTML files built by Lume specify the paths of the original images.
So, let's make it rewrite them during the build.
import { walk } from "httpeno.lad/d.ts";
~~~
Omitted
~~~
site.addEventListener("afterBuild", async () => {
const buildDir = site.dest() // Build output directory
for await (const entry of walk(buildDir, { exts: [".html"] })) {
if (entry.isFile) {
let content = await Deno.readTextFile(entry.path);
// Regular expression to convert URLs to .webp
const regex rcload"'\s]+\.(png|jpe?g|gif;
content = content.replace(regex, (match) => {
return match.replac(png|jpe?g|gif '.webp');
});
await Deno.writeTextFile(entry.path, content);
console.log(`Processed: ${entry.path}`);
}
}
});
Honestly, it's a secret that I thought it would be shorter, faster, and easier to rewrite this with sed within lume-watcher.
By the way, Lume's -w option is unstable
I have a daemon running, but Lume officially supports watching, so if you start it with the -w option as deno task lume -w, you can generate HTML every time a file changes without using a simple server, but it's unstable.
It often crashed in my environment.
In LumeCMS itself, within `apteme.ts` located in the CMS repository,
Start the watcher
const watcher = site.getWatcher();
deno-lint-ignore no-explicit-any
watcher.addEventListener("change", async (event: any) => {
const files = event.files!;
await site.update(files);
dispatch("previewUpdated");
});
watcher.start();
it seems to be called.
await site.update(files);
That seems to be the place.
Since it's called within the LumeCMS code, I feel like Lume itself should be the same, but I wonder why it's not.
That's all for now. See you again.