Thanks for the suggestions.
At first I was thinking that getting a new kernel probably wouldn't help, but right before you replied, I looked more into the "low latency kernel" thing and tried installing the Liquorix low latency kernel (
https://liquirix.net) - and voilą - the xruns are gone!
I did test the CPU performance setting briefly yesterday but didn't see any improvement. But I'm not completely sure if I did it properly.
Anyway, I'm very happy now with the low lacency kernel, currently running jack at 128 frames/period which gives a nice 5 ms latency. (Actually, qjackctl does report the odd xrun now and then at this latency, but it doesn't produce any noise; and if it does I can always raise it to 256 at least temporarily.)