Jellyfin has been fairly clunky for me too.
But I don't think you're going to gain much just by ditching the media player in jellyfin. I feel like the overhead is scanning, scraping, and displaying the media (ie. the browser). Then playing the media is just loading a player that you would need anyway. You might be able to hook up an external player but I doubt that would help much.
Personally I use Kodi instead but I think you'll run into the same issues there.
The problem probably isn't the software but more the 15+ year old hardware. It's more suited for like a PiHole than a media center.