Avalonia doesn't support self-querying display scaling on Wayland. You need to manually tell it via your environment variables. Presumably the old flatpak maintainer automated this process in the sandbox somewhere.
The simplest fire and forget fix is to go to /etc/environment and add the following line somewhere (where X is your scale such as 1.25, 1.5, 1.75, 2 etc.) then restart your DE:

AVALONIA_GLOBAL_SCALE_FACTOR=X

If you have two displays with different fractional scales you could instead use the AVALONIA_SCREEN_SCALE_FACTORS variable as described here: https://github.com/AvaloniaUI/Avalonia/wiki/Configuring-X11-per-monitor-DPI

I have the same issue with Appimage on Wayland

Ok, when I launch Ryujinx from the terminal with AVALONIA_GLOBAL_SCALE_FACTOR=2 it works. In the .desktop file, I wrote "Exec=env AVALONIA_GLOBAL_SCALE_FACTOR=2 ./Applications/Ryujinx.AppImage" so the scaling issue is also fixed when launching Ryujinx from the App Grid. Thanks, @bangfire !

I'm still on X11 but the environment variable worked as well. Maybe add it to the wiki on the config section for future users with the same issue?

Same issue with the ryubing package on NixOS under KDE Plasma 6 with Wayland. Running AVALONIA_GLOBAL_SCALE_FACTOR=2 ryujinx helped.

Avalonia doesn't support self-querying display scaling on Wayland. You need to manually tell it via your environment variables. Presumably the old flatpak maintainer automated this process in the sandbox somewhere. The simplest fire and forget fix is to go to /etc/environment and add the following line somewhere (where X is your scale such as 1.25, 1.5, 1.75, 2 etc.) then restart your DE:

AVALONIA_GLOBAL_SCALE_FACTOR=X

If you have two displays with different fractional scales you could instead use the AVALONIA_SCREEN_SCALE_FACTORS variable as described here: https://github.com/AvaloniaUI/Avalonia/wiki/Configuring-X11-per-monitor-DPI

The NexusMods app (https://github.com/Nexus-Mods/NexusMods.App) scales the UI fine on my 4K TV.
If I set the env var AVALONIA_GLOBAL_SCALE_FACTOR=2, Ryujinx scales fine, but the NexusMod app will be scaled *2.
AVALONIA_GLOBAL_SCALE_FACTOR=1 nexusmods-app => correct scale, AVALONIA_GLOBAL_SCALE_FACTOR=1 ryujinx too small.
This seems to have nothing to do with Wayland or X11, Ryujinx scales at "0.5" for me on i3 by default on my 4K TV.

The NexusMods app (https://github.com/Nexus-Mods/NexusMods.App) scales the UI fine on my 4K TV. If I set the env var AVALONIA_GLOBAL_SCALE_FACTOR=2, Ryujinx scales fine, but the NexusMod app will be scaled *2. AVALONIA_GLOBAL_SCALE_FACTOR=1 nexusmods-app => correct scale, AVALONIA_GLOBAL_SCALE_FACTOR=1 ryujinx too small. This seems to have nothing to do with Wayland or X11, Ryujinx scales at "0.5" for me on i3 by default on my 4K TV.

Seeing as the NexusMods app also closed a similar issue as not planned due the exact problem with the framework I described (Nexus-Mods/NexusMods.App#1277), I think there is something happening on your end via the DE/some other factor that you may be missing. They have no special mechanism that Ryujinx doesn't.

The NexusMods app (https://github.com/Nexus-Mods/NexusMods.App) scales the UI fine on my 4K TV. If I set the env var AVALONIA_GLOBAL_SCALE_FACTOR=2, Ryujinx scales fine, but the NexusMod app will be scaled *2. AVALONIA_GLOBAL_SCALE_FACTOR=1 nexusmods-app => correct scale, AVALONIA_GLOBAL_SCALE_FACTOR=1 ryujinx too small. This seems to have nothing to do with Wayland or X11, Ryujinx scales at "0.5" for me on i3 by default on my 4K TV.

Seeing as the NexusMods app also closed a similar issue as not planned due the exact problem with the framework I described (Nexus-Mods/NexusMods.App#1277), I think there is something happening on your end via the DE/some other factor that you may be missing. They have no special mechanism that Ryujinx doesn't.

Indeed I saw this issue when digging it, what's interesting is both ryujinx & nexusmodsapp are installed the same way on my system:
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=ryujinx-git
https://aur.archlinux.org/cgit/aur.git/tree/PKGBUILD?h=nexusmods-app-git
And I don't see what could influence the scale on any of those. Even if I run env -i nexusmods-app it's scaled properly 🤔