I've successfully written and ran a Qt/QML application (albeit with a simple interface) on a custom baseboard for the CM3 running Raspbian Lite without X. Not through framebuffer but using the VC
.sos provided for the Pi. If you used framebuffer on the Pi you likely used Qt without hardware acceleration. Qt has a separate QPA backend using VC libraries directly.
There are alternatives, but they are usually development kits and SoMs with their carrier boards rather then SBCs. Those are targeted at professionals who design custom devices with such processors.
You can for example look at Toradex i.MX 6 Quad SoM which is a Qt reference platform. They have Yocto support too. The SoM with it's carrier board will set you back about 200 euro though.
The official i.MX 6Quad Sabre board is 450$. And that's for SoCs which tend to be on the cheaper end of the range for their performance point.
Apart from the i.MX there are other series of SoCs which you can look at with their own SoMs and devkits, such as:
- STM32MP1, though those are new SoCs on the market and historically their MCU (ST didn't have MPUs on the market for quite some time) code quality was not satisfactory for me
- The Beaglebone series, they have weak CPUs but some quick Google suggests people have successfully ran Qt with QML on them (although the most powerful, BeagleBone AI, has no Yocto support as far as my searches indicated). Or other Texas Instruments processors in general.
The cheapest option would be the BeagleBone Black which is also the only hobbyist/maker board which has a chance of fulfilling your requirements that I know of.
Apart from the Pi's BCM2 every other SoC mentioned above has full documentation available (I often compare BCM283x's measly 150 pages to total 6k pages of datasheet+manual of the Sitara SoC found in the Beaglebone).
Lastly, there is a lot of maker boards with Allwinner or Rokchip SoCs but those are not fully open AFAIK.
Source: general knowledge gathered from working professionally with embedded systems (both Linux and MCUs).