I may be wrong, but I think the underlying reason is: Font.Size and Font.PixelsPerInch do not respect per-monitor scaling
This has been discussed a lot and font scaling is now re-evaluated. I proposed some changes for the VCL that can even be rolled out as a hotfix (i.e. no interface breaking changes), although not really a perfect solution, which requires TFont.PixelsPerInch moved to published. These changes do work, at least for the test cases we currently have. Unfortunately it cannot be used when compiled with packages.
The easiest workaround for the moment is to use ParentFont = True as much as possible. TForm.Font is currently handled quite well and that can be adopted with this setting. Of course one loses some design variance.