I use the Windows app on the Mac to connect to my PC over Remote Desktop - are you saying that the Windows version of this can't handle that?
Correct. They removed the ability to connect to anything but an Azure VM instance (of various kinds). There was talk of them 'working' on 'adding this functionality' as if it was never there in the first place and removed by a dickwad product manager to force people into the Azure ecosystem, but even if they did put it back in you'd still need either a corporate or educational - not personal - MS account to launch it.
The 'Remote Desktop' app in the app store, which is the product 'Windows App for Windows' was directly forked from, at first merely rebranded and then feature locked, does allow you to connect to local machines and does not require a corp or edu ms account to use. But that's dead now, so your only option from a Windows desktop is mstsc, which doesn't handle live desktop resizing etc etc.
Kind of weird how they treat Windows Desktop users as second-class citizens for their shiny new RDP app, but let IOS, Macos, and Android enjoy the nicer experience. Fucking weird. Which is why I'm thinking they're going to gimp those too at some stage soon.