Download the "Mandriva Free DVD" not the "Mandriva ONE KDE" or "Mandriva ONE GNOME"
The ONE editions are LIVE CDs that you can install from if you really need to.
The free DVD edition contains MUCH more software, but more importantly, it contains the REAL Mandriva installer program, that lets you chose anything you want or need.
It will handle what you wanted, either by selection XFCE and the Gnome apps you want, or just select both and after your first boot, just select XFCE at the log-in screen and it will default to it the next time.
I usually install all of the desktops, so that I can try out the others.
After you go online and go to add software, it will set up the repositories you need to download almost anything you could want, including a LOT of different desktop environments, like Enlightenment, IceWM, and almost any other you might want.
The really nice thing about Mandriva is that when you add a program, it shows up in the menus of almost all of the different desktops, so you do not have to set each one up separately.