32-bit x86 distributions are built for i386, i486 or i586. All of which do not have all the instructions available in your processor. On a 32-bit distribution, essentially everything is built for ancient processors in the name of compatibility, and it doesn't use your processor effectively.
That's simply not true. A lot of 32-bit distro are build for minimal i586 or even i686. E.g. Ubuntu is build for i686 starting with release 10.10.