I do agree that AIX does stand for "Alien Interpretation of UNIX", but even though it is squirrely, if an application runs on it, it runs well.
I am not disagreeing with the fact that AIX and Solaris are bit players. However, I would say that one problem is that both Oracle and IBM at best are focused on retaining existing customers. Neither have any marketing focus on getting people from VMWare and OpenStack onto their platforms. And without expanding the market, just as the parent stated if the market isn't growing, it is shrinking.
This is a hard thing to do. The trend has been for businesses to have projects to get off of SPARC and POWER onto commodity x86 hardware, because x86 hardware has a price advantage, and can be sourced from a number of vendors. Both IBM and Oracle will have to have a good reason (good as in financially appealing), but it could be done.
There is the security aspect. Solaris and AIX have long since went through their teething problems when it comes to security and are quite robust in this regard. Solaris has tossed root (as a user) in Solaris 11, and uses roles (this functionality can be reversed if needed), and AIX can run completely root-less, as well as use signed executables/libraries/scripts. If Oracle could put some R&D into security... and a reasonable way to manage/audit things, they might just gain some ground back.
However, it would have to be a -major- improvement in security features, beyond the delta from Solaris 1.x to 2.x, something as major as the jump from Windows 3.1 to NT. Plus, it isn't just features, it is ease of implementation. Something where Solaris can be marketed as, "if it runs on this OS, it is secure".
What might have to happen is that Oracle might have to license things from Microsoft. Exchange and Active Directory come to mind. This way, even if there is a major Windows exploit, core AD servers would still be protected because they would be running on Solaris. It is doubtful MS would license this, and it would take some coding by Oracle... but it is going to take a Herculean effort to get SPARC's marketshare to grow again anyway, so might as well try to get businesses to move to the platform by offering an alternative to a Windows backend.