Where are you getting these restrictions on prior art from?
There may be a rebuttable presumption of validity if you want to apply the same piece of prior art in the same way as it was used earlier on in the prosecution history, but there is nothing preventing you from using it court. Think about it: it would be fundamentally unfair to not allow someone to use something as evidence in court simply because it was brought up in an earlier ex parte proceeding.
As for the "nobody does it"--do you know the different in cost difference between a third party submission versus an infringement suit in court? We are talking thousands versus millions of dollars here.
There are certainly plenty of reasons why someone would prefer to take an issue to the courts, but sitting on a clear, obvious piece of prior art is not one of them. Of course, patent trolls are the exception to this rule.