There are legit reasons to "spoof" the caller number. For example, companies that call out from multiple sources but have one main-office line.
In my case, I have a VOIP line with no incoming #. I previously used my cellphone # for call-display so that people know that it's me calling.
In that case, there should be a single (authoritative) registered phone number. This is something the Telcos need to work out, my sense is it needs to be implemented at the switch level.