This is just baseless FUD.
If it was an unknown company then yeah you might have a point. But Canonical is not unknown, nor is it going anywhere. In the AMA Shuttleworth promised to deliver a top of the line hardware, even if that would mean he would have to take care of extra costs himself.
This phone is not going to be mass produced, which does mean that costs may be a bit higher but also there is more wiggle room for novel and expensive components.
The phone is still cheaper than an iphone5,