... I'm a pilot, and I wouldn't fly as a pax with most of the other pilots I know, especially not under circumstances they are unfamiliar with - like loading down the plane with people and luggage close to gross weight and doing a cross country with it.
Also, this is in some pretty serious gray area. A pilot may not "hold out" for passengers to share fuel on a trip he/she is planning to take. Any kind of "if someone else is going, I'm not going" makes it a Part 121 charter. If pilots start deciding not to go if they don't get a full plane, or if they wait around for an hour for the person who is late, I think they might find themselves violated.