* I don't look 'typical' for what the passport says
* I don't or usually don't live in the country that has issued the passport
* The passport for the country is issued in another country than what the passport is for
* I am not American but the tickets I use are often paid with an American credit card. When the (return) flights from Europe to USA were paid with an American credit card, that is an instant check - even when paid by spouse.
* If I was traveling to more interesting continents than just Europe and North America, I bet I would meet the rubber glove