The reason the penny costs so much to keep in circulation is not solely the cost of minting. If a penny costs $0.02 to mint but is used in 10,000 transaction in it's life time that would be ok.
The problem with the penny is that they don't get spent. The mint needs to keep producing new ones for retailers to give out and people go home and throw them in a coffee can.
Oddly, this is the exact argument in favour of $1, $2, and $5 coins. People don't spend coins as easily, they tend fall between couch cushions or collect in jars. Until those jars are emptied, and the couch cleaned those coins are basically a kind of interest free loan the government.