With most other currencies, there is always the chance that once a payment is done, it can be taken away. For example, you sell something on $AUCTION_SITE, get paid via credit card, then find the CC company slurped the money back out, as the credit card owner has disputed the charge... now it is your job to sue the individual and get your cash back, although you were technically "paid".
Or, you get a check, cash it, and get a NSF charge because of it bouncing.
Even cash has this problem. You take a $20, only to find it is a $1 note, except someone cut and pasted a part from a higher denomination bill on that, or the bill could be an entire counterfeit.
The good and bad about BitCoin is that once the transaction is made and runs through the system, it is permanent. No backtracking. That currency has transferred hands irrevocably, and outside of breaking cryptographic protocols, there is no reversing it, so a wallet with coins can have them pulled out without the wallet's private key.
If one knows that, and always takes that into account, it makes one extremely leery of exchanges and "BitCoin bank accounts", because eventually, someone is going to make off with the goodies, and there is nothing anyone can do about it. Caveat emptor.
This doesn't mean BitCoin is perfect either. If you want to be sure you are not going to on the losing end of a double-spending, you have to run the entire blockchain before accepting a transaction, and this takes a lot of time and a good net connection. Any shortcuts only can cause grief in the long run. BitCoin was designed so you didn't have to trust anyone, so might as well take advantage of it.