All it would take is for the JS that powers the site to be modified to send your passphrase(s) back to them. If at some point they wished to go back on their word not to ever look at your passwords. At that point you'd be safe until the next time you needed to log in to retrieve a password.

Guilty as charged... ;)