"Something you know" isn't what counts when it comes to protecting you from self incrimination; it is whether the "something you know" is incriminating you.
This leads to an interesting idea. Claim that you passphrase is a confession. If you plan ahead, you can even make that claim true. Encrypt your plan to assassinate the president with "I plan to assassinate the president OV:}A7MC".