Likewise, if there's good reason to believe that you have a different set of books in a safe, or perhaps a murder weapon - you can be compelled to give the combination. By providing the combination you are not confessing to fraud or murder - you already left that evidence; by not providing the combination you are standing in the way of the court to evaluate the evidence.
The fuzzy line is regarding whether there's reason to search in the first place (which is more of a question regarding the Fourth). Should the simple fact that a computer may have aided the crime be probable cause? Does there have to be evidence showing which computer was utilized (like only allowing the search of one computer behind a router's firewall instead of fishing in all of the computers - and what if there's multiple owners?...).
I think a really interesting test case would be if a criminal used a confession or otherwise key incriminating evidence as the pass phrase for an encrypted device. If they plead the Fifth, and then were compelled anyway - how much would be ruled inadmissible?