ssh-agent is inevitable.
Suppose you frequently use a program that communicates over ssh connections, e.g. cvs, rsync, svn, or darcs. You wouldn't want to type your key's passpharse each time you run the program. Of course you never want to leave your key's passphrase null. Therefore, you need ssh-agent.