This is similar to what we've done with WiKID (sourceforge.net). A hash of the server's cert is stored on the auth server and is sent down to the software token with the OTP. The token fetches the cert via the user's internet connection, hashes it and compare the two hashes. If it matches, the otp is presented and copied to the clipboard. and the default browser is launched to the website.

The key difference is that your server becomes the validation source and not a 3rd party.