I don't really get your comment. Of course you need to trust the signer's key which is what I mean by a "known valid version of the signer's key", and the basis for that is not another user's trust unless you really trust that user as well.
However you can look at multiple sources for the key before giving it some trust (mail archives, wayback machine, which key signed previous versions, etc). Best is to actually verify the signature in person, but that's not always possible.
Note that in any case it can't be worse that relying *only* on the checksum - without a valid signature, the checksum serves only for integrity verification, and if all you need is to check transfer integrity you really don't need anything better than MD5 as although insecure the chances of collisions is way too small for one to occur accidentally.