Yup, nothing is ever fully trusted. Unfortunately, trust is a much-used word in security, and govt tends to shove it into terms of art, such as "trusted channel" or "trusted path," which show up in common criteria.
One of the major issues I have with CC is that there is a lot of hand waving in the threats and assumptions section, including the assumption that the device is being administered or used by people who aren't actively hostile or incompetent. That's an awful lot of hand waving, and then leads to a situation where in technical terms we can talk about trust, but in any meaningful way we really can't. I can provide assurances around the abilities and configuration of two devices. They can talk over a "trusted channel," (i.e., a cryptographicaly secure channel with authentication by certificate or PSK, between processes on two devices) and be administered by trusted path (a similarly cryptographically secure channel between a remote administrator and the device).
A bad actor leveraging SSH to remotely connect to a device and do bad actor things is leveraging a "trusted path" just as much as the good actor.
So, yes, while "trust" may be a technical term/term of art, absolutely take with a grain of salt the trustworthiness of the secrecy of the communication based on factors such as strength of cryptography providing the confidentiality, integrity and authenticity of the channel, the likelihood that the endpoint (including your own) is compromised, and the likelihood that the remote terminal operator is compromised in some what (being blackmailed, actually a mole, has been replaced with someone else, etc.).
End of the day, though, "good enough" is probably usually good enough, residual risk will never be removed, and at some point you just need to live your life without being overly paranoid all the time.