I think you're confused. Plumes means "in space". The whole benefit of plumes is that you don't need to go under the ice at all, you can do flybies to collect ice particles, or have a lander observe and sample the plumes at the surface. The key is that it means a recent connection between the depths and the surface, and that would be huge for simplifying exploration.
We're nowhere near to being able to launching an ice boring / swimming probe. If I recall correctly the last thing I read on the subject, however, the most promising means for communicating with such a probe on an affordable mass budget was.... not communicating with it. Aka, having it fully autonomous - melting its way down, sampling/observing the ocean, then re-melting its way back to the surface - then and only then transmitting. The waiting period with no data would be stressful (as if it failed you'd never know why), but it could potentially be used on almost any icy solid body regardless of the ice thickness.
It's also possible that there's liquid water much closer to the surface than the global ocean. There are some inferred lakes at a depth of only a few kilometers, which is potentially short enough for a probe to maintain a fiber connection with the surface. And after JUICE and Clipper, we may well have found locations that are even shallower.