[The judges] do not want to give the impression of colluding with fugitives, since that could undermine the public confidence in the legal system.
You've just hit the nail on the head. Bear in mind they were not merely listening to Mr Assange, but were to appear along with him as fellow speakers.
But the judges instead give the impression of not understanding
I was under the impression that they had, in effect, been ambushed. The inclusion of Mr Assange as a fellow speaker was, to quote the Judicial Office, "at short notice and without consultation." So they freely admit they did not know this was going to happen. (If this is what you meant by not understanding what is going on).
Now the judges seem biased in favor of the established powers, blind to the allegations of abuse of powers.
They should seem so only to those ignorant of the reason the judges felt compelled to withdraw. Since you have enunciated that reason so succinctly, you of all people ought not to succumb to that unfounded interpretation.