TFA actually states that social services made a decision based on evidence available to them. We do not know if that information is also available to us. While I, of course, believe in being innocent until proven guilty, there is an additional factor in that if this bloke has been convicted of a sex crime in the past, social services may have access to that information, but they are not allowed to pass it on.
Also, if he IS a paedophile, and suspected the police were closing in on him, then he can now turn around and say "yes, of course that kiddie porn is there; I told you it was". In short, I'm reserving judgement on either party for this one. If he's not guilty, then he deserves to be able to have that decided in a court, rather than have people making up their minds based on a news article. On the other hand, there may well be a very good reason that access has been removed. If there is, then let the courts rule that the decision made by social services is fair.
It's right that he is allowed to challenge, but it's also right that social services are allowed to make a decision based on what they believe at the time to be in the best interests of the child.