Fear, sadly; IS a major part of why people don't speak up I believe.
Australia does -not- legally protect freedom of speech or expression, many of 'our' laws prohibit us in 'thought crime' style, this however, is not what I think scares people.
The problem is that, if for example, I stood up in public and stated my belief that looking at an image of a criminal act, is -not- a criminal act, most would agree with me. However, when that criminal act is sexual and involves children, wether real or imaginary, the -vast- majority of those same people would label me as a pedophile. This label has many concequences, not just for myself, but also likely for my family, and would effectively end my social life, if not also my career. --THAT-- is what people fear.