Holy Trinity isn't mentioned in the Bible either... The "it's not in the Bible" argument doesn't apply to the Catholic Church since it doesn't subscribe to the doctrine of Sola Scriptura.
Besides, if the Church simply wanted more members, why are they against non Catholics using contraception too? If they want more members, why do they not permit infertile couples from using artificial fertilization procedures?
The Church teaches that sex and reproduction go hand in hand. That isn't to say that all sex that doesn't result in fertilization is sinful, but there must not be an attempt to interfere with the process of fertilization (whether using condoms, chemicals, outercourse, etc).