You're giving E.T. Bell's "Men of Mathematics" (almost undoubtedly your source) entirely too much credit. While a fun read, it is not a serious work of historical scholarship; rather it is targeted at a popular audience. Bell's sources on the subject of Cantor are mainly Cantor's own letters; with regard to Kronecker's alleged attacks on Cantor, much of those those letters are directly contradicted by published transcripts of Kronecker's lectures.
Certainly it is true that some of "the smart people" may have not understood Cantor's theory of transfinite numbers; Kronecker, however, surely did, and Poincare did as well. What they objected to was the use of "number" -- a term which at the time had a radically different precise mathematical meaning than it does today -- to describe such constructions.