This is a truly archaic way of thinking about language as restricting what we can think about. We can certainly conceptualize things that we don't have words or language to express. That entire sentence seems paradoxical to me; how did we come up with a language to express anything to start with if we couldn't conceive of the ideas the language expresses without a language to express them?
To me, the key to education in mathematics is teaching problem solving, but the curriculum and teaching methods have moved to just a simplistic model of teaching rules to learn to regurgitate. These rules are so abstract that it's hard to conceive their use in real life application, and so they're learned for students' tests in class, if even that, and then shortly forgotten because they're so inapplicable.
I completely understand where Ramanathan is coming from, but I think it comes off in this summary as needing to completely abolish math at all. Truly, the best idea seems to be not to strictly teach these rules in a 'reference sheet' sort of manner, but to teach how to come to the conclusions that lead to these rules. This will lead to a better actual understanding of what math is, I believe.