Actually, I see it the other way around: you should know how the system is dealing with your data, what it does, when, how often, basically just how the computer processes. if you don't understand that then you're going to end up with unexpected results and bugs.
Learn the base first, then build on that.
ALL programming languages use variables, variables are memory pointers so learning about pointers is going to help you no matter what language you end up with.
MOST programming languages have functions and other re-usable code, the names of these are also pointers and can be handled as such (you can pass a function as the argument to another function (not just the result).
I've never met an object oriented language that didn't follow the same basic construct: data wrapped inside interfaces. I've come across two main types: c++ style and smalltalk style. Most major (widely used) OOP languages follow the c++ style. All use pointers in some way.
Without the knowledge and control of a language with pointers it's the difference between driving a car and using a keyboard to drive a computer game, you're totally disconnected.