If I understand correctly, higher gravity makes time pass more slowly
Correct. Or to be more general, time passes more slowly deep down in a gravitational well
so a clock in lower gravity will register more time from its perspective than the clock in higher gravity.
Yes. Every time a clock on earth ticks 1 time a (stationary) clock near a black hole will tick 1-R/r times. For example, a clock at a distance r=10/9*R will only tick 0.1 times every time a clock on earch ticks 1 time. The same applies to all other physical processes, not just clocks. So a person on Earch could wave their arms 10 times every time a person that close to the blck hole could wave them once. Or a person on earch could think 10 thoughts in the time a person close tot he black hole could think 1.
Therefore, from the reference frame of an object falling into a black hole it would seem that it takes forever.
No. If time goes slowly for somebody they don't percieve themselves to be going slowly, they see everything else going very quickly. From the reference frame of an infalling object, their clocks are ticking at normal speed (by definition - a local clock measures how fast physical processes happen locally). But the far-away world seems to be sped up.
But from our frame of reference, time passes normally,
Time passes normally to everybody in their own frame of reference.
so we would observe the object falling into the event horizon just as we would an object falling into a star, minus the red shift and vanishing?
No. In our frame of reference, all physical processes near the black hole, be that clocks, falling people or light, are moving in slow motion. In the absene of red shifts and other optical phenomena, we would observe the object to inch closer and closer to the horizon ever more slowly until it's hardly moving. A photon next to them would similarly move extremely slowly (the speed of light as measured locally is constant, but not in other parts of spacetime).
I guess what you thought was something along the lines of "people in areas where time moves quickly will see everything move quickly, and people in areas where time moves slowly will see everything move slowly". To see why that doesn't make any sense, imagine if I had a slow-ray and shot it at you. It would slow you down, but not me. But because I'm not slowed down doesn't mean that I will see you moving at normal speed. Your time passes more slowly than mine, so I will see you move more slowly than me. And since your mind is also affected by the speed of time, you won't perceive yourself to be moving any slower than normal. But to you the rest of the world would look like a movie in fast-forward mode.
So I have a hard time understanding how it would take an infinite time from the perspective of an outside observer
I hope this helps.