Neither driver or passanger can see each other's ratings. After a probationary number of trips, the driver gets a rolling lifetime average rating in the app, and a weekly average rating via email. Drivers and riders start out with a 5 star rating, so if you encounter a driver with a five star rating they are still on probation. I don't believe riders get a probation (Uber has never said as much) but if a passenger has a 5 star rating I find that usually means they are new to Uber.
All drivers are obligated to rate their passenger upon exit before they can log back on and continue driving for Uber. The passenger is generally unaware that they are being rated, but can see their lifetime average by logging into the site/app. I'm unaware as to any automatic cut off that would cause a passenger to be banned, though other drivers claim that the rider ratings are occasionally investigated. My lowest rated riders were in the 2.x range.
If the driver's lifetime rating falls below 4.7 that's grounds for being barred (fired) from Uber, though you can start over by taking a paid class from the company. For some drivers their rating is a source of stress, as they are driving cars leased through Uber at a rate of $800-$1600 a month, with a punitive $250-$1000 charge if they are fired.
AFAIK the rider rating isn't fully implemented yet, the drivers can't see the rider rating unless they go out of their way to check it in the app, which they can't legally do while the car is in motion to pick up the rider. Uber assures drivers that they are being kept safe from problem riders via the rider rating system.