The big problem I see with this is that an incompetant student on a competent team could be carried though while a competent student on an incompetent team could be failed.
What if any mechanisms did they have in place to try and prevent this?
Peer review at the end of each semester was the main mechanism. While the reviews weren't the only component to the grade they were factored heavily by the professor. The professor and TAs keep track of team progress through weekly (or if a team's struggling, more frequent) meetings and update presentations, plus they have enough experience overseeing these projects to get a good idea of who's actually doing what.
The other thing it helped with was teaching the students about how to deal with team dynamics in which the inevitable slacker tried to skate through. How many times have people worked projects in industry when you had a moron on your team and not known what to do with them? At this point in the curriculum the vast majority of the technically incompetant students should have been weeded out, so most of the problems occured due to clashing personalities and other 'team dynamic' problems. The year after I graduated one of the groups actually ousted their incompetant team leader through a (obviously bloodless) coup and they were able to complete their project successfully. The former leader was a decent engineer, he just sucked at the leadership/management part of his former role.
As difficult as the task seems, a team has to actively try to fail as a whole. The professor had been known to fail teams before, but only very rarely. If a team is struggling with a technical aspect of the design there's a lot of help available but the help is only for advice, they won't do the work for you. Peer pressure is usually a pretty good motivator to get adequate work out of everyone on a team even though there will be varying levels of individual effort. However if there's a rare outlier that just won't work for whatever reason they'll be pulled by the professor.