Yep, you're pretty close. The problem he had was with the clutch. Ferrari told him, basically "you're holding it wrong".
"It's very simple. In the past, I have bought some of the most famous Gran Turismo cars and in each of these magnificent machines I have found some faults. Too hot. Or uncomfortable. Or not sufficiently fast. Or not perfectly finished. Now I want to make a GT car without faults. Not a technical bomb. Very normal. Very conventional. But a perfect car," Ferruccio Lamborghini once said.
With regard to the Ferrari cars, the after-sales service also disappointed Ferruccio, so he expressed his frustration with Enzo Ferrari. However, following a lively exchange in which Ferruccio voiced his displeasure with the clutch and advised Ferrari to find a remedy, Enzo responded: "The clutch is not the problem. The problem is you don't know how to drive a Ferrari and you break the clutch."
According to the Motor Web Museum, Ferruccio replied: "Dear engineer, I'll never buy your cars again. From now on I'll make my own cars, then I can be sure they work the way I want them to." Fast-forward to 1963 and Ferruccio founded his grand touring car company in Sant'Agata Bolognese, with the intention of building the "perfect car."