For example I've heard that the Canadian tar sands oil production requires a minimum price of around $85/barrel to be viable. If we start digging up oil shale it's got to be a lot more than that.
It's worse than that: more exotic sources of crude oil require more energy to extract—potentially more energy than is available in the oil itself! Because energy density is high from crude oil, it may make sense to "waste" energy in extraction because the result is useful as a transportation fuel, and also to stabilize relative prices between energy sources (e.g. spending cheap natural gas to produce expensive crude). But it still means that some energy extraction is a net loss, which is a very bad starting point.