I don't think at all that man hasn't affected the climate in a way that tends to disturb equilibrium.
I'm not convinced, however, that a single global average temperature is a meaningful metric. Since climate is varied enough across the globe, that single metric seems to lose too much information. Far too much information averaged together.
For instance - yes the average has increased, but is that more or less important than the change in range of max to min temperatures? How does that vary with geolocation? If we know some areas are going to get better climates, why not start putting infrastructure there now to mitigate the "horrors of mass migration" in a few decades?
That's what bugs me about the "global" number - the effects are not equally distributed, so averaging everything together into that one number, while perhaps helpful in some broad sense, doesn't correlate directly enough with useful actions (especially because in general it affects probabilities or trends, not "concrete" events for enough people to understand).