Sure, those are all pretty reasonable comments. I'll describe two ways of working with Git that I think address this somewhat.
I guess first up I would say that Git doesn't really push you towards a specific way of working so if there is a particular way that you want to do things then chances are you can do that with Git (that said there are definitely ways to do things that would be less worthwhile than others).
We switched to Git from Subversion where I work about 6 months ago and it's worked extremely well for us. We have a small team but it is spread across two or three countries (some of us move around a bit). For us we use a central model where there is a single 'authoritative' repository which we all push and pull from. So I don't share my repository directly with the other developers we all push and pull from the same central one (which we integrate with Hudson for CI).
So really we're not using the distributed for the sake of sharing code directly between us but because having access to the full repository (and did I mention that Git is fast?) allows us to do stuff we couldn't before. Once you've done the initial clone Git is very very fast (and things like svn log are much much faster with git because you don't have to do any network operations). Essentially we still have a central repository that update from and commit too simimlarly to SVN it's just that we can continue committing without a network connection and split our commits into sensible chunks of work that go into the central repository etc.
Now, in terms of doing something really distributed, you can do that too more easily than you described (though I've not really done it in any seriousness so not sure whether it is worth setting up). If you give me access to your repository, I give you access to mine and we both have access to a third repository. We can set up our own repositories to pull from both the others - so it is not so much more difficult to fetch the work of the other repositories into our own working tree. There is a tiny bit of extra work here - generally Git commands are bit more fine grained than something like Subversion so you do end up with more commands. For my use (and for my team) I've written some simple scripts that tie some stuff together (so I run 'git up' which simulates 'svn up' or 'cvs up' you could easily do the same for fetching from multiple repositories if you wanted ... ).
Generally I wouldn't really recommend working that way for a small group. What would make more sense would if you had two teams working on a product (maybe a team doing support and a team doing a product release) the release team might want to pull changes from the support team as they're finished and make sure that they're incorporated into the next release. Generally I think you'd have one person doing that and pushing it into a central repository for the release team (so each team would have a central repository but you might pull changes from one into the other). All this is far easier with Git than with Subversion (and I've worked in situations in the past where something like what I just described would have saved a lot of heartache and angst between teams).
But generally your point about Git being more complicated to use is absolutely correct and, sadly, when you ask for help you don't always get a nice answer (sometimes it's more along the lines of 'well the manuals say you shouldn't do that so you did it anyway now you're screwed and it serves you right for not understanding the internals of our magically wondrous system). So... Git *is* awesome, I'm really glad that my company uses it but it does have a very steep learning curve.
In terms of ease of use I quite like the look of Kiln which is some proprietary code wrapped around Mercurial and looks relatively easy to use (in general my impression is that Mercurial is easier to use). That said - Git is free...
I hope that helps I'm certainly not an expert with the finer points of Git but I'm happy to tell you what I know if you're interested (even if you stay with whatever you use if you're more informed then you're still going to make better decisions)....