Well I don't see people joining PETA and saying "Hey you know what, our views are a little extreme, lets try be a little more level headed".
I don't see people joining Greenpeace and saying "Hey now, Genetic Engineering's alright y'all". And lets not get started on Sea Shepard.
You also don't see hippies and vegans going to MacDonald's or Wallmart and working there in the hope to make it more ethical.
The point I am trying to make is that GNU started as the environment for people who cared about those Freedoms. Linux became part of that and is Licensed under the GPL. It is part of the Ecosystem that cares about those Freedoms. To turn around and say, well maybe those Freedoms aren't important, maybe we should become more mainstream so we can cater to the masses who like MacDonalds and Wallmart and don't care about Hens in cages or sweatshops, is kind of besides the point.
We all have our own reasons for using Linux but it would not exist without those freedoms... If you have a different view on freedoms you can also use *BSD, Solaris or something like Haiku (Etc. etc.). If you don't care, there is NOTHING that is stopping you from using Windows or OSX.
I certainly know that if I emigrated to a country and started saying people should follow my political views I certainly wouldn't be well received, it's no different with the F/OSS sphere. It is what it is. It is what it is because of what it is and really, most of us have bigger mouths than we should.
The Developers are free to do what ever they want and their projects can go in what ever directions they want them to. Users like me can be thankful for what they give us. Yes some are more rabid in proclaiming the Freedoms, but then again if a single project isn't free enough, a half-assed effort of replacing it is at least made.
Long post after a tired and long day tl;dr: Freedoms could be only a concern for a minority, but a large part of what exists is because of them. Even if they aren't the most important thing doesn't mean they aren't important.