I clearly haven't communicated my point very well. I hold formal instruction in the rules of English in disdain, not effective communication. I'm well aware of how you would check what effective communication is, there are even a small number of studies (done by people in Communication Sciences rather than the English faculty) which are very helpful in this regard. They follow exactly the experimental paradigm you suggest.
"This is the experiment that will back up the idea that people in general can evaluate the quality of somebody's writing -- generally an uncontroversial statement. Having established that, guidelines (much more common than "rules") for effective writing can be established by people who specialize in the field on the weight of their own experience."
No, this experiment wont back this up. It will back up whatever rule or guideline was being tested (say 'use a small number of words on a presentation slide') and only if the metric of quality applies. There is no such thing as /*the*/ quality of somebody's writing in general so there is no way to judge it. You can judge my writing by how well I communicate my ideas because I have told you I am explicitly interested in and trying to convey ideas, but it is not a general property. Sometimes we value writing for its obtuseness. Art which conveys layers of meaning subtly for example. To judge my writing style in such a way that I care you need to know my intent (a fact which flies directly in the face of the ever popular New Criticism and other post modernist bullshit which permeates the English academy).
You judge writing style in couple of ways, you can relay your experience, which is fine but only matters if I care about communicating to you in particular, or you can relate my writing style to some generalisable epistemology. The only one of those we have is the scientific method. English does not use the scientific method, as far as I can tell it uses no effective epistemology what-so-ever. If anything those who study English (rather than communication) are more disinterested in effective communication than most scientists.
I never said I was disinterested in effective communication. I said English majors and people who adhere to rules about how writing should be done can fuck off. While the statistics and experimental methods of communication science might be a bit dodgy at times I have nothing but respect for that endeavour and have used the fruits of that particular scientific process to improve my own writing and presentation.
The rest of your post I largely agree with. You outline a collection of nuances and particulars which would impact a careful, scientific study of science communication. Something which people who complain about how I end this sentence would have trouble getting at. People who study English are doing a bad job because they think there is an objectively correct way to write. They are doing such a bad job I just praised Communication Science as a discipline in comparison.