For what it's worth, I was tangentially part of an effort in the University of Wisconsin Libraries to publish the open-access Journal of Insect Science. After perhaps a year of doing that, we looked at the actual costs and found that, IIRC, $30-$100/page are not actually unreasonable costs. Yes, there's a large variance.
"How," you ask, "could it possibly cost so much to produce an open-access journal? The author is working for free! The reviewers are working for free!" Well:
- The reviewers are generally not particularly excited to spend their time reviewing papers. They often say "sure" and then just never do the work. So you need to keep on them, and swap them out for other editors when they flake out. You need to do this without giving them a sad.
- Different reviewers have different areas of expertise. So you need to match the content of the article to suitable reviewers. Your editor should do this, but probably isn't getting paid for that effort, and so you may need to keep on him/her to get that to actually happen.
- Your authors don't know how to use word processing tools or graphic design tools. You'll get horribly-formatted documents, figures as 36-DPI .GIFs, strange-looking Powerpoint god-knows-whats, and gigantic tables that will never look good anywhere. Your job is to either guess at what the authors meant to do, reformat materials, and send them back for approval, or get your authors to re-do their stuff.
- Authors also routinely ignore things like word count limits and organization guidelines.
- While you're primarily targeting the Web, a good-looking print copy is still widely-valued. So you probably need to handle your layout nightmares twice.
- Sometimes, people want to do Something Innovative with regard to data vis. After all, you're online, so you should be able to make this interactive, right? So, you need to decide if you want to try and implement this innovative thing (and what does that look like in print, anyhow?) or say "no, sorry, we can't make your research look super neat."
- Online repositories work best with specially-marked-up XML. There are tools and services that will do this for you, but they all cost time, money, or both. XSLT to turn your XML into HTML or PDF can be made to automatically give you a product that is not quite nice enough to present to the outside world -- there are usually some special cases that want hand-massaging.
- Both faculty and grad students can, at times, act like complete jerks and require a bunch of time in damage-control.
I could go on, but you get the idea. The bottom line was that we found PLOSOne's costs to be broadly reasonable (also: did you know you can essentially say "I don't want to pay" and... not pay?). Maybe it would be possible to undercut them by a factor of two with real work in process management, but generally: there's a bunch of grunt work turning researchers' paper submissions into a good quality journal. And you could get really fast at formatting, but time for catherding and massaging egos (so you don't lose your reviewers) scales linearly with the number of articles you publish.