Catch up on stories from the past week (and beyond) at the Slashdot story archive

 



Forgot your password?
typodupeerror
×

Comment Re:Wish I could say I was surprised (Score 1) 178

However, not reading my post before commenting, taking me out of context, using various straw man arguments, etc is not constructive or valid.

I don't believe I have. I can only go on what you've written in each individual post. You seem to take offence awfully easily and jump straight from holding forth to flaming everyone who has criticised you. Come to think of it, now I've read the whole thread, there have been many carefully thoughtout critiques of your post and you've jumped straight to flaming and ad homenim there too.

As to the point of red asterisk, I pointed out that this was mostly for the lay community that gets a lot of their science news from the media that is full of a lot of people that don't know any better.

Yep, I got that, and if you took the time to read my reply you'd have read my reply, for starters.

I don't think it would help. The thing you need to do is educate people on what the red asterisk means, not just put it there. Once you've done that, you can simplyrealise that all work is de-facto red asterisked. That makes it redundant.

And then the scientists if only out of irritation because the lay community that ultimately pays for everything keeps bring it up... will have it reproduced somewhere thus removing the red asterisk and moving on.

The scientists do this anyway and not out of irritation. And instead of a red asterisk, you get a list of things citing the current article---most journals track that information. Of course you have to read the citations to see if they're reproducing it or simply padding a lit review.

But that information is already there. The trouble is you have to have the education to understand it. But you'd need that to understand the red asterisk as well.

Here you say but all of that is redundant... its obvious... except it isn't for the laymen. So it isn't redundant. It isn't obvious. Put it on the paper.

No it's not obvious but I think the red asterisk would be less obvious than you expected as well. Sometimes they'll get removed.

If you told the people, give us X dollars and we'll produce research that will yield everyone X*100 then you'd get all the money. ALL OF IT.

I doubt that: first, you can't tell which research results will pay off financially, or how much, or over what time scale. That's impossible to know. Secondly why would they want to give you money to tell them something they already believe they know?

This is a big factor in a lot of spending. Now you can't ever make those sorts of promises. I appreciate that. But giving people better reporting that is understood and can be turned toward something practical means your funding will flow a lot easier.

OK, so you pointed out my criticim with that already. I don't get it then. That's impossible. But I don't get how the red asterisk will solve the problem of turning science towards something practical. The people who do that are usually the scientists involved. They know if it works or not and understand enough about the field to beable to deduce red astrixes themselves.

But then don't bitch when the funding gets tight because the "trust us" argument is only good for limited funding.

And you accuse me of circular arguments? The mind boggles. Your solution to trust is to say "no honestly trust us, we've put a red star next to it". In terms of trust you're back to square 1.

If you want the money to flow... you have to give us something more. You have to make us understand.

You ask the impossible. No one can force someone to understand something.

Actually. Not stupid condescending cartoons. We're not stupid. We're not children. There are a lot of things laymen understand about a lot of things that scientists of whatever description know nothing about.

So your point is that people have specialisations and it's unrealistic to (for example) expect a scientists to understand the deep things that some laymen know about thrie own specialisations. That does not support your point: why would you expect non scientists to understand deep things about science when you expect scientists to not understand deep things about non scientific fields?

Lets not treat each other like garbage and instead do our best to help each other come to a common understanding and from that move forward together.

That's fine and a good goal, but your system of audits and red stars won't achieve it.

Comment Re:Wish I could say I was surprised (Score 1) 178

Here is what I want...

OK, you can want what you like. Hoewver for it to be a practical solution it has to be better than what we currently have and if more expensive, then the money has to be raised from somewhere. It also has to produce usefult results. I do not think your solution meets those criteria.

Therefore I think that what you want is counterproductive.

1. It is reasonable for scientists in the pay of the public to be required at intervals to publish the results or at least what they were currently doing over the past few months or year or whatever interval is deemed reasonable.

Reasonable? Yeah I guess. Pointless? You bet. The EC and RCUK already require this. The documents disappear into a black hole of woe never to be read again. Have you ever requested one? I believe they are a matter of public record and if not, a FOI request would surely work.

Most of what's worth writing already gets put into papers. Not everything (sometimes we abandon papers which ae too hard to get through review or for lack of time), but some of them wind up on arXiv and whatever anyway.

I do not know if it is a coincidence that the EC funding has the strongest reporting requirements and the smallest output pre unit of currency invested.

2. Works thus published should be subjected to reasonable audits to detect fraud, laziness, waste, or incompetence.

Audits are already done on the spending by some of the major funding councils, such as the EC Framework grants. If you meet the deliverables you agreed with the money they gave you're fine.

Laziness? Well, if people are lazy they don't have much to publish. That's easy enough to find by looking at research output.

Fraud is much harder. How do you propose to do these audits?

Incompetence is generally covered reasonably well by peer review. It's not perfect and doesn't have 100% success, but no system is ever prefect.

3. The nature of audits should make it difficult or impossible for conflicts of interest to corrupt the auditing process.

Well, that's just wishful thinking. I don't think it is possible to design an auditing process for use by humans which is corruprion free. Never mind in science just look around at all the other auditing that goes on. Sure you can want this if you like but that doesn't make it possible never mind practical.

Unless you can propose a practical solution for such audits...?

4. The auditing process should be sufficient to determine what is and is not valid science.

5. Reproduction of work obviously cannot be done with all papers however, they should be done with all significant work deemed significant.

6. The deeming of significant or insignificant work could be down to collective or crowd sourced choices made by other scientists to cite a given work or say they found it interesting or significant. When X number of scientists say its significant then someone in the community should be tasked with verifying it through reproduction.

You know this is already how it works right, except for a minor change in the last sentance? If work is deemed significant and interesting, then others will try to build on it. To do that they will naturally replicate it. There's no formal process, but it is nonetheless what happens. No one gets "tasked", they just do it anyway.

And really significant results do attract interest. High temperature superconductivity. Gigantic magnetoresistance. Cold fusion. GFP transfection. Treating cells with acid to gt stem cells.

All of those examples were really interestind and had huge potential for lots of interesting new work. As a result they got a lot of attention which hadthe effect of determining correctness or not. They're just some of the more well known example. In any field there are examples of the same sort of thing. Papers which are significant attract attention because they point the way forward (or not).

Do you have a problem with any of the above?

1 already happens to some extent and is generally pointeless. 2 and 3 are unworkable. 4, 5 and 6 already happen in an informal manner.

Comment Re:Wish I could say I was surprised (Score 2) 178

Well, when it comes to doing browbeating, you're doing a bang up job. You are assuming that your solution is right and will not accept any criticism of it no matter how much of a bad idea it is. Your solution is unfortunately unworkable.

Is the current system perfect? No, not even slightly? Is your solution actually a solution? No, yours is a cure worse than the disease, or at least unworkable.

The point is that money goes to scientists and the people that provide that money have a right to expect something be done with it.

That's what publish or perish is! The money is largely provided by the public and not unreasonably, they want to see that the money is being used. Spending that money going in circles replicating results for the sake of it doesn't yield much of use, sadly. So, you'll have to convince people that it's a good thing that now (say) half the amount of research is getting done.

Furthermore, they must share information... you don't like the term publish or you're going to get asinine on the issue? fine... We'll start using other words. I'll speak chinese if I need to get you to stop trying to make this a semantics debate.

The point about publishing is that as soon as you have a result worth sharing, you share it. In fact if anything, publish or perish encourages that. The opposite: dumping out stuff because there's a reporting deadline does actually happen. For example any EC funded projects of which there are many hav mandatory reporting deliverables. I think a few of the RCUK bunch do as well.

Guess what? No one ever reads the damn things, mostly because any results worth sharing are published as papers. So in fact what you advocate does happen and is demonstrably useless. You can in fact go and request copies of these documents if you wish. I believe they are a matter of public record.

By all means... put out as much research as you want that no one could possibly verify or reproduce. Make my fucking day. But it gets the red asterisk.

I don't see what function that would serve. All research is already considered to have a big red asterisk by default. You might weigh the liklihood of correctness by the content of the paper, the believability of the result and the track record of the researchers, but new publications are generally taken to be unverified.

So your red asterisk would not serve any purpose.

You claim you want to make it easier for "laymen" but the red asterisk won't help, because it's already effectively there. The best thing you can do is stump up the money to educate them instead.


6. As to the money to reproduce it, that can be provided by the same institutions that hire the scientists in the first place as part of their quality control policy. Which is in large part what all of this in the first place.

And where does that money come from? Who wants to double the research costs of universities just to do what's already being done by a less formal method? Whether it's privately or publicly run, the people stumping up the hard cash are going to want to know why the output has halved or the cost has doubled.

Would that money go to the same scientist or the same type of scientists? Probably not. We might have specialists that ONLY reproduce other people's work. That might be literally all they do. And they might be paid by the scientists that produced the paper who are themselves taking the money from their grants or working budgets as a cost of publishing.

Well that would shoot a huge amount of research in the foot. I've published a fair chunk. All of my most major publications were done either on the side with no budget at all or in a small research group that had enough money for 3 PhD students (and this was way back when we were 10k per year) and almost no equipment (seriously we were on hand-me-down computers that the better funded groups junked due to obsolescence).

Come to think of it quite a lot of people I know got their big breaks by looking at some side project that interested them and wasn't really part of the main funded research.

The money you're talking about simply doesn't exist. There was never even money to to the research never mind pay someone else to do exactly the same.

here you'll tell me they don't have enough money to do that... well obviously not because they didn't need to do that before so they weren't given the money to do that.

So where will the money come from? You can't magically "get it from the institution" because the institution has to get it from somewhere. In the UK this means taking it out of paid taxes.

The thing is that despite your efforts, your method still doesn't solve the fundemental problem of whether you know something was correct or not. The verifier could make a mistake in either direction, or worse you'll never eliminate fraud.

The current system does however eventually sort it out. If it is an important result (defined as one that people care about) other people will look at it. And soon the truth will come out.

Two good examples: high temperature superconductivity and cold fusion. Both HUGE results with massive implications. So, scientists swamed over them like flies. The first was replicated and built on and is now a hige field in its own right. The second is now dismissed as an error.

The thing is this required no formal system of verification, but he measure of "importance" was automatically crowdsourced (I hate that phrase) among the scientist's peers. Interesting work attracts more scrutiny. And now, it doesn't matter if the original results were verified or not. Frankly it doesn't matter if the sampes and data are long lost. High Tc is firmly established as a fact.

Slashdot Top Deals

THEGODDESSOFTHENETHASTWISTINGFINGERSANDHERVOICEISLIKEAJAVELININTHENIGHTDUDE

Working...