i always find it interesting when someone hits a meme.
Granted, i never will have to worry about that, what with being at the Used Slurpee Cup level of bloggers, but it's always interesting to watch the ebb and flow that goes with suddenly being in the collective consciousness of the digital hive mind.
Still, it's fascinating to see the sort of wave crash effect hit someone like good Madame danah boyd. Ms. boyd is a quite accomplished scholar of social systems and her thoughts tend to carry a fair amount of weight. She's an acknowledged expert in a given field, yet as an academic, she has two different "modes" of presenting her thoughts. One is very formal, involving calculated and well researched statements reflecting research and study and highlighting correlation and conclusion. In other words, your typical research paper.
The other is the equivalent of sitting at a coffee shop for an hour.
The funny thing is that it's often those latter statements that seem to have the highest likelihood of spreading like an Atlanta lawyer on a transcontinental flight.
Kent and i got into a quick discussion about this and i think he touched on something kinda key. He noted that it won't be long before academics realize that they don't necessarily need the structure and peer review that they previously always had. They can be more open and have greater impact without it. i disagreed, since i know that often that peer review system exists for a good reason and a lot of folks actually like it since it helps them refine and defend their assertions instead of just shouting them into the void.
What i countered was that the established peer review journal isn't threatened at all, it's simply undergoing it's own peer review process, as folks realize that there are some larger assertions that require that sort of review to lend weight to their importance while lesser observations may not.
So what you'd have is qualified research, and the academic equivalent of Mythbusters which may find out something just as important, but will probably also include broken stuff on fire while a bunch of folks are rolling on the floor laughing. Think of that later group as unqualified intellectuals.
Where the danger lies is for the general populace (that would be you and me) relying too heavily on the unqualified and ignoring or dismissing the qualified. In other words, if the smart folks are willing to be a little less rigorous with their observations, it's more important for us to pick up the slack. Recognize that an off-the-cuff statement by an expert is still an off-the-cuff statement and that while they may have insight we don't, it doesn't mean they're infallible.
For what it's worth, i'm actually really in favor of that. This is how it's supposed to work, and we all get a little smarter because of it.
And i can use the extra smart.
Well, by "peer review", I don't always mean "Ok, here's a group of five selected individuals who sign off". I think, if anything, the internet has expanded the sort of "peer review" process far beyond that.
Still, money corrupts, and I agree that it's always a good idea to research the backgrounds of any proclaimed expert to determine where their biases lie.
(That's why I'm happy not to be an expert and simply be a man of overwhelming biases.)
Save This Page
If your field is politic (as in, our pharmaceutical drug cost millions to develop, let's get some positive results out there), peer review is almost non-existent.
Pharmaceutical companies write "unbiased" papers, pay researchers big $$$ to put their name on them,and submit them to medical journals.
Pharmaceutical and medical journals get most (if not all) of their funding from pharmaceutical companies.
Over 90 % of all new drugs are withdrawn due to adverse effects within their first year, more than 99 % are withdrawn within the first five years … and all of these had "solid" research, published in "solid" papers, behind them.
So. Trust peer review, in a field where money is involved? Pull one of the other ones, it's got bells on.