how not to run a research study

[nonelvis]

I know several people on my flist are scientists, researchers, or generally interested in science research, and since a few of you may not have heard yet about the fan fiction survey, consider this an object lesson in Doing It Wrong. Seriously, how do you get to the point of having a PhD in a scientific field and not realize that you can't change the questions in the middle of the survey without affecting your data? (And that's the least of the problems with this little project – it's just one of the most obvious points of Science Fail.)

The survey is finally down while the researchers "digest all this great feedback." Personally, I agree with the anonmeme theory that this is subtle code for "BRB, BU IRB SMACKING OUR ASSES, AND NOT IN THE GOOD WAY YOU FOLKS WRITE ABOUT."

Comments

[papilio-luna.livejournal.com]

And this, children, is why we pilot test our research instruments before loosing them on the world.

[nonelvis]

Why do that when fandom can do it for him?

*headdesk*

[papilio-luna.livejournal.com]

...And I just did more clicking around (this is honestly the first I'd heard of it!) and woooooow.... *___* my god it's full of stars. Cripes.

[nonelvis]

It's like they were shooting for the Nobel Prize for Science Fail. Well done, gentlemen. Well done.

[elliptic-eye.livejournal.com]

I can't decide whether this one (http://ogi-ogas.livejournal.com/3544.html?thread=112600#t112600) is my favorite, or the subtle WTFery of this one (http://ogi-ogas.livejournal.com/3544.html?view=267224#t267224), or the not-so-subtle weirdness of this (http://ogi-ogas.livejournal.com/3544.html?thread=120024#t120024).

(The cluelessness here (http://ogi-ogas.livejournal.com/681.html?thread=3241#t3241) is rather impressive, as well. LURK MOAR, BB. 4 SRS.)

[nonelvis]

Girl or boy activities?!?!?! I don't know how I missed that question, but it is jaw-droppingly stupid and offensive. (Well, I'm sure you could ask that question in a way that isn't offensive on a properly designed and vetted survey, but this sure as hell isn't it.)

I do remember seeing his response about novel-length fics, and like virtually every other answer he's given, it made me facepalm. How can you even pretend you're going to get valid answers studying fic writers if you haven't spent more than five minutes learning what fic is?

[ionlylurkhere.livejournal.com]

I'm convinced the whole exercise was only ever looking for anecdotal evidence for their non-peer-reviewed book, with lots of strange leading questions that seem to be designed to get people to tell stories about their involvement with fandom that can be used to point up all those Essential Gender Differences.

[nonelvis]

Ah, a "social experiment" ;)

It would make somewhat more sense if that were the case ... though if all he wanted was stories about involvement with fandom, I think he's got plenty of first-hand experience now.

[prof-pangaea.livejournal.com]

yep. and i could see where that survey was going from the first two questions -- "what was your biological sex at birth?" and "what is your current gender?". since their book deal is for a book that will "prove" essential gender differences, it's easy to see how that's going to dovetail into "hey trans people, you're not really a man becuse your BRAIN is female!" and vice versa, completely leaving out any kind of evidence for the roles of socialisation and, of course, individuality, since their whole point is to make a "unified fabric of human desire".

[elliptic-eye.livejournal.com]

This. It's just so fucking patently bad faith through and through, and you know what? I've realized something. People presenting lie as fact as pertains to gender makes me angry. But that's probably just because I'm an irrational wimmins and ruled by my emotions and things.

[nonelvis]

And your hormones. Don't forget those. It's why we can't be president, you know, 'cause we might get so crabby during the wrong time of the month that we'd start a nuclear war.

[prof-pangaea.livejournal.com]

yah, i'm not very fond of people presenting lie (or lazy presupposition) as fact in regards to anything, but gender is definitely one of the big DNWs.

[iainpj.livejournal.com]

And now people are so mad that apparently they're creating RPS and art about the researchers for the express purpose of humiliating them publicly and shoving it in their faces. One could say that the researchers are indeed getting the full and forthright fandom experience.

[nonelvis]

I confess the one commentfic I read made me laugh even as I cringed, because funny as it was, I'm not really okay with creating RPS of non-public figures.

I haven't seen any artwork and would like to keep it that way. Again, it's the fact that they aren't really public figures that bothers me. They are both utter tools and deserve to be roundly insulted, mocked, and excoriated for their behavior, but not like this.

Still, as you say, they are getting the full fandom experience, and are about to get themselves into yet another shitstorm, if you can believe the letter at the end of this post. 99% of me feels like these dumbfucks need their asses handed to them ... the other 1% feels like they need a gentle reminder about the fact that people on the far right tend to carry guns.