I recently had the great pleasure of meeting up with Kathryn McKinley, a faculty member at the University of Texas at Austin. She let me know that she is publishing an editorial pushing for double-blind reviewing in SIGPLAN conferences. The editorial is EXCELLENT. Here is a snippet:
“The purpose of double-blind reviewing is to focus the evaluation process on the quality of the submission by reducing human biases with respect to the authors’ reputation, gender, and institution, by not revealing those details.”
Simply put, sexism and nepotism are real. They may be unintended, but they happen. Double-blind reviewing improves quality and ensures that the best work really does rise to the top!
jazzyb said,
August 9, 2008 at 12:16 am
More than sexism and nepotism, I think its about affiliations. The academia would definitely take a paper from MIT more seriously(and subsequently accept it) than from an assistant professor from a 100th ranked university if they are both doing incremental research. Past reputation and fame also precedes reviews. And frankly, I don’t think sexism exists today, atleast against women.
parolefemmine said,
August 9, 2008 at 9:20 am
Thanks “jazzyb” for your feedback. As you can probably guess, we disagree on the issue of whether sexism exists.
amisha said,
November 5, 2008 at 10:52 am
hi, interesting points on sexism in academia. i’m sure it still exists. one should also look at the salaries of female professors. i have a feeling that they are paid less than male profs who are at the same level and experience. btw, are you the same vibha who is married to vijay ravindran who works at catalist? i think i’ve met you guys in seattle.
parolefemmine said,
November 9, 2008 at 11:22 pm
Yes, that is me. I don’t know if there is a salary difference between
men and women in academia. I do know that many salaries at public universities are publicly accessible. Thanks for writing!
emerson said,
November 15, 2008 at 8:06 pm
Howdy Vibha,
We met once at CCSW; 2003 I think. I ran into your student, Yit, and FSE this week, and was interested to see that we’re working on similar stuff.
Anyway, I’m not a big fan of double-blind reviewing. As McKinley noted, it’s typically not difficult to determine who wrote what paper, if you’re familiar with the field, so I don’t think that it reduces the problem of reputable authors/institutions getting preference.
I have no idea if it reduces sexism, but the hypothesis that it does can be tested. Do conferences/journals that instituted double-blind see an increase in acceptance from female first authors?
Also, if you’re suggesting that there’s sexism in the review process, how do you think that sexism arises during that process? What might a sexist reviewer be thinking?
Cheers, and hope to see you at some conference next year.
parolefemmine said,
November 17, 2008 at 4:26 pm
Hi Emerson,
Thanks so much for visiting my blog and taking time to comment!
If interested, please see an earlier post about a study of double-blind reviewing and
increased female author acceptance.
I disagree, even with Kathryn McKinley, that it is not difficult to
determine who wrote what paper if submission is double-blind. There must be tens of thousands, if not more, research institutions in the world. How can one person know what all people are doing everywhere? That’s like the claim that one knows everything relevant to their topic on the Internet. I am confident anyone willing to make that bet will lose. I read a healthy amount of non-computer science stuff,
and you would be amazed how many people in operations research, human factors engineering, linguistics, etc. do stuff very similar to us.
How does one claim to be familiar with an area? They go to conferences and read top-tier published journals? They review what top labs are up to? By definition, that encourages nepotism to me. Even as a reviewer, I see unpublished work, but I usually only see 30 out of 300 submitted. There is a lot I don’t see in any depth.
I see an attitude at conferences that I’ll express like this:
“Yeah, I don’t know everyone in my field, but I know everyone that’s good.” That’s
exactly the kind of thinking I want to challenge. The world is too big for one person
to know everyone’s that’s good.
It’s possible that in a double-blind situation that PCs will only accept work they recognize. That will be quite sad, because they’ll only recognize incremental work.
I think inevitably some new stuff will get in, but everyone will have a fair shake at getting their new stuff in if reviewing is double-blind.
Research is fundamentally changing. My lab, HCIL, has faculty in it from Psychology,
iSchool, Computer Science, Business, English, Education, and more.
I have trouble keeping up with what everyone in my own lab does, let alone the whole world!
You ask how sexism manifests itself. Virginia Valian, author of Why So Slow?, posits one hypothesis called “Perception of Competence.” You might be interested in the following resources:
1. http://www.bu.edu/diversity/search-manual/raise-awareness/index.html
2. http://researchnews.osu.edu/archive/genster.htm
3. http://www.bnl.gov/rhic/news/060308/story1.asp
4. http://www.nature.com/news/2008/080423/full/452918a.html
5. http://www.springerlink.com/content/l265hk88l5q62270/
and, while I’m at it:
something about minorities:
1. http://findarticles.com/p/articles/mi_m0DXK/is_9_20/ai_104521293
Thanks again for writing. –Vibha
Anonymous said,
December 20, 2008 at 1:23 am
There must be tens of thousands, if not more, research institutions in the world. How can one person know what all people are doing everywhere? That’s like the claim that one knows everything relevant to their topic on the Internet. I am confident anyone willing to make that bet will lose.
It depends a lot on how broad the field is, how selective the conference/journal is, etc. My only experience with double-blind refereeing is for the yearly Crypto conference. The research community is fairly small and important results are typically well disseminated before submission as preprints, seminar talks, or just word of mouth. My experience is that when I review an important paper, or any paper by a top researcher (even if it is not one of their very best), I typically know perfectly well who wrote it. There are also a lot of papers where I may not be absolutely certain who wrote it, but where I can easily narrow it down to half a dozen possibilities. For example, it’s sometimes clear that it was written by a student of X, or by one of several people who sometimes collaborate. The net effect is that I can regularly identify the authors of half the papers I review, and I can make a good guess about some of the rest. The papers where I have no idea are often pretty bad, and there really aren’t many fully anonymous papers that are serious contenders for acceptance. So double-blind refereeing still makes a difference, but only in a limited number of cases.
I’m told this varies tremendously between fields. Some fields (for example in biology) have overwhelmingly more researchers, and researchers in some fields are less likely to distribute their work before submission.