Sunday, August 8, 2010

"Forum culture" in academic journals

I don't read academic journals as much as I, a grad student in history, should. Journals are important because they acquaint scholars with the state of the field, often in a more comprehensive way than simply reading monographs. (In the historical profession, monographs are emphasized more than I understand them to be in, say, political science or legal scholarship, where journal articles are even more important than in history for tenure decisions, etc. Of course, I stand to be corrected.)

One characteristic thing about journals is the occasional "forum" issues, where several--usually three--scholars opine about a work--usually a monograph, but possibly a previous journal article--of another author. These forums (fora?) give readers exposure to a variety of viewpoints, and in that respect are quite useful and important.

But they also follow a vexingly predicable format. Consider a forum about the work of author A in which scholars X, Y, and Z participate. Here is the format that is usually, almost always, followed:
  • X writes a long, often 20+ page, thought piece on author A's work.
  • Y writes a shorter, about 5 to 10 page, piece.
  • Z writes an even shorter piece, usually only 2 or 3 pages.
  • A writes a "response" that usually runs like this: "X, Y, and Z bring up many good points, but in general I'm right and they're wrong, and here's why....."
Just once, I'd like to read a forum where author A says "I enjoyed reading the commentary of X, Y, and Z. Although I think my original work had made many valid points that need consideration, it appears I will have to fundamentally rethink what I wrote before."

Perhaps this does happen occasionally. As I've said, I don't read journals nearly enough. Still, it would be nice to actually see it happen.


James Hanley said...

My (relatively limited) experience with peer review leads me to believe that most scholars often don't really get the original article, make a big deal about a minuscule error, or use their review to grind their own intellectual axes. So I almost have to wonder if in fact there aren't that many cases where the response you'd like to see--and I agree it would be delightfully refreshing--is in fact warranted.

For example, I once had an anonymous review misquote me multiple times, then scathingly critique the misquotes. I actually agreed with his critiques--had I written what he said, I did, I would have deserved all the scorn he delivered. And my mentor and I were denied an NSF grant despite two glowing and thoughtful reviews on the basis of the third reviewer, who wrote a single paragraph, none of which dealt with specific merits/demerits of our proposal, and most of which was encapsulated in his final phrase; "I think this evolution stuff is bunk and shouldn't be funded."

I did once write a response to an author who had written a critique of a paper I'd co-authored. We actually had to request that he rewrite his critique, because we couldn't make heads or tails of it. Once we had an intelligible version, we recognized that he had in fact noted a significant error we'd made, but that--for very simple reasons--none of what he said flowed from it actually did, and our original conclusions were still the necessary ones (it was a fairly formal analysis, so conclusions could in fact be necessary), although the reasoning had to be changed in one important instance.

I don't know that peer review works so badly in all disciplines. I think in some of the sciences conclusions can be fairly hard to refute by cheap responses. But in disciplines where we're dealing with less certain empirical facts, it can be more easy to find justification for rejecting an unpopular conclusion. History may fall into that, as well as political science. People like me tend to opt out of the game (I mostly write for non-peer reviewed sources now.) We tell ourselves we're opting out of the game, while others say we couldn't do good enough research. Probably six of one, half dozen of the other.

As to journals v. monographs, I think you're right that political science puts more emphasis on journal articles than monographs. But for tenure purposes, a single monograph will usually ensure it, whereas it might take several journal articles.

Sorry for the long-winded reply.

Pierre Corneille said...

Mr. Hanley,

Thanks for the comment. I apologize for not responding sooner, but I have been more or less off the grid for the last few weeks because I've been moving.

I probably went a little overboard in criticizing peer review, and my overboardness was probably so much more overboard because I haven't actually yet published, or tried to publish, in a peer reviewed forum. I have, however, presented a pitiful number (3) of conference papers, and I can honestly say that at one conference, I received a very critical, yet friendly and well-put, comment that set my research off on a different, and in my opinion better, direction.

I like your example about the commentator whose comment was incomprehensible. I obviously don't know the specifics, but I wonder if it had something with him or her having a legitimate criticism, and yet wanting to be polite and non-confrontational in expressing it. I have sometimes, in non-academic settings, worded comments and criticisms so abstrusely so as to avoid confrontation that my interlocutor didn't even realize I was offering a criticism and probably didn't even know what I was saying.