In light of my post back on 1/5/2016 I wanted to share my experience and also happy news that I had the opportunity to review my first article recently. I can’t reveal the article until if/when it is published but I can share my general experience (and continued experience) with the process.
In some ways the review process seems kind of like what you go through with your labmates and friends when they are pitching you a new idea. They’ve come up with some kind of problem and are really trying to convince you that their way of solving it is correct. In this case the “problem” is actually a hypothesis that a lab group somewhere is trying to support or refute. A sound hypothesis should be the first thing to look for when reviewing an article. Do the writers actually have a well framed question that they are asking? In my mind this is even more important than the motivation for the work. You can “we’re going to cure all cancer” as much as you want but that idea is so vague and complex that it’s difficult to approach it with any kind of scientific method. They don’t need to come out and say directly, “our hypothesis is X” but I feel that as a reviewer I should be able to put together a claim for this paper that can be tested to be true or false.
After determining what claims or hypotheses the paper makes, I need to be able to determine what data exists to support or refute this claim. One important point that I think we all should note is that not all of the data has to support the claim for it to be good science. In fact, I highly doubt every single experiment that this group did worked out positively. It’s almost more worthwhile to build a paper around both positive and negative results. This tells me something about the limits or the scope of the author’s hypothesis. So how confident do these results need to be in order for me to accept their evaluation of their hypothesis? This is the subjective part of the “peer review” process. I’ll say for myself that I’ll use the legal standard of “beyond a reasonable doubt”. Do I believe that 95% of the evidence (again not all) goes one way or another to support or refute the hypothesis? You’ve heard me say before that the scientific process is about asking questions and developing a response to those questions. We have to be careful to realize that what we are looking at are discoveries and not absolute facts. For this reason I think the 95% standard holds pretty well for determining validity of results.
Finally, the wishy washy part of the process, and really the last part in my opinion, is to determine whether or not the paper fits in that particular journal. This is something that I’ll have to report back on as the decision is ultimately not up to me but up to the editor. I would say that this paper certainly has “merit” but is it reaching the correct audience? The ultimate goal would be for this paper to be cited and to be used to further other scientific researcher. Predicting a papers reception by the scientific community is a tough job; that’s why you get your “peers” to review it. I hope that my opinion of this paper aligns with that of the community that will use it.
I don’t know if this post provided a guideline so much as just a little taste of what it’s like to review a paper. When there is another development in the process I’ll update you about it. Thanks for taking the time to read and best of luck in publishing your own papers!