PEER Review or Pier Review?

As has been stated previously, “scientific” research that can not be validated is not scientific and certainly not research according to the standards of accepted practice. I have stated that publications of reports in the new media that use words like “may”, “could”, “might” and many other conditionals should not have been published as scientific work because the comments obviously express the writer’s opinion of what he or she read or observed.

The one constant has been that if something is published in a peer reviewed publication, experts have reviewed it and made comments on the validity of the methodology, the thoroughness of the investigation, and the soundness of the conclusions. If modifications are needed, the author may need to resubmit the manuscript with changes for further review. Once through this process and published, the reader of the publication can understand that the information being presented is accurate.

Unfortunately, there has been a major revelation that a major scientific organization has been publishing research results that is presented as peer reviewed but is filled with many flaws and false information. [1]

What is involved in “peer” reviews? As a founding and past editor-in-chief of a peer reviewed publication, I have some experience with the process. Usually, a paper is submitted to the organization for review and publication. The paper is logged into a data base for tracking purposes. The paper normally has an abstract and a list of key words. The editor

If a summary or overview paper is requested for a special topic, the review process might be somewhat faster because it will not be presenting new research results and as a summary paper will not be totally comprehensive on all possible authors or related papers. It should cover the major topics. The review of this type of paper can be done by as few as two people plus the editor.

The assigned editor reviews the paper and makes a decision that includes: a) contents/topic acceptable for the publication; b) contents acceptable, but the format is not acceptable and the author will need to reformat the paper to fit the requirements of the publication before being considered; c) recommend that the paper be submitted to another publication; and, d) outright rejection due to inappropriate topic or sloppy work. If the work belongs in the publication, the editor sends the paper out for review. For research papers, ideally there are four reviewers assigned, although three can be acceptable as long as all return comprehensive reviews. There is normally a date provided for when the review need to be returned. In the vast majority of cases, the reviews are volunteers and receive no compensation for their work. With additional publication constantly being created, this causes a workload issue for reviewers. Typically, a reviewer would not receive more than two or three requests per year for peer reviewed papers.

The reviews are received and compiled by the editor. The comment are compiled anonymously and the critique of the paper is sent to the author with a recommendation. This normally falls into one of five categories. The first is publish unchanged. In more than ten years as editor-in-chief, I do not recall ever having a single paper where that occurred. The second category is publish with minor changes. The third is requiring a revision of a section or clarification of some section. This can be a restructuring of the contents to have an easier flow to subject and analysis, which I have occasionally done as a reviewer. This may or not require another pass through the peer review process. The fourth possibility is to require that the paper have additional work and/or additional sections added to the paper, which then must go through the peer review process again. The last category is to outright reject the paper for technical, research procedure, or other issues that invalidate the paper.

Some areas, such as new technological developments, can be challenging to find enough capable reviews for rapidly developing technologies. The editor has to be knowledgeable in the subject matter to be able to reach out to possible reviewers that possess the background to review the work. Anything short of this process is not a true peer review process.

From some of the published information on authors picking who reviews their papers does not provide confidence in the results. To me, it is like two researchers going down to a lake on a nice summer afternoon, having some beverages, and sitting on the pier talking about what would be good to publish. And, then publishing that result.

The problem with bad science is that people will believe it if it sounds credible. Legislatures will make regulations based on it. People in general will be subjected to the consequences coming from the bad science. This is especially true for an emerging field like nanotechnology. Where does that leave us? Basically, trust nothing that does not have peer reviews with sufficient details of the experiments, the results with all the data, and conclusions based on the preceding information.

References:

[1] http://online.wsj.com/articles/hank-campbell-the-corruption-of-peer-review-is-harming-scientific-credibility-1405290747

About Walt

I have been involved in various aspects of nanotechnology since the late 1970s. My interest in promoting nano-safety began in 2006 and produced a white paper in 2007 explaining the four pillars of nano-safety. I am a technology futurist and is currently focused on nanoelectronics, single digit nanomaterials, and 3D printing at the nanoscale. My experience includes three startups, two of which I founded, 13 years at SEMATECH, where I was a Senior Fellow of the technical staff when I left, and 12 years at General Electric with nine of them on corporate staff. I have a Ph.D. from the University of Texas at Austin, an MBA from James Madison University, and a B.S. in Physics from the Illinois Institute of Technology.
Misc Ramblings, Nanotechnology Risk Management

Leave a Reply