Peer review is a cornerstone of science: before work is published in a journal it is sent to an independent expert, who quickly and anonymously assesses its quality; if the work is up to scratch, it gets published. This seemingly sensible system is now, unfortunately, outdated – and easy to manipulate.
There are now so many journals that the same work can just be submitted again and again until it finally gets accepted. All of the work of past reviewers is never seen by the community. Even the reviews at the journal of publication are never posted. This made sense when a few publishers printed real physical journals and physically shipped them around the world, but in the modern age of online publishing all of these important critiques could easily be put online alongside the papers.
Some new journals are trying to implement a much more sensible and open system, for example F1000 Research and ScienceOpen. For more traditional publishers who like to keep the old system, websites have been set up where people can review the papers after publication. However there is a bit of a disagreement on anonymity: with some believing that reviewers deserve credit, such as Publons or PubMed Commons, while others, such as PubPeer, believe anonymity to be important.
Some more forward-thinking journals such as PLOS ONE have set up an online comment system. In theory, this is perfect: now scientists can openly and publicly discuss science. What could be better than the free flow of ideas? Online post-publication peer review definitely has had its successes – arsenic-based life, nanochopsticks, and STAP being the most famous. But it also seems to have its dark side.
My paper criticising the evidence for striped nanoparticles seems to be the gift that keeps on giving when it comes to exploring issues in scientific publishing. Previously we made quite a stir when traditional journals refused to grant us copyright to reuse figures as part of our criticism. (Not to mention the fact they were also completely uninterested in publishing the criticism itself.) For post-publication peer-review it fails to disappoint.
Starting off on a positive note: A pre-print of our paper was first published on the arXiv, and it was extensively discussed at PubPeer. One of the reviewers there was Brian Pauw, an expert with important insight into some of the newest data we discussed. He even blogged on it, and eventually became an author on the final paper, which was improved by his expertise.
That same PubPeer thread was not entirely a shining example of a victory for post-publication peer review. PubPeer have a very sensible policy of numbering registered users so that a conversation can be followed without removing anonymity. Registered users also have the option to forgo anonymity. To register, you must be first or last author on a published paper – the system helps to prevent sockpuppetry and spam. PubPeer, however, also allow unregistered users to comment, and these unregistered comments have no identifying numbers. Some unregistered comments were helpful, and some were also signed with a pseudonym to help people follow the conversation. Unfortunately the vast majority appeared to be a Gish Gallop, an endless number of inconsistent and repeated arguments to give the appearance of a true debate, from someone we began to refer to as UnReg. This is where complete anonymity fails us. Anyone who wants to disrupt an online debate can simply flood it, and then hide under the veil of anonymity. Other, signed unregistered comments from ‘ProfSTM’ and ‘Bionanochair’ agreed with UnReg, but due to the lack of any registration these could easily be the same person. Sockpuppetry is a huge problem in online debates.
Anonymous comments are useful for someone to raise an issue, which can then be answered, but lengthy debates with one-way anonymity are dangerous. If you put your name to a comment, you are probably willing to stand by it, and willing for people to know this is your stance. Anonymity allows one to comment first, and run away from it later. This is unfortunate, as there have been a number of very helpful anonymous comments.
Our paper also received post-publication peer review on the PLOS ONE website, where all commenters are asked to identify themselves with their first and last names and their geographic location. We first were contacted by someone claiming to be a Gustav Dhror. We engaged with him, but after a quick google we realised that it is a fake name. Not only is there no academic record for Dhror, there is no record of anyone of that name ever having existed. Once this was mentioned, ‘Gustav’ went quiet.
Soon afterwards, we were contacted by someone claiming to be Wei Chen, with a homepage of the Suzhou Institute of Nano-tech and Nano-bionics (SINANO). Checking the SINANO website, Chen is real. Finally, a real person to engage with! But the tone of the discussion felt oddly familiar, which prompted co-author Raphaël Levy (who has also blogged on this) to phone Chen just to confirm it was really him. The real Chen had no idea about the comments, and copied me into an email where he confirmed he had “never posted anything at PLOS ONE”.
Commandeering the identity of another researcher for the purpose of sockpuppetry is truly shocking. Furthermore, to accuse other people of dishonesty while using a stolen identity is beyond belief. We have no idea who is behind this, but I would wager that the comments of UnReg, ProfSTM, Bionanochair, Gustav Dhror and Wei Chen were all written by the same person.
Sockpuppetry and fake identities are what we expect from flame wars in YouTube comments, not in scientific discussions. I also highly doubt this is a random internet prankster. They are able to quote text from inside paywalled journal articles, which implies they are at an institution with access to these papers and have taken the time to read the literature.
Post-publication peer review has the potential to speed up and enhance scientific discussion. But, whether or not comments are anonymous, there need to be checks to verify the identity of commenters – otherwise it quickly degrades into farce.
Image: Sockpuppets are fun for children but a nuisance for scientists. Credit: Shutterstock/Africa Studio
The post The dark side of post-publication peer review appeared first on physicsfocus.org.