On peer reviewing and a bizarre case of rejecting a review
A sick system gets sicker
This post is inspired by a recent bizarre experience I had with an “open research” journal rejecting my review that it invited. I have written about rejection of my papers elsewhere, but this is different — a rejection of my review.
1. Introduction
I have been involved in the scientific publishing nexus with several roles, as author, reviewer, associate editor and editor. I was in the cockpit (editorship) of the most historical hydrological journal for 12 years. I know the problems of the peer-review system. I have studied them and written about them many articles, editorials, joint editorials, etc.
I have reviewed and/or edited about 700 papers in 45 journals. In all my reviewing and editorial transactions I have been eponymous. I strongly dislike anonymity and I believe it is at the root of the sickness of the system. In each one of my reviews I include the following statement:
Reviewer’s assertion: It is my opinion that a shift from anonymous to eponymous (signed) reviewing would help the scientific community to be more cooperative, democratic, equitable, ethical, productive and responsible. Therefore, it is my choice, consistent with my aesthetic attitude, to sign my reviews. Furthermore, I believe the current trend in the review system to seek credit for anonymous transactions (by asking for recognition for anonymous reviews through Web of Science, a practice also encouraged by journals) is problematic on ethical and aesthetic grounds. Only eponymous transactions can deserve recognition.
After the introduction of chatbots, which can produce automatic reviews superior to the typical average review, I believe that the peer-review system needs a major overhaul on the basis: TEAR —Transparency, Eponymity, Accountability, Responsibility.
Reviewer’s clarification: The references included in this review have the same meaning that references have in scientific documents, i.e., they justify the reviewer’s statements and indicate where further details can be found. They are not intended as suggestions to the author(s) to include them in the paper in review.
2. Open peer review
The system I support is the genuine Open Peer Review, that based on TEAR. In it every transaction is open to the public and eponymous. Authors, reviewers and audience are aware of the identities of all players, who take full responsibility for what they write.
While this should be the meaning of Open Peer Review, only distortions of it have been materialized. For example, in the journals of EGU (European Geosciences Union) , the reviews are accessible by the public and anyone interested can post a comment. However, the formal reviewers appointed by the journals can be (and usually are) anonymous. MDPI (Multidisciplinary Digital Publishing Institute) allows the review material to be posted online, but only if (a) the paper is accepted and (b) the author consents. Again the reviews can be (and usually are) anonymous.
In the past, there was one exception among MDPI’s journals. In 2019 it launched the journal Sci as with an innovative, community-driven, so-called post-publication peer-review system.1 In it a decision for (pre-)publication was made by an editor and then reviewers were invited (or volunteered, as everything was public and transparent). After one or two review rounds, a final decision was made and, if this was positive, the paper would be included in a journal issue. This journal attracted the interest of my coauthor Z.W. Kundzewicz and myself, and we submitted our first hen-or-egg paper there.2
Unfortunately, the acceptance of our paper coincided in time with a major change of Sci’s peer review pattern to the conventional (single-blind) system.3 We have written elsewhere about this experience.4 In brief, we communicated our complaints to the editors and the publisher, explaining that we would not submit our paper in that journal if it was run with the conventional peer-review system. We also expressed our disappointment that a big step of progress was followed by a big step of regression. The publisher (the MDPI owner) replied and we understood his reasoning that there existed a clear conflict: If Sci continued with this system, Clarivate Analytics would not include it in its indices. In fact, Clarivate has invested a lot in anonymity through its system of recognition of anonymous reviews in Web of Science. It is thus explainable that it discourages an innovative progressive system in favour of the traditional, anonymous, transactions, which serve the establishment better.
My search on alternatives, i.e. journals with genuine Open Peer Review, led me to the journal F1000. According to Wikipedia:
F1000 is an open research publisher for academic works. Its model focuses on publishing findings quickly using a post-publication peer-review system. Authors submit an article and all of its underlying data. F1000 does a prepublication check and publishes the article, usually within a couple weeks. After the article is published, an expert is assigned to conduct a peer-review of the work. The peer-review is done publicly, online, and on an ongoing basis. The expert conducting the peer review discloses their name and any vested interests, abandoning the double-blind, anonymous peer-review system that is typical in academic publishing.
However, this idyllic description continues as follows:
Additionally, other organizations like the Bill & Melinda Gates Foundation (platform Gates Open Research) and the European Commission (platform Open Research Europe) contract out the development and support of their own open-access publishing systems to F1000.
I have written elsewhere about Gates and other “philanthropists” of this type. I thought their involvement guarantees censhorship and therefore I had abandoned the idea of submitting any of my research items to F1000.
My experience as a reviewer in F1000
On 2024-12-18, I received an invitation from F1000 to review a manuscript. I accepted, as I have done with numerous invitations from other journals (even though I have recently reduced this activity due to other, non-professional, problems). I was curious to see if my censorship hypothesis would be confirmed. The manuscript that I was invited to review is this:
Analysis of the stationarity and correlation of the global temperature and carbon dioxide time series [version 1; peer review: awaiting peer review]
As can be seen from the site I linked, the manuscript was posted there on 31 Aug 2023. One year and a half later, the journal has failed to secure a single review — a pathetic state of affairs.
I did the invited review, which I copy in the last section of this post, and submitted it in time. However, my review did not appear online. Instead, I received an email asking me to remove the references to my papers that dealt with the very topic of the paper. I refused and explained that these references were necessary to be included in my review.
My two emails copied below, along with their final reply that I “have been marked as declined from [their] system” confirms my censorship hypothesis. (Notice that I have been declined, not only my review).
As seen in my Email #2, I had notified them that I would post my review elsewhere (which I am doing now), and their reply was that I am welcome to do that (but it goes against their policy). Now I am notifying the journal of this post so they can respond in the comments if they wish.
Email #1 On 2024-12-30 11:11, Demetris Koutsoyiannis wrote:
Dear xxx,
I am puzzled by your message...
Haven't you read my report? I guess not...
So here are the relevant extracts:
5. [...] Recent research based on advanced stochastic methodologies (Koutsoyiannis and Kundzewicz, 2020; Koutsoyiannis et al., 2022a,b, 2023; Koutsoyiannis, 2024a,b) has totally been omitted even though this research has tacked the issue that constitutes the subject of this article.
7. [...] Stationarity and nonstationarity are properties of the stochastic processes that are used to model the time series. In other words, they are properties of models, not of the real world (see more details in Koutsoyiannis, 2006; Koutsoyiannis and Montanari, 2015; Koutsoyiannis 2023).
I estimate you invited me to review this paper because I have published research on that subject. It appears that the authors of this paper are not aware of this published research as they don't cite it. Therefore I had to urge the authors to see it.
Isn't it the rule in scientific publishing that a new paper considers and discusses the existing literature? If they missed to do so, is it wrong that a reviewer suggests the authors to do so? If the reviewer has done research on the issue (which is the very reason he is invited to do the review), shouldn't he mention his papers on the subject because they are his?
Difficult to understand your reasoning and your request to "provide more information on why these citations are necessary"...
Kind regards.
Email #2 On 2025-01-23 10:56, Demetris Koutsoyiannis wrote:
Thank you for your late response.
I do not think it addresses my comments, and I consider your practice to be censorship, which is unacceptable.
Please post my review online. It is not ethical to censor scientific opinions and their justification based on references to related literature.
I take full responsibility for what I have written. It is not up to you to decide on an issue for which I am responsible.
If you don't publish my review, I will have to do it on my personal site, because I have been fighting for transparency and against censorship for a long time.
Besides, while in your site you mention COPE, the link you give does not work (see below).
My review
Review of: “Analysis of the stationarity and correlation of the global temperature and carbon dioxide time series” by Upul Rupassara, Sarah Frantsvog, Ashley Holen and Karen Robinson4
Demetris Koutsoyiannis
29 December 2024
1. The article aims to investigate the correlation between global temperature and carbon dioxide levels, also examining issues of (non)stationarity.
2. The article does not present any useful results. As stated in the abstract: “Neither the carbon dioxide time series nor the global temperature time series lag or lead with regard to the cross-correlation function.”
3. The general setting is not that of a scientific article. The statement “In order to preserve Earth’s future, action must be taken now” suggests that the authors’ aim is not of scientific but of activist type. Yet, they do not accomplish this aim as they do not present useful results.
4. The authors use several climate alarmism stereotypes, such as “The rapid and ongoing phenomenon of global warming has negatively impacted both the Earth’s environment and its inhabitants” and “global warming has severely and negatively altered Earth’s environment and its inhabitants”. Again, these are not scientific and are unsupported. Is it within the scope of the article to say whether the effects of global warming are negative or positive? And if it is, what evidence does the article provide?
5. While the authors state that the “correlation between global temperature and carbon dioxide levels has been investigated by many researchers”, the literature review is very thin and quite selective. Recent research based on advanced stochastic methodologies (Koutsoyiannis and Kundzewicz, 2020; Koutsoyiannis et al., 2022a,b, 2023; Koutsoyiannis, 2024a,b) has totally been omitted even though this research has tacked the issue that constitutes the subject of this article. In addition, the body of the article gives references to sites providing general information of alarmist type (e.g. NASA, 2023) rather than sites providing data access.
6. While the data on which the study is based are available on a monthly scale, the authors examine only the annual scale. This is a crucial weakness, as the above referenced research has shown that [CO2] changes lag behind temperature changes by several months—not by several years (unless larger time scales such as decadal are examined; Koutsoyiannis, 2024a). Therefore, the appropriate scale to deal with the problem in question is the monthly.
7. A key point of the article is to “transform a non-stationary time series into a stationary one” (as stated in the Conclusions section within the Abstract). Even though this is regarded as a common approach in literature, it involves several problems. First, a time series is a series of numbers and not a family of stochastic processes, and hence a time series can neither be stationary nor nonstationary. Stationarity and nonstationarity are properties of the stochastic processes that are used to model the time series. In other words, they are properties of models, not of the real world (see more details in Koutsoyiannis, 2006; Koutsoyiannis and Montanari, 2015; Koutsoyiannis 2023). And different models can be devised to model the time series, which can be more or less successful. Second, the authors do not provide an explanation why stationarity is necessary when examining a couple of processes in combination. Nonstationarity may not be a problem per se, as the relationship between the two processes could be stable and valid, even if each of the processes is nonstationary. What causes problems in such cases are the high autocorrelations rather than the nonstationarity. Indeed, high autocorrelations must be decreased by some transformation before any statistical inference be possible. And differencing is indeed a method to reduce the autocorrelation. See details Koutsoyiannis and Kundzewicz (2020) and in Section SI2.2 from the Supplementary Information of Koutsoyiannis et al. (2022b).
8. According to comment 7 above, differencing is a proper method to deal with the subject in question. But differencing must have a meaning. Specifically, the first-order differencing indeed has a meaning, as it represents the temporal changes of the processes in question. For instance, the differenced temperature series quantify the change of temperature in, say, a year. Strangely, the first-order differenced series are not presented in the article at all. What is presented is the second-order differenced series—but this is done without assigning any meaning to it. What do the second-order differenced series represent in physical terms? How could any findings from these be interpreted?
9. The most important finding of the article seems to be this (a couple of lines above Discussion): “According to Figure 11b most dominant cross-correlation for stationary series occurs at l = 0 and l = 1”. In turn, Figure 11b shows the “Cross correlation of global temperature anomaly and carbon dioxide time series from 1960 to 2022: […] Stationary (d = 2)”. This description of this finding is totally insufficient. The “most dominant cross-correlation for stationary series […] at l = 0” just reflects the inappropriate choice of time scale—annual instead of monthly (see point 6 above). Most importantly, the fact that the “peak” at l = 1 is negative is glossed over. What does a negative cross-correlation between temperature and CO2 (as represented by second-order differences) mean?
10. The article has several linguistic imperfections; for example, in the phrases: “conclusion that the impact of CO2 on our atmosphere is trending toward a detrimental conclusion” (conclusion that… conclusion) and “According to a new isothermal analysis by Palmer et al. (2007) in order to produce a more accurate depiction of the underlying warming.” (no verb in sentence).
11. Scientific articles addressed to the international audience should use of the Système International (SI). Fahrenheit, used in Figure 1 and its description, is not an SI unit. (By the way, I did not understand the relevance of that figure.)
References
Koutsoyiannis D: Nonstationarity versus scaling in hydrology. Journal of Hydrology. 2006; 324 (1-4): 239-254 Publisher Full Text
Koutsoyiannis D, Montanari A: Negligent killing of scientific concepts: the stationarity case. Hydrological Sciences Journal. 2015; 60 (7-8): 1174-1183 Publisher Full Text
Koutsoyiannis D, Kundzewicz Z: Atmospheric Temperature and CO2: Hen-Or-Egg Causality?. Sci. 2020; 2 (4). Publisher Full Text
Koutsoyiannis D, Onof C, Christofides A, Kundzewicz Z: Revisiting causality using stochastics: 1. Theory. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2022a; 478 (2261). Publisher Full Text
Koutsoyiannis D, Onof C, Christofidis A, Kundzewicz Z: Revisiting causality using stochastics: 2. Applications. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences. 2022b; 478 (2261). Publisher Full Text
Koutsoyiannis D, Onof C, Kundzewicz Z, Christofides A: On Hens, Eggs, Temperatures and CO2: Causal Links in Earth’s Atmosphere. Sci. 2023; 5 (3). Publisher Full Text
Koutsoyiannis D: Stochastic assessment of temperature-CO2 causal relationship in climate from the Phanerozoic through modern times.Math Biosci Eng. 2024a; 21 (7): 6560-6602 PubMed Abstract | Publisher Full Text
D. Koutsoyiannis, The relationship between atmospheric temperature and carbon dioxide concentration, Science of Climate Change, 4 (3), 39–59, doi:10.53234/scc202412/15, 2024b.
D. Koutsoyiannis, Stochastics of Hydroclimatic Extremes - A Cool Look at Risk, Edition 3, ISBN: 978-618-85370-0-2, 391 pages, doi:10.57713/kallipos-1, Kallipos Open Academic Editions, Athens, 2023.
NASA: Global climate change, Vital signs of the planet. 2023. https://science.nasa.gov/climate-change/.
Replies to the questions of the review form
Is the background of the case’s history and progression described in sufficient detail?: Partly
Is the work clearly and accurately presented and does it cite the current literature?: No
If applicable, is the statistical analysis and its interpretation appropriate?: No
Are all the source data underlying the results available to ensure full reproducibility?: Yes
Are the conclusions drawn adequately supported by the results?: Partly
Is the case presented with sufficient detail to be useful for teaching or other practitioners?: No
My verdict: Approved With Reservations
Competing Interests: No competing interests were disclosed.
Reviewer Expertise: water resources engineering, stochastics, civil engineering, hydrology, climatology.
Reviewer Declaration: I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
Rittman, M. and Vazquez, F., 2019. Sci—An Open Access Journal with Post-Publication Peer Review. Sci, 1, 1, doi: 10.3390/sci1010001.v1
Jacob, C., Rittman, M., Vazquez, F., and Abdin, A.Y., 2019. Evolution of Sci‘s Community-Driven Post-Publication Peer-Review. Sci , 1, 16, doi: 10.3390/sci1010016.v1.
Vazquez, F.; Lin, S.-K.; Jacob, C., 2020. Changing Sci from Post-Publication Peer-Review to Single-Blind Peer-Review. Sci, 2, 82, doi: 10.3390/sci2040082.
Compliments. Well argued backed by quantitative evidence and analyses.
A sick system gets sicker / It's amazing to me that the voice of reason just cannot be understood in today's world. I sure hope your efforts make a difference at some stage.