Category: Scientific Literature

Posted on

What is a Digital Object Identifier (DOI)? | Is it an Indicator of Quality?

A man holding a card that says DOI

What is a Digital Object Identifier (DOI)?Digital

In many articles on this blog we will refer to a “DOI”, which stands for Digital Object Identifier. But what does this mean and what do DOIs do?

 

DOIs are permalinks. That is, they are URLs that do not change even if the location of the files, on the internet, changes. DOIs are most often used for scientific journal articles, although they can be used for other items as well, such as data repositories, technical reports etc.

 

A take home message of this article is that in the context of scientific publishing and predatory publishing a DOI is NOT an indicator of quality. Just because a scientific paper has a DOI, it says nothing about the quality of either the paper or the journal. If a journal is suggesting that, because it has a DOI, it is somehow a validation of its quality, you should question why they are suggesting that this is the case.

 

In the rest of this article, we look in a little more detail at DOI’s.

Why do we need Digital Object Identifiers (DOIs)?

A DOI is a permalink (most commonly) to a scientific paper. This means that the same URL will always “point to” the paper, no matter where it is stored on the internet. Even if the paper’s location on the internet changes, the DOI will remain the same, enabling you to access it using the same URL. That is, you have a permanent link to the article.

 

Indeed, you are unaware where the paper is located, nor do you care. Your only concern is that you can locate it and the permalink, by virtue of its DOI, enables you to do that.

 

If there was no concept of permalinks and the paper used to be accessed using the URL:

http://www.Old-Domain.com/ThePaper

 

… but it has been moved, so that its new URL is

 

http://www.New-Domain.com/ThePaper

 

… you would need to know this and use the new URL, rather than the old one. This can be a pain, especially if you have to update URLs on numerous web sites. It is even more problematic if you are not the only person/organization who gives access to the paper. You may have it in reports, others may have tweeted it, there will also be links to the paper that you are unaware of so you cannot let the relevant person know.

 

If the paper has a DOI of, let’s say, 10.1234/12345.987, you just would use the URL

 

https://dx.doi.org/10.1234/12345.987.

 

This URL would have pointed at the old domain (unbeknown to you), but when the domain was changed, the domain owner would be responsible for making the change, to point the DOI at the new location. You would then be able to use the same URL that you always had (https://dx.doi.org/10.1234/12345.987). That is, you have a permalink to the article that can be used no matter where the paper is located. The owner of the article takes on the responsibility so that the DOI points to the correct place on the internet.

 

To be clear, when an article is moved, those responsible for the article (typically a publisher) has to update the “DOI system” to tell it that article has moved from one location to another, but that is totally transparent to the user. The user still has the option to use the actual URL, but why bother? You may as well use the DOI permalink.

 

Another way of looking at it, is to think of a DOI as a mapping between the DOI and the article. The DOI never changes (it is permanent) but where it points to can change. As long as you know the DOI, where it maps to does not matter. The DOI system takes care of that for you.

 

So, we need DOIs so that once a paper has been allocated a DOI it has a URL that never changes, so that you can easily locate the paper.

The format of a Digital Object Identifier (DOI)

The format of a DOI is defined under the International Organisation for Standardization (more commonly known as ISO). To be precise it is defined under ISO 26324. 

The format of a DOI is available on the DOI web site. In summary, the format is.

  • The first two characters are always ’10’, followed by a period.
  • This is followed by a registrant identifier. This uniquely identifies any entity that has registered with DOI.
  • This first part (’10.’ and the registrant identifier) of the DOI is known as the DOI prefix.
  • Following the DOI prefix is a forward slash.
  • Next comes the DOI suffix,which is a string of characters that is defined by the DOI registrant. It can be of any length and is used to uniquely identify each publication (or other item) that they wish to assign a DOI too.

As an example, the DOI 10.1081/E-ELIS3-120044418:

 

… has the standard prefix if ’10.’, followed by a registrant code of 1081. The rest, the DOI prefix, (E-ELIS3-120044418) has been defined by the registrant.

 

You can see that this DOI links to a book by using this URL – http://dx.doi.org/10.1081/E-ELIS3-120044418.

How to use a Digital Object Identifier (DOI)?

Once you know the DOI of an article, you can access it by using the URL:

 

https://dx.doi.org/DOI

 

… where DOI, is the DOI of the paper you wish to access.

 

For example, if a paper has a DOI of

10.1007/s12109-999-0022-2

 

… then you would type

https://dx.doi.org/10.1007/s12109-999-0022-2

 

… into your browser. This will lead you to the paper, which, in this case, is a paper called “The Digital Object Identifier (DOI)“, which might be of interest to those reading this article.

When you get to the page for the paper, you’ll see that the URL address has changed to

https://link.springer.com/article/10.1007/s12109-999-0022-2.

 

You could have used this URL (if you knew it) but if the publisher, Springer in this case, move the paper’s location, that URL may no longer work, but the DOI URL will always work, as it is a permalink to the article.

Who manages the Digital Object Identifier (DOI) system?

The DOI system is managed by a not for profit organization, called the International DOI Foundation (IDF). They have a web site at https://www.doi.org/, which includes details of the DOI system, how to use it and how to register to use DOIs. If you have an interest in DOIs, this web site is a must visit.

History

The DOI system can be traced back to the 1997 Frankfurt Book Fair, where it was formally announced. In the same year, the International DOI Foundation was also formed to manage this initiative.

 

In 2000, Crossref were the first agency to adopt the system, and start linking electronic articles. Other registration agencies have since been appointed.

 

The syntax of the DOI system was also standardized at this time, and was later formally approved by ISO in 2010.

 

If you want further information about the history of the DOI system, the DOI web site has a section dedicated to this topic.

How do you get a DIgital Object Identifier (DOI)?

To get a DOI (registrant identifier), you need to register with one of the recognized agencies. These are listed at https://www.doi.org/registration_agencies.html. You might also want to take a look at the DOI handbook which talks about DOI agencies.

 

We have no affiliation with any of these, but we know that it can be confusing to know which one to choose. Only because we have had experience with https://www.crossref.org, we suggest that you start there. Crossref was also the first agency to adopt the DOI system in 2000, so they have a good track record.

How do you assign a Digital Object Identifier (DOI) suffix?

Registering with an agency provides you with a DOI registrant identifier. When you use the DOI, you will add your own DOI suffix (that is “/“ followed by a string of characters chosen by you) to make up the URL that users will utilize.

You need to give the DOI suffix some thought and Crossref offers some advice. As an example, it says to be careful of using page numbers in a DOI as you may not know the exact page numbers before you want to assign a DOI to a particular paper.

Is a Digital Object Identifier (DOI) an indication of quality?

As mentioned in the introductory remarks, the fact that a paper has a DOI does not say anything about the quality of the paper or the journal. It is relatively simple to register as a publisher, or a journal to get a DOI prefix, and then start assigning papers to that prefix to enact a permalink.

 

The International DOI Foundation provides a very good service at managing DOIs, but it does nothing, and nor should it, to check the quality, authenticity or validity of the articles that are using a DOI to get a permalink. 

What does this mean for predatory publishing?

In the context of predatory journals and/or publishers, the fact that their articles have a DOI means nothing in terms of the quality of the journal/publisher.

 

If a particular journal/publisher is suggesting that, as their papers have a DOI, this validates the quality of the publisher/journal/paper, this should raise a red flag that the journal may be predatory.

 

It is good for journals to use a Digital Object Identifier (DOI) to provide access to its papers, but if it uses this fact to promote (suggest) its quality, this should be seen as a negative point, rather than a positive.

Article history

Where an article has been updated since first being written, we provide a history of the changes. Why? Why not :-).

  1. The original article was published on 6 January 2020.
  2. The article was updated on 20 June 2023. The main content was not changed but we reformatted it a little (as we now have more experience with the tools we use) and also to try and improve the flow. We also updated some of the H@ headings to assist SEO.
  3. 25 Oct 2024. Based on a response to one of our tweets about this post, it was pointed out that “Document” had been used instead of “Digital”. These oversights were corrected – thank you to the person that notified us of this.
Posted on

Why (and how) would somebody write more than a letter every day to scientific journals?

Image showing four letters to scientific journals and a screen shot from Scopus showing how many letters have been published.

Introduction

We have no evidence that there is anything wrong here, but just want to seek the view of others. Please take a look at the article below and see what you think.

Letters written

So far this year (16 Aug 2024, extracted from Scopus) Wiwanitkit, V. has published 379 ‘documents’. Of those, 345 of them are letters.

That is, in 2024, to date (as at 16 Aug) he has published 1.51 letters each day. By way of comparison, in 2023, he published 414 letters, which is 1.13 letters each day.

Example Letters

By way of an example, the header image shows four of the letters.

These were chosen at random (we just chose the first four that were returned from Scopus). This shows that two of the letters utilize an AI tool and two did not; at least there is no acknowledgement.

Our questions

  1. What is the benefit in writing so many letters?

    Okay, it boosts the number of publications you have but letters are not counted in my research returns. For example, we don’t think that QS     recognizes letters when it looks at how many articles have been published by a given institution.

  2. What is the workflow for publishing letters?

    Look at new articles published each day and write a letter about that article? To us, that is a lot of time/commitment, but we can’t think of another way.

  3. The letters appear to follow a similar format.

    We have not looked at all the letters, indeed, only a small sample, but they follow a similar format. They say, they are responding to the article. highlight one or two ‘flaws’ and suggest some other work that could be done.

  4. AI tools are used – sometimes.

    Two of the letters we looked at (see green highlight) say “The author use language editing computational tool in preparation of the article.“

    We are not quite sure what this means. Does it means that (something like) ChatGPT was used to correct a draft that was written? Or does it mean something else?

    We have previously suggested that an acknowledgement that an Large Language Model (LLM) has been used should be much more detailed (e.g. show the prompt and the generated text in a supplementary file). Moreover, there should be an explicit statement that an LLM was not used.

  5. There is an alternative workflow.

    i) Get a daily (or weekly) list of relevant articles published
    ii) Develop a set of LLM prompts that asks the model to analyze the article and write a letter that gives an introduction, suggests a few ‘flaws’ and suggests further work.
    iii) Wrap it all up in a letter, and send to the relevant journal. We still think that it would take at least an hour to do this, but with a team of people perhaps that is a good investment?

Your thoughts/views

Please let us know what you think about what we say above.

Posted on

The publication record of Sayed Mohamed Eldin

Bubble chart showing how many papers Sayed Mohamed Eldin published in 2022-2024

In a twitter post Nick Wise (aka @nickwizzo) posted about Sayed Mohamed Eldin of Future University in Egypt. We thought we would take a look ourselves.

  1. The author appears to go via several names. This may be a cultural thing and may not be trying to deceive. On the DSDimensions page that Nick refers to, the names given are Sayed Mohamed Eldin, El Sayed Tag Eldin, Elsayed Mohamed Tag-Eldin, Sayed M Eldin, Sayed M El Din, El Sayed M Tag El Din and El Sayed Mohamed Tag-Eldin.
  2. As noted by Nick Wise, Eldin has published quite a number of papers in 2022 and 2023. We looked on Scopus. Searching for his name, like point 1, a number of names were returned but we managed to get an author id (57925959400). The figures we report here are from a Scopus search (‘AU-ID(57925959400)‘).
  3. In 2022, Eldin published 61 papers and in 2023 (so far, our Twitter post was 26 Nov 2024 – updated figures are below) he has published 377 papers. He has not published any papers before that. Note, these are Scopus indexed papers.
  4. A personal comment is that we have never come across anybody who starts publishing in a given year and publishes more than 60 papers.

    In fact, he published a paper every 5.98 days.

    Is it really possible for somebody to publish a peer reviewed scientific paper every six days, whilst adhering to the usual/expected guidelines for authorship?

  5. In 2023, Eldin has published 377 papers. This year, we have had 329 days (up to 25 Nov 2023). This means that Eldin has published more than one paper every day (in fact 1.14 papers every day).

    We ask the same question – Is it really possible for somebody to publish more than one peer reviewed scientific paper every day, whilst adhering to the usual/expected guidelines for authorship?

  6. The top three publishers are Elsevier Ltd. (112), Elsevier B.V. (62) and MDPI (57).

    It would be really interesting if these publishers could show us the “author record” for all the papers where Eldin is an author. There may not be anything wrong, but it would be interesting to look at the data.

    We are in no doubt that the publishers will not give us access to this data.

  7. The top three journals that Eldin has published in are Case Studies in Thermal Engineering (74 papers), Scientific Reports (38 papers) and Journal of Materials Research and Technology (21 papers).
  8. Many (if not all of the papers) are published as #OpenAccess. It would be interesting to see how many of the papers are open access (we may do that analysis, but it is not as easy as, perhaps, it should be).

    But, let’s assume that they were all open access and the average article processing fee (#APC) was $USD 1,000 (this will be a large underestimate).

    This means that Eldin (and his co-authors) have spent $USD 438,000 on the 438 papers he has published.

    We have two questions:

    i) We wonder who actually paid the APCs? Did it come from a single source or were they spread across many different sources?
    ii) We also wonder whether APCs provide the funders a good return on their investment? Would the, approaching, half a million dollars be better spent funding actual research, rather than paying to publish the results of the research?


Updated Stats

Now that we are well into 2024, we thought we would run the same Scopus search again (i.e. ‘AU-ID(57925959400)‘.

 

This showed the following.

 

2024: 24 papers

2023: 378 papers

2022: 61 papers

 

Surprisingly (to us) the 2023 figure have only risen by one. We were expecting the 2023 figure to be much higher, possibly over 400.

For 2024, as of today (07 Apr 2024, at the time of writing), the 24 papers represents publishing a paper ever (98/24)=4.08 days, which is worse than 2023, when he published a paper every (365/378)=0.97 days, but it is early days yet and who knows how many papers are under review.

We plan to keep an eye on it, but you can see his 463 publications here.

Final Comments

We find it amazing that somebody can publish a paper every six days (in 2022), let alone one paper a day (in 2023).


We cannot think of a reason that this is ethically possible. Perhaps there is, but we are struggling. If you can think of a reason, that can be defended, please let us know.

 

The other thing we find amazing is that his publication record is from a standing start. Nothing published prior to 2022 and then published 61 papers in that year. If you think we are being a little harsh, than let us know.

Posted on

Should it take three years to investigate a paper?

Header image for a blog post showing a tweet that says an issue has been outstanding for three years.

Background

“Publishers justify their margins by saying that they provide quality control and act as gatekeepers for science. They should act accordingly. Their role does not end at peer review + upload onto a website. This ought to be retracted. Three years ago.“

This quote is from a tweet from Paolo Crosetto, which refers back to a previous tweet which talked about an issue that has been raised for the past three years, and which has yet to be addressed.

The previous tweet said: “In November 2020, my students and I discovered a completely fake meta-analysis, now cited more than 100 times. I notified Hindawi but, as shown below, they have no intention to act. Each year, on the anniversary of the discovery, I re-post this thread…“

 
The image associated with the tweet can be seen in Figure 1.
Image showing a tweet, which raises an issue going back three years.
Figure 1: Tweet from 'Panteleimon ("Paddy") Ekkekakis'

Our Comments

In this article, we make some observations. We are not looking at the paper itself, and coming to a view whether the paper should be retracted or not. We are not discipline experts so it is not right that we offer this view. Rather, we give some observations after spending some time looking at the facts surrounding the paper.

  1. In November 2021 nine people sent a letter to Hindawi, raising a ‘publications ethics’ concern about a paper published in BioMed Research International. You can see the letter that was sent here (we also provide a copy here, in case you cannot access Twitter(X)).
  2. You can see the article here. We have archived the page here.

    The full citation is:

    Zhenzhen Qiu, Kai Zheng, Haoxiang Zhang, Ji Feng, Lizhi Wang, Hao Zhou, “Physical Exercise and Patients with Chronic Renal Failure: A Meta-Analysis”, BioMed Research International, vol. 2017, Article ID 7191826, 8 pages, 2017. https://doi.org/10.1155/2017/7191826

  3. According to the home page of the article, the article has been viewed 10,670 times and downloaded 3,505 times (as at 20 Dec 2023).
  4. The journal is a member of COPE (Committee on Publication Ethics). We wonder whether they can offer the journal, and its editors, any advice/guidance as to how they should progress this matter, and under what timescale they should operate?
  5. Searching by ISSN (2314-6133 and 2314-6141), the journal is a not a member of DOAJ (Directory of Open Access Journals) (see Figure 2).

    In our tweet, that this article is based on, we did find the journal indexed by DOAJ, but there was some ambiguity.

    In response to our tweet, DOAJ said:

    “DOAJ removed BioMed Research International on 23rd March 2023. The journal you highlight is an older, discontinued version and not the same publication. It should have also been removed at the same time. This was an oversight on our part and has now been done.“

    We thank DOAJ for their quick action and for responding to our tweet.

  6. The journal is indexed by Scopus. It has a 2022 CiteScore of 5.3 It is listed in two categories – Q1 in one and Q2 in the other. See Figure 3.
  7. The journal is indexed by Clarivate. It has a 2021 impact factor of 3.246. It appears in four categories, all Q3. See Figure 4.
  8. On Google Scholar, this article has been cited 109 times (as at 20 Dec 2023). See Figure 5.
  9. On Scopus, the article has 51 citations (as at 20 Dec 2023).
  10. On Clarivate (Web of Science), the article has been cited 29 times (as at 20 Dec 2023).
  11. If this article (although this is a more general question) is retracted, what should happen to all the papers that cited it? Should they all be reviewed again?
  12. There is a letter to the editor about this paper, which is available via the Hindawi page which hosts the journal. The letter has some comments with respect to the procedures and results of the study. We have archived the page.
  13. A PubPeer comment has been made.
  14. The current APC for this journal is US$2,550 (page archived here).
  15. In 2016 (which is when the paper would have been submitted) the APC was US$2,000.

    We wonder what happens to the APC if the article is retracted. Presumably, the journal has done the work that it was ‘paid’ to do, so they would just keep it?

  16. Hindawi states their mission as:

    “By placing the research community at the heart of everything we do, we strive for a future where researchers are motivated to work together, empowered with the tools and services they need to do so, and freed from any barriers that stand in their way. We aim to maximize the impact of scientific research through openness and global collaboration as we truly believe that science works best when research is open.“

    Are they living up to this mission statement (especially about openness and collaboration) in the case we are looking at here?

    We have archived the page with Hindawi’s mission statement on.

  17. On the information for authors page for the journal it says:

    “High editorial standards, ensuring all published manuscripts undergo an in-depth peer review process.“

    Does that peer review stop once the paper is published or, if a concern is raised should it not be investigated in a timely manner?

    The information for authors page has been archived.

  18. On its publications ethics page it says:

    “Hindawi is a member of the Committee on Publication Ethics (COPE). Read the COPE Ethical Guidelines for Peer Reviewers for information on best practice in peer review.

    Reviewers should raise any concerns about publication ethics to the Research Integrity team.“

    We have archived this page.

  19. We believe that the issue discussed in this paper has been raised with the Research Integrity team but it appears that little (if any) action has been taken.

Finally

Surely, the publisher should have completed their investigation within three years and either retracted the paper or told the person who reported the case why that no action will be taken?

Figures

Screenshot from DOAJ web site showing that two ISSN's were not found in its database
Figure 2: Search on DOAJ showing that ISSN's 2314-6133 and 2314-6141 were not located in its database.
Showing the Scopus CiteScore (5.3) of BioMed Research International
Figure 3: The Scopus CiteScore (5.3) of BioMed Research International
Showing Web of Science's impact factor (3.246) for BioMed Research International
Figure 4: Web of Science's (Clarivate) Journal Impact Factor (3.246) for BioMed Research International
Google Scholar screenshot showing that 'Physical Exercise and Patients with Chronic Renal Failure: A Meta-Analysis' has been cited 109 times
Figure 5: Google Scholar screenshot showing that 'Physical Exercise and Patients with Chronic Renal Failure: A Meta-Analysis' has been cited 109 times
Posted on

How big is science’s fake-paper problem?

Graph showing the rise in paper mill papers

In a recent X (tweet) we mentioned a paper from Richard von Noorden, who was writing in Nature. You can see the paper here.

The paper is well worth a read, and we would encourage you to do so but, but here are the main points that we took away.

  1. Over the past two decades, more than 400,000 research articles have been published that show strong textual similarities to known studies produced by paper mills. Around 70,000 of these papers were published last year (2022). It is estimated that 1.5–2% of all 2022 scientific papers could be from paper mills. 
  2. ClearSkiesAdam says that it is impossible to know whether all of the papers he identified are from paper mills but the few percent is a reasonable, conservative estimate.
  3. MicrobiomDigest says that ClearSkiesAdam‘s estimate, ‘although staggeringly high, is not impossible‘, adding that it would be useful to see the full details of Day’s methods and examples. 
  4. JAByrneSci says ‘Sadly, I find these estimates to be plausible‘.
  5. Day (ClearSkiesAdam) believes that his estimate is a lower bound, as his method will miss paper mills that avoid known templates. Paper mills target particular titles, rather than there being an even distribution across all journals. Day does not reveal the publishers which appear to be the most affected.
  6. In June 2022, C0PE (Committee on Publication Ethics) said that 2% of submitted papers could come from papers mills for most journals and could be as high as 40% for some journals.
  7. Bernhard Sabel posted a preprint in May 2023 suggesting that a paper with an author who is affiliated with a hospital, and gives a non-academic email address, should be flagged as a paper that could be from a paper mill. Sabel estimates that 20–30% of 2020 papers in medicine and neuroscience were possibly from paper mills, later reducing this estimate to 11%. He acknowledges that his method would flag false positives. 
  8. Retraction Watch records fewer than 3,000 retractions related to paper mills, out of the 44,000 papers that it has retracted.
  9. ivanoransky (co-founder of Retraction Watch) says the figure would be higher as some publishers avoid the term ‘paper mill’ when explaining the reason for retraction.

Whatever you take away from this article as the important points, the key takeaway is that paper mills are a very big concern and are undermining the integrity of the scientific archive (there is a strong argument that the damage has already been done).

 

We let Day have the final word. The final part of the article reads:

 

Those retraction numbers are “only a small fraction of the lowest estimates we have for the scale of the problem at this stage”, says Day. “Paper-mill producers must feel pretty safe.”

You can see our tweet, on which this article is based here.

Posted on

Elsevier changed a published paper without any explanation

Pub Peer screenshot showing an Elsevier paper that was changed

Introduction

In an X (tweet) Guillaume Cabanac raised an issue that Elsevier had published a paper and then updated the published version of the paper. Elsevier did not explain why the paper had been changed and they did not do the update in a way that most people would expect.

Elsevier ignored that the authors had used an AI tool when writing their paper, yet failed to acknowledge it. This goes against the publishers guidelines.

Further details are below.

"Regenerate Response"

The paper contained the phrase “Regenerate response“, which is a standard (re-)prompt to AI chatbots (such as ChatGPT).

If the authors copied AI generated text and failed to delete their prompt to regenerate some of the text, it is evidence that they had used a generative AI tool. This is subject to context, as some papers will use the term legitimately, for example, when carrying out research into this aspect of generative AI.

In this case, the authors had used an AI tool (by their own admission), but did not acknowledge it in the submission to the journal.

The paper was published, (with the “regenerate response” pharse in it – see the header image of this article), but the paper was changed to remove that phrase, without explaining why the paper had been updated.

PubPeer

The change in the paper was noticed and was raised on the PubPeer platform. You can see the PubPeer comment here.

Once this was brought to the attention of the journal, they simply updated the paper.

 

Updating a scientific paper

Once a peer reviewed, scientific article has been published, it should not be changed. The correct way to address errors is to issue an erratum, in a future issue of the journal.

This was not done in this case. This means that different people could be looking at different versions of the paper, which should never happen with articles that are in the scientific archive.

It is an important principle that anybody with access to a peer reviewed, scientific paper is reading the same version of that paper as everybody else. This is the reason why you should issue an erratum if an error is found.

Our comments/thoughts

  1. Why did Elsevier just change the paper, without any explanation? They should have done this via a corrigendum or an erratum, surely?
  2. The fact the authors used an #LLM and did not acknowledge it goes against the publishers guidelines. Should this be an automatic retraction?
  3. The publishers have removed the “regenerate response” but there is still no ackowledgement in the paper, from the authors, that an AI tool has been used. Therefore, they are still in breach of the publishers guidelines.
  4. Perhaps the publishers were embarassed that this had slipped through the peer review net and they thought that the best way to deal with it was to simply update the paper, surely “nobody would notice?“. Actually they did.
  5. Perhaps the publishers thought that it was just a minor change and it was not a big issue to update the paper? The change may have been minor, but (in our view) you should still not change a paper that has been published.
  6. What sort of message does this send when one of our leading publishers appears to be complicit in unethical publishing. If legitimate publishers are acting unethically, what hope is there for the integrity of the scientific archive?
  7. On their author guidelines (archived here) Elsevier says:

    “Authors should disclose in their manuscript the use of AI and AI-assisted technologies and a statement will appear in the published work. Declaring the use of these technologies supports transparency and trust between authors, readers, reviewers, editors, and contributors and facilitates compliance with the terms of use of the relevant tool or technology.“

  8. They also state (you need to click one of the links (i.e. “In which section of the manuscript should authors disclose the use of AI-assisted technologies, and where will this statement appear in the chapter or work?“) at the bottom of the page to see this text).

    “We ask authors who have used AI or AI-assisted tools to insert a statement at the end of their manuscript, immediately above the references, entitled ‘Declaration of Generative AI and AI-assisted technologies in the writing process’. In that statement, we ask authors to specify the tool that was used and the reason for using the tool. We suggest that authors follow this format when preparing their statement:

    During the preparation of this work the author(s) used [NAME TOOL / SERVICE] in order to [REASON]. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.“

  9. As Guillaume Cabanac (@gcabanac) points out in his PubPeer comments, the authors did not acknowledge that they used an #LLM, and still don’t.
  10. The fact that Elsevier acknowledged that an #LLM was used, means that they also accept that the authors went against the guidelines they specify. Surely, by simply changing the article raises a lot of concerns.

===

 

You can see the original tweet from Guillaume Cabanac here. Our tweet, in response and on which this article is based, can be seen here.

Posted on

Most impactful journal on Predatory Publishing?

Graph showing the five journals that have published the most papers on predatory publishing and the h-inex of those articles

What journal has the most impact for its papers on Predatory Publishing?

The journal that has published most papers is Learned Publishing, but the one that appears to be having the most impact is the Journal of Korean Medical Science, as it has published eight papers and has an impact factor of seven.

We had a quick good look at Scopus, looking for articles on Predatory Publishing.

The graph shows the journals that have published most papers, along with the h-index for those articles.

Learned Publishing

Learned Publishing has published the most articles (15), and they have an h-index of 8 (meaning that 8 of those articles have been cited 8 or more times).

Journal of Korean Medical Science

We were particularly impressed with the Journal of Korean Medical Science. This journal has published 8 papers on predatory publishing, with 7 of those papers contributing to its h-index.
… and 4 in the liist below only needs one more citation to give these papers an h-index of 8 and have all papers contributing to the h-index.

If you are interested, here are the 8 papers published by the Journal of Korean Medical Science.

 

  1. Predatory publishing is a threat to non-mainstream science (2017), 32 (5), pp. 713 – 717, Cited 23 times. DOI: dx.doi.org/10.3346/jkms.2
  2. Statement on publication ethics for editors and publishers (2016), 31 (9), pp. 1351 – 1354, Cited 25 times. DOI: dx.doi.org/10.3346/jkms.2

  3. Publishing ethics and predatory practices: A dilemma for all stakeholders of science communication (2015), 30 (8), pp. 1010 – 1016, Cited 44 times. DOI: dx.doi.org/10.3346/jkms.2

  4. Predatory publishing practices corrode the credibility of science, (2015), 30 (10), pp. 1535 – 1536, Cited 7 times. DOI: https://dx.doi.org/10.3346/jkms.2015.30.10.1535

  5. Improving scientific writing skills and publishing capacity by developing university-based editing system and writing programs (2019), 34 (1), art. no. e9, Cited 12 times. DOI: dx.doi.org/10.3346/jkms.2

  6. The pressure to publish more and the scope of predatory publishing activities (2016), 31 (12), pp. 1874 – 1878, Cited 56 times. DOI: dx.doi.org/10.3346/jkms.2

  7. Manuscript submission invitations from ‘predatory journals’: What should authors do? (2017), 32 (5), pp. 709 – 712, Cited 19 times. DOI: dx.doi.org/10.3346/jkms.2

  8. Dangerous predatory publishers threaten medical research (2016), 31 (10), pp. 1511 – 1513, Cited 82 times. DOI: dx.doi.org/10.3346/jkms.2

Metadata

Scopus Search Term: ‘TITLE-ABS-KEY ( “predatory publish*” )’

Date of search: 14 Sep 2023

Posted on

Detecting AI generated papers: Two recent articles

Two papers that address the topic of detecting AI generated text
Introduction
As well as using generative AI (Artificial Intelligence) to help write papers, there is also active research which looks at ways to detect papers that have been written using AI tools. This article highlights two such papers.
 
ChaptGPT and Large Language Models (LLM)
ChatGPT, and other LLM tools, are already having a massive impact on the world at large and the scientific community is, if anything, more affected than many other sectors.
 
Detecting papers written with AI Tools
There is no doubt that AI tools can be of benefit to those writing scientific papers, but there is a growing need to detect when these tools have been used.
 
Why? There are many reasons, but one is being able to check papers when authors have not declared when they have used Generative AI to assist in their paper.
 
Many journals ask that the use of AI is declared. Like plagiarism tools, it would be useful to have a tool to check whether the paper has, even partly, been written using one of the multitude of AI tools that are now available.
But, detecting AI generated text is not an easy task, which has given rise to research on this topic.

 

 
Two recent papers
These two papers, are just a couple of recent examples of papers that is addressing this topic.
 
1) Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text (DOI: https://doi.org/10.1007/s40979-023-00140-5)
 
Abstract: The proliferation of artificial intelligence (AI)-generated content, particularly from models like ChatGPT, presents potential challenges to academic integrity and raises concerns about plagiarism. This study investigates the  capabilities of various AI content detection tools in discerning human and AI-authored content. Fifteen paragraphs each from ChatGPT Models 3.5 and 4 on the  topic of cooling towers in the engineering process and five human-witten control  responses were generated for evaluation. AI content detection tools developed by  OpenAI, Writer, Copyleaks, GPTZero, and CrossPlag were used to evaluate these  paragraphs. Findings reveal that the AI detection tools were more accurate in  identifying content generated by GPT 3.5 than GPT 4. However, when applied to human-written control responses, the tools exhibited inconsistencies, producing false positives and uncertain classifications. This study underscores the need for further development and refinement of AI content detection tools as AI-generated content becomes more sophisticated and harder to distinguish from human-written text.
 
 
2) (Mis‑)Classification of 17,721 Journals by an Artificial Intelligence Predatory Journal Detector (DOI: https://doi.org/10.1007/s12109-023-09956-y)
 
Abstract: The SCImago Journal Rank (SJR) ranks journals into four quartiles (Q1–Q4). SJR
serves as a safelist for journal selection, when trying to avoid predatory journals, as journals that have been indexed by SJR are seen as having stringent publishing standards. An AI-based tool, the Academic Journal Predatory Checking System (AJPC), claims to be able to differentiate suspected predatory journals from normal journals. In this study, we asked (25 March 2023) AJPC to classify the top 2500 journals (based on SJR ranking). We found that 65.64% of these journals were classified as “suspected predatory”, 33.28% were classified as “normal”, while 1.08% of the journals could not be classified. A follow-up run (30 March 2023) returned  different results (89.20%, 10.16% and 0.64%). This set of results is worrying, not only because of the number of SJR journals that are thought to be “predatory”, but also because of the difference in results when using the same tool a matter of days apart. We extended our analysis and looked at seven stand-alone publishers (Elsevier, Frontiers, MDPI, OMICS, Springer Nature (incl. Nature Portfolio), Taylor & Francis and Wiley). In total, we asked AJPC to classify 17,721 journals. The results confirm that we should be concerned about the classifications provided by AJPC. For example, 100% (4756) of Elsevier journals were classified as “suspected predatory”. Even Springer Nature (which includes the journal that published the AJPC paper) has journals which were classified at “suspected predatory”. We thus urge caution against relying on AJPC at present. We also urge the AJPC’s authors to provide an explanation as to why a journal is classified in a particular way, as part of their user interface. We are willing to conduct further analyses should AJPC be revised and updated as it could provide an important service to the scholarly community.
 
Finally
If you are interested in this area of research, these two papers might be good starting points, especially if you also access the papers that they cite.
 
 
Posted on

Selling of authorships: Elsevier please investigate

Image showing two papers that were selling authorships and subsequently published

This article was originally posted as a tweet, but we thought it was worth posting on this blog, in the hope that it will reach even more people.

Selling of authorships

It is easy to find adverts which are offering authorships for sale. What is a little unusual is to find the advert with the title of the paper and the journal where the article will be published, … and then to see the article in print. The header image shows two examples of this.

Journal of Energy Storage

The Journal of Energy Storage, an Elsevier journal, should (in our view) look into this issue, as they seem to have published two papers where the authors paid to be an author on the paper.

That is, an advert for the sale of authorships, for two papers, and the subsequent publication of those papers with the same exact titles in the Journal of Energy Storage.

We hope that the EiC (Luisa F. Cabeza) is aware of this and will take the appropriate action.

 

The two articles can be seen here:

  1. https://doi.org/10.1016/j.est.2022.105311
  2. https://doi.org/10.1016/j.est.2022.106166

 

Pubpeer

We note that these two articles have been mentioned on PubPeer. You can see the PubPeer entries here:

  1. Optimal economic scheduling of microgrids considering renewable energy sources based on energy hub model using demand response and improved water wave optimization algorithm
  2. Optimal dispatch of dynamic power and heat considering load management, water pump system, and renewable resources by grasshopper optimization algorithm

 

Our previous experience

In one of our previous lives, we have been EiC of a journal and if something like this happened (nothing like this happened, but things did), the EiC convened an independent committee to look at the issue and provide a recommendation.

Not saying this is what should happen here (every publisher will have slightly different SOP’s) but the fact that there was somebody offering authorships for a paper of this title and the fact that a paper appeared with this title must be enough to start an investigation? Indeed, such an investigation may have already started.

 

No assumption of guilt

We are not saying that the authors are guilty. Perhaps:

  • They are totally innocent and this can be explained
  • The authors have been duped and are, at best, naive
  • One of the authors coerced (at least) one of the other authors
  • … or some other explanation

 

Access to information

The journal will have access to things like:

  • The peer review reports
  • When authors were added/changed, and at one stage of the publication
  • Acceptance from the author that they participated in a way that warrants being an author
  • Access to the corresponding author

It is also possible to look at the history of the authors and, if the authors have published in other Elsevier journals, then they should be able to look at the history of those papers just to see if there are any worrying patterns.

 

Could we investigate ourselves?

We could investigate ourselves, but we don’t have the same access to information that the publisher/journal does so, we hope, that they will take the initiative.

 

Finally

Everything may be above board and the journal, EiC, authors etc. have nothing to answer here, but it does need looking at. If nothing else to protect the reputation of the publisher, the journal and the EiC. And also to take appropriate action should that be necessary.

 

Tweets which motivated this article

You can see the original Twitter posts from, @author_for_sale, on which this post is based here:

  1. https://twitter.com/author_for_sale/status/1697298062685642877
  2. https://twitter.com/author_for_sale/status/1697297679678628291
 
Posted on

1,000 ChatGPT papers have been published this year, so far

Image of the most highly cited ChatGPT paper as an 22 Aug 2023

Almost 1,000 papers have been published THIS YEAR that have ChatGPT in the title.  As at 22 August 2023, the actual number of papers published is 985.

Since ChatGPT came to the general public’s attention in November 2022, it has been subject to scrutiny from almost all walks of life. The research community has not been slow to investigate this AI tool. 

 

Looking at Scopus and ONLY searching for ChaptGPT in article titles (so NOT looking at the abstracts and keywords), the search returned 985 papers. Moreover, only three papers were published in 2022, so 982 papers have been published this year (and counting – this data was extracted on 22 Aug 2023).

 

Figure 1 shows the top five papers, with regard to the number of times they have been cited. As can be seen, they are receiving a lot of attention. It is interesting to note that four of the papers were published by Nature

and the other paper (the top cited paper) was published in Science.

The top five ChatGPT papers in 2023 (as at 22 Aug 2023)
Figure 1: Top five ChatGPT papers in 2023, as at 22 Aug 2023

The top five papers

Here are links to the five papers listed above.

  1. ChatGPT is fun, but not an author: https://dx.doi.org/10.1126/science.adg7879
  2. ChatGPT: five priorities for research: https://dx.doi.org/10.1038/d41586-023-00288-7
  3. ChatGPT listed as author on research papers: many scientists disapprove: https://dx.doi.org/10.1038/d41586-023-00107-z
  4. Tools such as ChatGPT threaten transparent science; here are our ground rules for their use: https://dx.doi.org/10.1038/d41586-023-00191-1
  5. Abstracts written by ChatGPT fool scientists: https://dx.doi.org/10.1038/d41586-023-00056-7

Do you want to see all the papers?

We are testing a new service, where you can request a full list of the ChaptGPT papers that have been published in 2023. If you would like to receive this list, please go to: predatory-publishing.com/request-more-i

 

… we have to say that this is the first time we have done this, so please bear with us and we apologise if there are teething problems.