Some journals, especially those which do not have a Scopus or an ISI Web of Science (aka Clarivates) impact factor, will often quote other impact factors. Some of these are not to be trusted but Google Scholar is a trusted name so if the journal quotes a Google Scholar impact factor, can we trust the data provided? The answer is proceed with caution.
For any impact factor, you should be able to verify (i.e. reproduce) it. With Google Scholar, this is possible but can be time consuming and it also requires a certain level of knowledge that many may not posses.
In this article, we look at how to use Google Scholar and how you would access the data to enable you to calculate an impact factor, so that you can verify statements made by a given journal. But first we look at the general issues of why other impact factors might not be everything they claim to be.
What is wrong with quoting other impact factors?
There is nothing wrong with journals using other impact factors but first let’s be clear what we mean by other.
In the context of this article, “other” means anything apart from Scopus or IS Web of Science (aka Clarivates). We specifically mention these two as they are the two that most journals want to be associated with. Some journals will even say they are indexed by these bibliographic indexes, even though they are not.
Let’s assume that the journals are being honest, as least with respect to not misrepresenting their association with Scopus and ISI. However they would still like to advertise some sort of impact factor, to give the impression that they are running a high quality journal and that the articles it publishes are being cited. And, indeed that might be the case but we should ask ourselves two questions.
- Is the impact factor that is being quoted an impact factor that you can have confidence in?
- Can you verify the value of the impact factor being stated?
To address these two issues, we make the following observations.
Some impact factors are just like predatory journals, in that they exist to provide an impact factor that the journal can use, without really have any strong basis for the value they quote. Even if they outline their process, some of it might be subjective so that reproducability is virtually impossible.
One of our previous articles , “When is ISI not ISI“, looked at one of these services that we believe gives the impression that a journal is indexed by ISI Web of Science when, in fact, this is not the case.
The other thing that you need to be able to do is to be able to verify the impact factor you are being quoted. This requires 1) you know where the data comes from and you can access it, importantly, in a way that does not rely on the journal just providing you with figures and 2) you need to know how to calculate the impact factor.
You will be surprised how often these two pieces of information are missing, in which case, you should question whether the impact factor is a valid measure.
Predatory impact factors (for want of a better phrase) is not something we have really focused, as part of blog or our Twitter account, but it is really an area that we should give more attention to, and we will look at more of these in future blogs. Let us know if there are any ones you would particularly like us to look at?
Why are we looking at Google Scholar?
We focus on Google Scholar as many journals/publishers state that they are indexed on Google Scholar and then state an impact factor that they have calculated. We would like to be able to verify that impact factor. Alternately, a journal may give an impact factor, with no information about where it has been derived from. In these cases it would be nice to know if this has been calculated from Google Scholar, by comparing our calculated figure with theirs.
It is understandable that journals want to associate themselves with Google Scholar. It is a trusted name on the internet and the Google Scholar service is free, so both journals/publishes and the research community (indeed the general public) can access all of this information free of charge.
One of the issues with Google Scholar is validating the impact factor that a journal presents you with. Let’s say that a journal says that it has a Google Scholar impact factor of 7.429, how do you know whether this is a valid figure, or something they have just made up, knowing that 7.429 will sound like an impressive number to most people, probably higher that what would be expected from Scopus and ISI’s Web of Science.
In general, you would expect a Google Scholar impact factor to be higher than Scopus/ISI as they will consider citations from a wider variety of sources that Scopus/ISI, so we should not be too worried about the magnitude of the number, but we should be able to verify it.
However, this is quite difficult on Google Scholar (unless anybody knows differently) as we discuss below.
Two ways a journal can be found on Google Scholar
There are two ways, that we are aware of, which enables a journal to have a Google Scholar presence. One way is where the journal has set up a specific account. A second way is when the journal does not have an account, but you can still search for the journal using Google Scholar’s advanced search feature.
We look at both these options below.
When a journal has a Google Scholar account
In the same way that individual scholars can have a Google Scholar account, a journal can also register for an account. This makes it easy to find the journal and its publications.
To use as an example, we chose the journal “International Journal of Computer Science and Information Technologies“. This is a journal that we have previously tweeted about.
This journal has its own Google Scholar account, which you can see by following this link.
Figure 1 shows the account. You can see that the name of the journal appears at the top of the screen, where you would normally see the name of a scholar.
We have also showed the user name (this does not appear on the page, but we show it for information). If you know the username, you can go directly to their Google Scholar page using the URL:
https://scholar.google.com/citations?user=gcY_JeAAAAAJ
… you’ll see that that last part of the URL is the username.
When a journal does not have a Google Scholar account
Many journals have a Google Scholar account, but many do not; the majority in our experience, although we have no real data to support that statement – it is just our experience.
However, there is still a way that you can search for articles that a journal has published, even though it does not have Google Scholar account.
How to use Google Scholar Advanced Search
Google Scholar has an advanced search function that enables you to look for articles published by a specific journal, even if that journal does not have an explicit Google Scholar account.
Figure 2 shows you how to use Google Scholar’s Advance Search functionality.
- If you go to the Google Scholar home page, at the top level is a drop down menu (see 1, in Figure 2).
- The drop down menu has an entry called “Advanced Search” (see 2, in Figure 2).
- Accessing the Advanced Search menu leads to the form shown in 3, in Figure 2. Simply enter the journal name in the “Return articles published in” field, and click on the search icon.
- This will return the articles that have been published by that journal (see 4, in Figure 3).
In the example in Figure 2, we have used the same journal (International Journal of Computer Science and Information Technologies) as before just to demonstrate how it works, but the journal does not need to have a Google Scholar account.
What can you Google Scholar tell you?
You can find out the papers a journal has published (according to Google Scholar) in at least two ways and the way you choose depends whether the journal has an explicit Google Scholar.
We have shown examples of these above.
Armed with these search results, what else can we find out?
The two things that many people are interested in are the h-index and the impact factor.
Google Scholar h-index
For a journal (or person) that has a Google Scholar account, Google will automatically calculate the h-index of that journal or person.
We are planning an article that goes into more details about the h-index, but you may want to look at the wikipedia entry in the meantime.
Essentially though, the h-index is “the maximum value of h such that the given author/journal has published at least h papers that have each been cited at least h times”.
If you look at Figure 3, it shows that the journal we looked at in Figure 1 has an h-index of 28.
This means that 28 papers have been cited 28 times or more. You will not be able to find 29 papers that have been cited 29 times, or more as that would give the journal an h-index of 29.
If a journal does not have a Google Scholar account, so you can only search for the journal as shown in Figure 2, the the h-index is not available through Google Scholar. You could calculate it manually by looking at every paper that the journal has published, seeing how many times it has been cited and then carrying out the necessary calculations. The idea behind this process is not that difficult but it will be time consuming if there are a large number of articles, unless you can automate it in some way – which we discuss below, when looking at the impact factor.
Google Scholar impact factor
The Google Scholar impact factor is often quoted by a journal, but is there a way that we can validate it? Unfortunately, the answer is no, at least not easily. The impact factor is not something that Google Scholar displays so we need another way of calculating it ourselves.
However, that is easier said that done. You really need to download all the data so that you can carry out the analysis, using a spreadsheet, or some other suitable tool. Downloading the data though is not easy, as Google Scholar does not provide this functionality (i.e. there is no export option), so you would have to rely on an external tool. Even those seem few and far between, but you may want to take a look at this link, which discusses this topic.
If you want/need to do it manually, you would need to copy/paste each paper, ensuring the collect the important information such the paper title, how many times it has been cited and the year it was published, as this information is important when calculating an impact factor.
Once you have the data, you then need to carry out the analysis. At this stage you still have key decisions to make, such as the number of years that the analysis will cover and the exact formula for the calculation. We will look at this in a later article.
Conclusion
We would like to leave you with better news, but calculating (or verifying) a Google Scholar impact factor is not easy. This is problematical as you may not want to verify the figure that is being provided by an external source, such as a journal that is quoting a specific impact factor.
We are currently developing a tool that enables us to do this. That is, if we provide a Google Scholar identifier, it will return various statistics, which will include the impact factor.
