Disclosure: This page may contain affiliate links. This means, at no additional cost to you, we receive a commission if you click through and make a purchase.
We look at the African Quality Centre for Journals (AQCJ) and consider whether it can be used as a reliable way to measure the quality of a journal. Our conclusion is that you cannot. There are several issues with AQCJ which we explore in this article.
Most people will have heard of impact factors, which measures the number of times that the papers in a journal has been cited. It is trying to measure how much impact the journal is having on the basis that if other scholars are citing papers from the journal, those papers are more useful than those that receive fewer citations.
Web of Science and Scopus are probably the most trusted impact factors and when people refer to an impact factor, they are likely, without any other context, referring to one of these, most likely Web of Science, more commonly referred to as ISI (Institute for Scientific Information) but is now more correctly called the Clarivate Analytics Impact Factor.
How many journals are indexed?
AQCJ has indexed 985 journals, split into five categories (see Table 1).
| Category | # |
|---|---|
| Engineering | 241 |
| Humanities and Social Science | 188 |
| Medical and Health | 133 |
| Pharmacy | 208 |
| Miscellaneous | 215 |
| TOTAL | 985 |
Motivation for AQCJ
The home page of AQCJ says “The academic community has long been demanding more transparency, choice and accuracy in journal assessment. Currently, the majority of academic output is evaluated based on a single ranking of journal impact. African Quality centre for Journals (AQCJ) perform this job as precisely as possible.”
We would question the statement that the academic community has long been demanding more transparency, or at least we are not aware of this. We would be happy to accept this if AQCJ could provide something to support this statement.
We agree that the majority, which we take to mean Web of Science andor Scopus, of academic output is evaluated based on a single ranking of a journal impact. What is not mentioned is that to get listed by Web of Science or Scopus requires meeting many of the criteria that are also listed by AQCJ, for example the frequency of publication and the quality of the editorial board. As far as we are aware, Web of Science and Scopus do have one measure (impact factor, measured by citations), but to get to the point where you can be listed by Web of Science and/or Scopus there are significant hurdles to overcome.
We are happy to note that AQCJ state that they will perform this as precisely as possible, which we take to mean that the evaluation methodology will be both exacting and transparent. However, as you’ll see below, this is not really done.
Evaluation Methodology
Below is the evaluation methodology as shown of the AQCJ web site:
Citation: The impact factor for a journal is calculated based on a three-year period, and can be considered to be the average number of times published papers are cited up to two years after publication.
Originality: AQCJ checks random selects published article’s originality and quality. Only citation is not perfect way of Impact factor calculation.
Time publication: Periodicity of publication should be uniform. If it is not uniform, the quality of particular publication cannot impressible.
Geographical coverage: Only particular small area based publication cannot get good marks as it is not covering all around world research.
Editorial Quality: Editor Board of particular Journal gives the direction to any Journal. So it must be good and considerable for evaluation.
As measures of quality, using citations, originality, timing of publication, geographical coverage and editorial quality are laudable. It is how they are measured, the weighting given to each term and whether it can be understood and reproduced by the wider community that is important.
However, we cannot find anything on the web site that says how these factors feed into the overall impact factor that is assigned to a given journal. There are, for example, no worked examples, no formulas, nothing to say how each of the criteria is weighted, nothing to say how subjective views (such as the global reach, quality of the editorial board are converted into a number that makes up the final impact factor.
As an example, the top-rated Engineering journal, IEEE Transactions on Pattern Analysis and Machine Intelligence has an AQCJ impact factor of 12.685. We were unable to find how this figure is calculated. We just have to to accept that this is a valid impact factor, which has been calculated using the five criteria given above. Given that the impact factor is given to three decimal places, suggests that the calculation is quite detailed, which would lend itself to being described in a little more detail.
By comparison, if you look at the same journal on Web of Science, looking at the Journal Citation Report. It is explicitly stated how the impact factor is calculated. Figure 1 shows how this calculation is made.
It is difficult to see how the impact factors assigned to journals by AQCJ can be reproduced and we have to take the figures they provide as a matter of faith.
Attention to detail
As we were writing this article, we noticed a few things that suggests that ACQJ has not really paid the attention to detail that we would expect from an organisation that says on its home page that it will “perform this job as precisely as possible”. Here we just a few examples.
English
The English, as written, is understandable, but there are many areas where it could be improved, as well as some parts which are not totally clear, with us putting our own interpretation on it. As an example, in the evaluation methodology it says “Only citation is not perfect way of Impact factor calculation”. This is understandable, but it could be phrased a lot more clearly. In the originality section of the evaluation methodology (see above) it says “AQCJ checks random selects published article’s originality and quality”. Not only is this difficult to understand, but it is also opaque in its meaning.
We also note that sometimes ‘centre’ in African Quality Centre for Journals is capitalized, and sometimes not. Only a minor detail but again it brings into question the attention to detail and the academic rigor that underpins AQCJ.
Table Headings
If you look at Figure 2, you will see that one of the table headings misspells Factor (as Fcator). This might seem to be nit picking, but it is this attention to detail, and rigor, that we except, and deserve, in the scientific community as it casts doubt over other aspects of the operation.
Repetitive entries
Looking at Figure 2 we note that same journal (IEEE Transactions on Control Systems Technology) has been shown twice, once at position 25 and once at position 29. Moreover, they have different impact factors.
This really does raise a red flag about the robustness of the evaluation methodology. It is difficult to understand why a journal has been repeated but even if we take this is as a genuine mistake, how does the same journal end up with a different impact factor, especially when AQCJ claims that it “performs this job as precisely as possible”. It makes you wonder what checks and balances are in place to allow this sort of error to get all the way through to the web site.
Incorrect ISSNs
We have not been able to check all of the 985 journals indexed by AQCJ. We would love to, but it is too labor intensive. However, we did check a few random journals and their ISSNs.
We found an error with the Journal Physical Education and Sport. On the AQCJ web site (see Figure 3) it says the journal has an ISSN of 2006-2483. In fact, this is the ISSN that the journal had up to about 2008, when it got a new ISSN, actually three (see Figure 4), that are shown in the figure. We also validated this ISSN with Scopus (see Figure 5).
Latest news
There is a scrolling news feed on the AQCJ web site. All of the news items are dated 2013, suggesting that news feed has not been updated for about seven years. One of the news items (see Figure 6) says that the AQCJ international board will be announced on 25 November 2013. We could find nothing on the web site where this board is listed.
Conclusion
Given what we have said above, we do not believe that AQCJ impact factor is a reliable source of information for scholars.
One of the most worrying points is that that evaluation methodology is not clear, it is not reproducible and we can have no confidence in how the impact factor was derived.
The other worrying point is the lack of rigour, both in the levels of English and the data itself. We have found several anomalies, which suggests that those managing the web site do not have the relevant checks and balances in place and are not applying academic rigor which is so important when supporting the scientific archive.
In conclusion, the African Quality Centre for Journals is not a reliable source of information and its impact factors have no meaning.