Caption: As of 2013, there were 7.8 million researchers globally, according to UNESCO. This means that 0.1 percent of the people in the world professionally do science. Their work is largely financed by governments, yet public officials are not themselves researchers. To help governments make sense of the scientific community, Russian mathematicians have devised a researcher typology. The authors initially identified three clusters, which they tentatively labeled as “leaders,” “successors,” and “toilers.” Credit: Lion_on_helium/MIPT Press Office

A June 28, 2018 Moscow Institute of Physics and Technology (MIPT; Russia) press release (also on EurekAlert) announces some intriguing research,

Researchers in various fields, from psychology to economics, build models of human behavior and reasoning to categorize people. But it does not happen as often that scientists undertake an analysis to classify their own kind.

However, research evaluation, and therefore scientist stratification as well, remain highly relevant. Six years ago, the government outlined the objective that Russian scientists should have 50 percent more publications in Web of Science- and Scopus-indexed journals. As of 2011, papers by researchers from Russia accounted for 1.66 percent of publications globally. By 2015, this number was supposed to reach 2.44%. It did grow but this has also sparked a discussion in the scientific community about the criteria used for evaluating research work.

The most common way of gauging the impact of a researcher is in terms of his or her publications. Namely, whether they are in a prestigious journal and how many times they have been cited. As with any good idea, however, one runs the risk of overdoing it. In 2005, U.S. physicist Jorge Hirsch proposed his h-index, which takes into account the number of publications by a given researcher and the number of times they have been cited. Now, scientists are increasingly doubting the adequacy of using bibliometric data as the sole independent criterion for evaluating research work. One obvious example of a flaw of this metric is that a paper can be frequently cited to point out a mistake in it.

Scientists are increasingly under pressure to publish more often. Research that might have reasonably been published in one paper is being split up into stages for separate publication. This calls for new approaches to the evaluation of work done by research groups and individual authors. Similarly, attempts to systematize the existing methods in scientometrics and stratify scientists are becoming more relevant, too. This is arguably even more important for Russia, where the research reform has been stretching for years.

One of the challenges in scientometrics is identifying the prominent types of researchers in different fields. A typology of scientists has been proposed by Moscow Institute of Physics and Technology Professor Pavel Chebotarev, who also heads the Laboratory of Mathematical Methods for Multiagent Systems Analysis at the Institute of Control Sciences of the Russian Academy of Sciences, and Ilya Vasilyev, a master’s student at MIPT.

In their paper, the two authors determined distinct types of scientists based on an indirect analysis of the style of research work, how papers are received by colleagues, and what impact they make. A further question addressed by the authors is to what degree researcher typology is affected by the scientific discipline.

“Each science has its own style of work. Publication strategies and citation practices vary, and leaders are distinguished in different ways,” says Chebotarev. “Even within a given discipline, things may be very different. This means that it is, unfortunately, not possible to have a universal system that would apply to anyone from a biologist to a philologist.”

“All of the reasonable systems that already exist are adjusted to particular disciplines,” he goes on. “They take into account the criteria used by the researchers themselves to judge who is who in their field. For example, scientists at the Institute for Nuclear Research of the Russian Academy of Sciences are divided into five groups based on what research they do, and they see a direct comparison of members of different groups as inadequate.”

The study was based on the citation data from the Google Scholar bibliographic database. To identify researcher types, the authors analyzed citation statistics for a large number of scientists, isolating and interpreting clusters of similar researchers.

Chebotarev and Vasilyev looked at the citation statistics for four groups of researchers returned by a Google Scholar search using the tags “Mathematics,” “Physics,” and “Psychology.” The first 515 and 556 search hits were considered in the case of physicists and psychologists, respectively. The authors studied two sets of mathematicians: the top 500 hits and hit Nos. 199-742. The four sets thus included frequently cited scientists from three disciplines indicating their general field of research in their profiles. Citation dynamics over each scientist’s career were examined using a range of indexes.

The authors initially identified three clusters, which they tentatively labeled as “leaders,” “successors,” and “toilers.” The leaders are experienced scientists widely recognized in their fields for research that has secured an annual citation count increase for them. The successors are young scientists who have more citations than toilers. The latter earn their high citation metrics owing to yearslong work, but they lack the illustrious scientific achievements.

Among the top 500 researchers indicating mathematics as their field of interest, 52 percent accounted for toilers, with successors and leaders making up 25.8 and 22.2 percent, respectively.

For physicists, the distribution was slightly different, with 48.5 percent of the set classified as toilers, 31.7 percent as successors, and 19.8 percent as leaders. That is, there were more successful young scientists, at the expense of leaders and toilers. This may be seen as a confirmation of the solitary nature of mathematical research, as compared with physics.

Finally, in the case of psychologists, toilers made up 47.7 percent of the set, with successors and leaders accounting for 18.3 and 34 percent. Comparing the distributions for the three disciplines investigated in the study, the authors conclude that there are more young achievers among those doing mathematical research.

A closer look enabled the authors to determine a more fine-grained cluster structure, which turned out to be remarkably similar for mathematicians and physicists. In particular, they identified a cluster of the youngest and most successful researchers, dubbed “precocious,” making up 4 percent of the mathematicians and 4.3 percent of the physicists in the set, along with the “youth” — successful researchers whose debuts were somewhat less dramatic: 29 and 31.7 percent of scientists doing math and physics research, respectively. Two further clusters were interpreted as recognized scientific authorities, or “luminaries,” and experienced researchers who have not seen an appreciable growth in the number of citations recently. Luminaries and the so-called inertia accounted for 52 and 15 percent of mathematicians and 50 and 14 percent of physicists, respectively.

There is an alternative way of clustering physicists, which recognizes a segment of researchers, who “caught the wave.” The authors suggest this might happen after joining major international research groups.

Among psychologists, 18.3 percent have been classified as precocious, though not as young as the physicists and mathematicians in the corresponding group. The most experienced and respected psychology researchers account for 22.5 percent, but there is no subdivision into luminaries and inertia, because those actively cited generally continue to be. Relatively young psychologists make up 59.2 percent of the set. The borders between clusters are relatively blurred in the case of psychology, which might be a feature of the humanities, according to the authors.

“Our pilot study showed even more similarity than we’d expected in how mathematicians and physicists are clustered,” says Chebotarev. “Whereas with psychology, things are noticeably different, yet the breakdown is slightly closer to math than physics. Perhaps, there is a certain connection between psychology and math after all, as some people say.”

“The next stage of this research features more disciplines. Hopefully, we will be ready to present the new results soon,” he concludes.

I think that they are attempting to create a new way of measuring scientific progress (scientometrics) by establishing a more representative means of measuring individual contributions based on the analysis they provide of the ways in which these ‘typologies’ are expressed across various disciplines.

For anyone who wants to investigate further, you will need to be able to read Russian. You can download the paper from here on MathNet.ru,.

Here’s my best attempt at a citation for the paper,

Making a typology of scientists on the basis of bibliometric data by I. Vasilyev, P. Yu. Chebotarev. Large-scale System Control (UBS), 2018, Issue 72, Pages 138–195 (Mi ubs948)

I’m glad to see this as there is a fair degree of dissatisfaction about the current measures for scientific progress used in any number of reports on the topic. As far as I can tell, this dissatisfaction is felt internationally.