It’s a little misleading to call this a third assessment as the first two were titled “The state of science and technology” whereas this time they’ve thrown “industrial research and development” (which previously rated its own separate assessment) into the mix as I noted in my July 1, 2016 post about this upcoming report by the Council of Canadian Academies (CCA).
To whet our appetites, the CCA’s expert panel has released some preliminary data according to a Dec. 15, 2016 news release (received via email),
The Council of Canadian Academies is pleased to release the Preliminary Data Update on Canadian Research Performance and International Reputation. This document represents the early work of the Expert Panel on the State of Science and Technology and Industrial Research and Development in Canada. It contains a preliminary update of key bibliometric and opinion survey data comparable to that published in the 2012 CCA assessment on the state of science and technology in Canada.
“This update provides a window into some of the data we are using to explore the state of research, development, and innovation in Canada,” said Max Blouw, Chair of the Expert Panel and President and Vice-Chancellor of Wilfrid Laurier University. “Our intention is to provide timely access to a body of evidence on Canada’s research performance that may serve as an important input to ongoing federal policy development.”
Highlights of this work include updated data on research output and collaboration, research impact, international reputation and stature, and data on research fields.
This data update is part of a larger project to assess the state of research, development, and innovation in Canada. The Expert Panel continues to work on its final report, which is expected to be released in late 2017.
I have taken a look at the material and these are the research highlights from the preliminary report,
Research Output and Collaboration
• Canada ranks ninth in the world in research publication output and accounts for 3.8% of the world’s output.
• Canada’s research output is growing at a rate comparable to that exhibited by most developed countries. Developed countries, however, are increasingly being overshadowed by the dramatic growth in research production in China and other emerging economies over the past decade.
• Canadian researchers continue to be highly collaborative internationally, working with international co-authors in nearly 46% of their publications.Research Impact
• Citation-based indicators show that Canadian research continues to have relatively high levels of impact. By ARC score, Canada ranks sixth out of leading countries: its research is cited 43% more than the world average across all fields of study.
• The impact of Canada’s research, as reflected in citations (ARC, MRC, and HCP1%), has increased in recent years. However, these increases have been often matched or exceeded by other countries. Canada’s rank by ARC declined slightly in many fields as a result.International Reputation and Stature
• Canada’s research contributions continue to be well regarded internationally according to a survey of top-cited researchers around the world. The share of top-cited researchers who rate Canada’s research as strong in their field of study rose from 68% in 2012 to 72% in 2016.
• Approximately 36% of surveyed top-cited researchers identify Canada as one of the top five countries in their research fields. As a result, Canada ranks fourth overall, behind the United States, United Kingdom, and Germany.
• The share of top-cited researchers who have worked or studied in Canada, or collaborated with Canadians, has increased since 2012.Data by Field of Research
• Preliminary analysis of Canadian research by field reveals patterns similar to those presented in the 2012 S&T report.
• All fields of research in Canada were cited at rates above the world average in 2009–2014. Few fields in Canada have experienced major shifts in output or impact in recent years, though the specialization rate of Clinical Medicine gradually increased and that of Engineering decreased relative to other countries.
• Fields in which Canada has both a relatively high degree of specialization and a high impact (above the G7 average) include Clinical Medicine; Biology; Information and Communication Technologies; Agriculture, Fisheries and Forestry; Earth and Environmental Sciences; and Economics and Business.
• Canada’s research contributions in Physics and Astronomy continue to be highly cited despite a lower publication output than might be expected. Chemistry and Enabling and Strategic Technologies (Energy, Biotechnology, Bioinformatics, Nanoscience and Nanotechnology, Optoelectronics and Photonics) are other areas in which Canada’s research output is low relative to other countries.
• When analyzed by field of study, results from the international survey of top-cited researchers are consistent with those from the 2012 survey. Canada continues to rank among the top five countries in three-quarters of fields.
• Canada’s research reputation is the weakest in core fields of the natural sciences such as Mathematics and Statistics, Physics and Astronomy, Chemistry, Engineering, and in Enabling and Strategic Technologies. [p. 5 PDF; p. v print]
As the panel notes they have the same problem as their predecessors. Bibliometric data, i. e., the number of papers your researchers have published, how often they’ve been cited, and in which journals (impact factor) they’ve been published are problematic as indicators of scientific progress. Excellent research can end up in an obscure journal and be ignored for decades while more problematic (substandard) work may be published in a prestigious (high impact) journal thereby gaining more attention. Unfortunately, despite these and other issues, bibliometric data remains a basic indicator of scientific progress. The expert panel for the 2012 report (State of Science and Technology) attempted to mitigate some of the problems by using other indicators. If I remember rightly, one of those indicators was an international survey of researchers (which is also problematic in some ways) about their awareness of and opinion of Canadian research. It seems this expert panel has also gone that route,
Qualitative evidence can be a useful complement to bibliometric data in assessing research performance, especially when drawing on the insights of researchers and scientists who are highly accomplished in their fields. Similar to the 2012 S&T report, a survey was sent to the top 1% of highly cited researchers by field worldwide, asking them to identify the leading countries in their areas of expertise. The results of this survey are comparable to those from 2012 and illustrate that Canada’s international research reputation remains strong across most fields of research.
6.1 CANADA’S OvERALL RESEARCH REPUTATION
Researchers were asked to identify the top five countries in their field and sub-field of expertise. As shown in Figure 6.1, 35.5% of respondents (compared with 37% in the 2012 survey) from across all fields of research rated Canada within the top five countries in their field. Canada ranks fourth out of all countries, behind the United States, United Kingdom, and Germany and ahead of France. This represents a change of about 1.5 percentage points from the overall results of the 2012 survey. There was a three percentage point decrease in how often France is ranked among the top five countries; the ordering of the top five countries, however, remains the same.
When asked to rate Canada’s research strength among other advanced countries in their field of expertise, 72% of respondents rated Canadian research as “strong” (corresponds to a score of 5 or higher on a 7-point scale), and 47% rated it as “very strong” (Figure 6.1 and Table 6.1). These ratings increased from 68% and 42%, respectively, in the 2012 report.16 [p. 29 PDF, p. 23 print]
Taking into account that there are no perfect measures, here’s what the preliminary report has to say overall,
Canada continues to rank within the top 10 countries in total output of research publications, but fell from seventh place to ninth between 2003–2008 and 2009–2014. Canada produces 3.8% of the world output.6 During the period, Canadian researchers produced about 496,696 publications (see Table 3.1).7 In the 2012 S&T report, Canada ranked seventh in 2005–2010 with roughly 395,000 scientific publications. Although India and Italy overtook Canada to reach the seventh and eighth positions, respectively, the distance separating Canada from Italy is negligible (over 2,000 publications). The United States continues to lead in number of publications, but the gap with China is rapidly narrowing.
This data update presents country rankings in a similar manner to the 2012 S&T report. Note that research output may be normalized by various measures to produce alternative rankings. For example, output can be examined relative to the size of the population or the economy of a country.
Figure 3.1 shows overall output of publications relative to a country’s population. By this measure, Canada ranks fifth with about 14 publications per 1,000 inhabitants in 2009–2014. This indicator shows China’s rank to be lower on a per capita basis; however, this could also indicate China’s potential for considerable future growth. For countries like Switzerland, high publication output reflects a high level of international collaboration and the presence of major scientific research facilities, such as CERN, which are associated with global networks of researchers. [p. 11 PDF; p. 5 print]
This represents a few bits of information from the panel’s 34 pp. preliminary report. If you have the time, do take a look at it. As these things go, it’s readable. One last comment, the panel notes that nothing about industrial research has been included in the preliminary report.