In releasing its strategy document Australia’s Innovation Catalyst the CSIRO has described the document as “its masterplan to improve Australia’s record in innovation and help the country respond to global changes and digital disruption.”
It is now well understood that to grow GDP per capita we need to increase productivity and that this comes from technology-based innovation and change.
The call for innovation is echoed by CSIRO chief executive Larry Marshall in his introduction to the strategy:
“Australia must be a high performing innovation economy. In an interconnected world of accelerating, technology-driven change, our future prosperity, health and sustainability is closely bound to our capacity for innovation. However, in 2014, global rankings of innovation such as those by Cornell, INSEAD and WIPO ranked Australia 10th in the world for innovation inputs, but 81st for innovation efficiency. We rank last in the OECD for research‑business collaboration, and our relative rankings for STEM education are in decline.”
Unfortunately, this paragraph reflects on how poorly the innovation discussion in this country is informed by useful data. The CSIRO chooses to repeat the claim about Australia’s ranking 81st for “innovation efficiency.”
Despite the impression given by the CSIRO, this is not the outcome from “global rankings of innovation such as those by Cornell, INSEAD and WIPO” but from just one ranking performed by the three together and published as the Global Innovation Index 2014 (GII).
The GII is a very impressive piece of work. The GII reports seven pillars, each is divided into three sub-pillars, and each sub-pillar is composed of three to five individual indicators. Sub-pillar scores and pillar scores are calculated as the weighted average of the individual indicators or sub-pillars respectively.
In all, 81 separate indicators are used to construct the index; 56 of them are objective or hard data, five are survey data, and the remaining 20 are other composite data. All seven pillars provide the overall Global Innovation Index, the Innovation Input Sub-Index is drawn from five pillars (Institutions, Human Capital and Research, Infrastructure, Market Sophistication and Business Sophistication), and the Innovation Output Sub-Index is drawn from the other two pillars (Knowledge and Technology Outputs, and Creative Outputs).
The ratio of the output sub-index to the input sub-index is the measure of “innovation efficiency” that the CSIRO refers to.
The question that needs to be asked before we draw any policy conclusions from this Global Innovation Index is whether it is a good or useful measure of innovation.
The Global Innovation Report provides a very interesting diagram (Figure 6 in the report) that relates GII to GDP per capita. It provides an upward sloping trend-line that leads to the interpretation that a higher GII is a useful explanation of a higher GDP per capita. The text, however, makes clear that they are using the graph to illustrate that a GII will be dependent on GDP per capita – that is, richer countries should have higher GIIs.
In language reminiscent of Australian policy, they class countries as either leaders/learners or underperformers depending on which side of the “trend-line” they sit, with the leader/learner distinction based on wealth.
But it is a useful question of whether the so-called “trend-line” has any meaning at all. It has been fit to the data as a polynomial of degree three on a log-linear graph. The R-squared for the trend-line is only 0.7163, which is actually quite low.
Australia on this measure winds up as an inefficient leader, which really seems to be a contradiction in terms.
If an innovation score is of any use then we would expect it to correlate to the issue of interest that is the growth of GDP per capita. The GII performs very poorly on that score.
Using the same International Monetary Fund 2014 database as the GII to generate a five-year cumulative average growth rate in GDP per capita (on a Purchasing Power Parity basis) the linear trend-line has a downward slope (that is the higher the GII, the lower the growth rate), but since the R-squared is only 0.1253, it is more correct to say there is simply no correlation between the two.
It is simply remarkable that something that has no predictive power at all has been able to garner the publicity that attaches to the GII. It is more amazing that our pre-eminent research body would draw on it as evidence.
The CSIRO CEO also cited Australia’s position on the OECD research-business collaboration measure.
As InnovationAus.com has noted, an interesting aspect of this measure is how much worse it is than the assessment of Australia’s own business community as reported in the Global Competitiveness Report.
But that inaccurate survey data also happens to be one of the five indicators in the GII drawn from survey data. As a consequence, we know that at least one of the indicators used of Innovation Inputs provides an inaccurately high measure for Australia.
Following on from the use by policy makers of the unsupportable claim that 75 per cent of the fastest growing occupations require STEM skills and knowledge, the use of the GII data perpetuates the failings of innovation policy development that it is not evidence based.
An explanation of the widespread referencing of the GII in the face of its unreliability is worth exploring. In approaching any policy data we should consider the principles stated by E.H. Carr in What is History? Carr wrote “that when we take up a work of history, our first concern should be not with the facts which it contains, but with the historian who wrote it.”
The Global Innovation Index is sponsored by WIPO – the World Intellectual Property Organisation. In the Index the “outcomes” are measured by a variety of intellectual property outputs (films, patents) and business formations rather than actual economic output.
Given that the CSIRO touts itself as “Australia’s largest patent holder” it is perhaps not so surprising that they cite the GII.
However in the same release the CSIRO also repeats its claim to be “the people behind fast Wi-Fi”. The 2011 book The Innovation Journey of WiFi makes no reference to the CSIRO or the team that developed their Wireless LAN patent. Americans generally regarded the CSIRO as a patent troll, and resented the prospect that US firms were slugging citizens to fund the Australian government’s research efforts. From the Australian perspective this has been a gargantuan effort by little old Australia to assert its intellectual property rights.
The question for Australia though is whether we’d have been better served by the CSIRO engaging directly in the global standard development activity rather than a spin-off trying to commercialise its patents with its own chipset. That perhaps is the difference between the CSIRO and other innovators in wireless.
Qualcomm is a business that “develops and patents mobile communications technologies”, though it recently has questioned the ongoing structure of fee payment for patents that are incorporated in standards.
Genuine innovation policy in Australia would start to ask the question of whether we have the linkages that connect our research capability to global standards development; whether we understand that the value of research-business collaboration isn’t a new product or patent, but a position in a global supply structure.
We will only answer these questions when we move beyond policy discussion based on poor data.
Do you know more? Contact James Riley via Email.