Jul 27, 2020
Is Innovation Stagnant?
Written by William Tomos Edwards
Introduction
Billionaire investor, Peter Thiel, calls out scientists, entrepreneurs, and anyone else who might be culpable every chance he gets. The charge: they don’t display anywhere near the bravura rate of innovation that their institutional forefathers once did.
Being a Trumpist in Silicon Valley, Thiel needn’t look very hard to find people who disagree with him. Bill Gates has replied rather glibly that Thiel, in his longing for science fiction inspired marvels, takes for granted just how far we have progressed in areas like smartphone technology. Certainly, many people second Gates’ assertion that “innovation is moving at a scarily fast pace.”
In a recent debate for UCLA Engineering, Thiel defended his position against University of Texas at Austin professor Robert Metcalfe. Thiel posits that Silicon Valley’s narrative of success and progress doesn’t “resonate in the economic data” or “the cultural experience.” Thiel notes that despite the unprecedented success of information communication technologies, we haven’t found a way to bury transmission lines and salaries are stagnant. In contrast, Metcalfe pays tribute to advances in geothermal power, CRISPR, immunotherapy, lithium-ion batteries, quantum computing, AI, and reusable rockets.
Evidently, it’s easy for these debates to manifest in the language of high-level abstractions, anecdotes, and appeals to intuition. A fine-grained analysis of the metrics that underlie innovation is preferable.
The Academic Discussion about Innovation
There is rigorous scholarly work surrounding the innovation debate, and it tends to hinge on one single very important metric; total factor productivity (TFP). Put simply, TFP is the ratio of economic output to economic input (e.g., capital and labour). Crucially, and subtly, TFP should map to technical progress, because, absent technical progress, the same inputs will always be required for the same level of output. TFP measures the extent to which we can do more with less.
Economist, Tyler Cowan has devoted a considerable part of his career to arguing that innovation is unacceptably stagnant in America by documenting how TFP growth has been stagnant across U.S. industries since the early 1970s. There is considerable nuance to this situation, and it will be addressed in the next section, but the secular, aggregate level decline in TFP growth is indisputable.
The most effective and succinct defense of the American innovation environment addresses TFP stagnation directly. Economist, Paul David was the first to propose that explosive innovations take time to diffuse into the wider business environment and positively impact TFP. Economists continue to advance the argument that technological implementation lags (such as the delayed implementation of AI), are the most important reason for TFP stagnation in the contemporary business environment.
The reality is that there are enumerable metrics that map to innovation, and when considered in their entirety they tell a complicated story. The next section will provide a detailed breakdown of some of the most important innovation-related metrics.
A Survey of Innovation Related Data
There are a number of indicators that the pace of innovation is experiencing no notable friction. The rate of approved patents in America has been steadily accelerating since 1980.
Furthermore, new technologies are now being adopted by American households faster than they were previously.
It took 30 years for the vacuum to be adopted by 60% of households versus 7 years for the tablet.
Computing power continues to increase unabated. A variety of measures related to information communications technology have closely followed Moore’s Law since the early ’70s.
Furthermore, the speed of IT adoption over 40 years has largely tracked the speed at which electricity was adopted from the late 19th century onward.
All of the above metrics imply that the American innovation environment is as healthy as it ever was. However, there are other indicators that display a worrisome trajectory. As stated previously, aggregate level TFP has been stagnant in the U.S. and this is made all the more perplexing when considering that the effective number of researchers has increased by a factor of 23 since 1930.
Despite the increase in researchers, there are indicators that it’s getting harder to make scientific progress. One-thousand physicists at top institutions were asked for their opinion on the importance of Nobel Prize-winning discoveries. The average opinion amongst these scholars is that there has been a secular decline in the importance of discoveries since the 1960s.
The researchers who conducted the survey bring attention to some other vexing metrics pertaining to scientific research in their piece for The Atlantic. Much larger teams are required to take on research questions today than were required decades ago. In figure 7 the average team size for scientific studies is shown with the dashed red line, and team size for studies with an above-average impact is shown with the black line.
Despite an exponential increase in the rate of published scientific studies, the amount of unique concepts discussed in those studies has only increased at a linear rate (figure 8).
Evidence from simulations indicates that the regimen of experiments bio-chemists are using to discover chemical networks is sub-optimal (figure 9). The red line in figure 9 displays the actual number of experiments required to discover a given percentage of chemical networks. The light and dark blue lines display the hypothetical-optimal number of experiments for discovering 50 or 100 percent of chemical networks respectively.
Figure 9 suggests that the dominant research programs in biochemistry are optimized for discovering a small percentage of given chemical networks.
It isn’t just the process of scientific discovery that is experiencing problems. Increasingly, more man-hours are required to produce new inventions. Inventors are increasingly older when they obtain their first patent (figure 10) and increasingly there are more inventors per patent (figure 11).
There are additional tangible indicators of an innovation slow down. Using the data on technology adoption rates provided in figure 2, I have built a chart displaying household technologies according to their type and the 40 year period during which the first wave of early adopters acquired them.
I removed certain technologies that seemed tenuous to include, as can be seen in Appendix I. A slowdown in innovation does seem to be reflected in the consumer experience. The data is certainly not exhaustive for household technologies, but assuming it is unbiased with respect to era, the last 40 years saw the introduction of fewer meaningful household technologies. An unprecedented amount of information communication technologies (ICT’s) were introduced, and virtually no meaningful non-ICT technologies were introduced to households. Intuitively, one may suppose that innovation has stagnated in some sectors but not in others.
A Deeper Dive into American TFP Data
U.S. Federal Reserve Economist, John Fernald, was perhaps the first person to take a meaningful deeper dive into the TFP stagnation. Fernald has demonstrated that TFP has in fact continued to increase in some sectors while becoming stagnant in others.
In order to further investigate the trajectory of TFP by sector and industry, I mined data from the U.S. Bureau of labor statistics (BLS) pertaining to TFP between 1987 and 2018. As demonstrated by Fernald, the correlation between time and TFP for the durable manufacturing sector is high (r = .96), however, when controlling for the TFP growth in the “computer and electronic products” industry (r = .99), it can be demonstrated that the proceeding industry is responsible for virtually all the growth in the durable manufacturing sector (partial r = -.36). The unweighted average TFP trend for all durable manufacturing industries minus computer and electronic products is demonstrated in figure 14.
Aside from Computer and Electronic Products, a number of other industries managed to beat the dominant trend. Appendix II lists every industry that had a correlation between TFP and time of at least r = .5 and 2018 TFP at least 40% greater than in 1987. Figure 15 gives the top five performing industries from a growth standpoint.
The industries in figure 15 have the greatest ratio of 2018 to 1987 TFP. The “Securities, Commodity Contracts and Investments” industry has been stagnant since the early 2000s but saw extreme growth between 1987 and 2001. Conversely, Oil and Gas Extraction was stagnant for the entire period except for the 2010s during which it experienced explosive growth.
Some industries haven’t necessarily seen extreme TFP growth since 1987, but they have seen steady growth with no protracted period of stagnation. Figure 16 gives the top five industries for steady uninterrupted TFP growth.
The data indicates that TFP growth may happen in certain industries for idiosyncratic reasons without diffusing to all industries within the same sector. Figures 17 and 18 demonstrate how this is the case in the “agriculture, forestry and fishery” and “mining” sectors respectively, where rare success stories have been observed in Oil and Gas, and Farming.
Among the many industries with stagnant TFP growth, some extremely worrying trajectories have occurred in almost the entire financial sector, the non-durable manufacturing sector, and important service sector industries that involve the application of technical and scientific knowledge.
Stagnant and even declining TFP has been observed in hospitals, education, and scientific and technical services. A segmented breakdown of the American TFP data complete with the above charts, and more, can be downloaded from my company Bright Tapestry Data.
Stagnant or not Stagnant?
First and foremost, it must be noted that there has not been stagnation across the board. The digital revolution has been truly innovative according to any reasonable standard. Enumerable “general-purpose technologies” have been introduced while computers have become faster and more efficient, and TFP has increased unabated across associated industries.
The full, comprehensive story is one of asymmetry. TFP has slowed for some firms and industries and moved along at an impressive pace for others. This is partly due to asymmetric implementation of the best new technologies. It also seems fairly clear, that the pace of discovery and innovation has slowed within some scientific avenues.
The top-performing industries for TFP growth (Appendix II) seem to fall into two major categories:
1) Industries involved in ICT and the digital revolution.
2) Industries involved in the transportation of goods throughout the global market place.
A few of the TFP success stories seem to have occurred for their own unique reasons (e.g., Oil and Gas Extraction, Farming). At the same time, extremely problematic TFP stagnation and even decline have been observed in enumerable industries of profound importance to society (see figures 19 to 21).
Serious problems with implementation were already rigorously documented, prior to this piece being written. In the data presented in the previous section, there is at least one striking example of how tricky it can be to implement new technologies in order to increase TFP.
The recent success in Oil and Gas Extraction fits the “implementation lag” hypothesis. The industry was stagnant for two decades until finally, TFP exploded. According to the Federal Reserve Bank of Kansas City, the implementation of an array of engineering innovations factors heavily into this industry’s TFP success. Many of those innovations such as hydraulic fracturing and horizontal drilling had been in development for decades. This finding hints at the distinct implementation asymmetries that exist between firms and industries because other industries that are similar in character to the oil industry remained stagnant or declined in TFP during the same period.
Aside from the failure to implement inventions and discoveries, scientists are having a harder time coming up with new ones. Figures 6 through 12 provide an overview of numerous indicators that it’s getting harder for scientists to produce new results. At the same time, some areas of scientific research, like biotech and AI, seem to be producing exciting results. Importantly, the asymmetric progress in science across disciplines mirrors the asymmetric growth in TFP across industries. The TFP growth in farming is partly due to bio-tech.
Put simply, scientific research has its own TFP problems; while more inventions are being patented each year (figure 1), far more man-hours are required to produce those patents. Thus the broader TFP stagnation in America isn’t simply due to a failure to implement, the TFP problems extend all the way down to primary research.
Nonetheless, there is reason to take heart seeing as there has indeed been technical progress in some areas. However, there is only one thing that can be said in defense of those fields where innovation has clearly stagnated; “AI, and ICT’s more generally, will kick-start scientific discovery and help us to implement those discoveries soon enough.” It remains to be seen whether or not that’s true.
Republished with permission from my company Bright Tapestry Data
