AI Research Rankings 2020: Can the United States Stay Ahead of China?
Please subscribe to our Twitter if you’d like to follow our Deep Tech research. 🤖
Introduction
Welcome to our annual AI Research Rankings, 2020 edition (here are the 2019 edition and the 2017/2018 pilot). For 2020, we analyzed publications at the two most prestigious AI research conferences: International Conference on Machine Learning (ICML 2020) and Neural Information Processing Systems (NeurIPS 2020). Using conference proceedings, we went into each of the 2,986 accepted papers (1,087 papers at ICML and 1,899 papers at NeurIPS) and compiled a list of authors and their affiliated organizations, and then calculated the Publication Index for each organization (see “Methodology” section below). The most intuitive way to think of the Publication Index is from the point of view of full paper equivalents: Google’s Publication Index of 220 can be interpreted as if Google published 220 full papers at the two leading AI conferences in 2020.
We will start this analysis with details on methodology, continue on to AI research rankings for 2020, then show further interesting descriptive charts, proceed with the changes between 2019 and 2020 rankings, discuss whether the United States can stay ahead of China, and finally conclude with a link to the dataset.
Methodology
The methodology of our Publication Index is inspired by the Nature Index:
To glean a country’s, a region’s or an institution’s contribution to an article, and to ensure they are not counted more than once, the Nature Index uses fractional count (FC), which takes into account the share of authorship on each article. The total FC available per article is 1, which is shared among all authors under the assumption that each contributed equally. For instance, an article with 10 authors means that each author receives an FC of 0.1. For authors who are affiliated with more than one institution, the author’s FC is then split equally between each institution. The total FC for an institution is calculated by summing the FC for individual affiliated authors. The process is similar for countries/regions, although complicated by the fact that some institutions have overseas labs that will be counted towards host country/region totals.
The only difference is that our Publication Index counts overseas labs towards the headquarters country/region (instead of the host country/region). This is a contentious point, but we believe that this approach better reflects the assignment of intellectual property and respective accrual of benefit to the headquarters, rather than the local lab. To make this point explicit, let’s look at DeepMind, an AI research lab located in the UK that Google acquired in 2014. In this analysis, we count papers published by DeepMind towards its current owner, Google, and hence the United States, which might disappoint our friends in the UK. However, given the complexity of locating each author on the map using conference proceedings alone, this was the only consistent treatment of authorship we could find. Let’s hope that conference organizers will make further authorship details available in the future so that we can create two versions of rankings, one based on corporate ownership structure, and the other based on the physical location of the authors.
Here is an example of the Publication Index calculation. If a paper has five authors — three from MIT, one from the University of Oxford, and one from Google — each author will get 1/5th of one point, or 0.2. As a result, from this paper alone, MIT will increase its Publication Index by 3*0.2=0.6 points, the University of Oxford will increase its index by 0.2, and Google will add 0.2. Since MIT is based in the United States, MIT affiliation will increase the Publication Index of the United States by 0.6. Similarly, since the University of Oxford is based in the UK, the EEA + Switzerland + UK region will increase by 0.2. Finally, Google is a multinational corporation headquartered in the United States, therefore the United States will increase its Publication Index by an additional 0.2, for a total increase of 0.8. If an author has multiple affiliations, we split his/her/their fraction across each of those affiliated institutions. For instance, in the case above, if the last author listed two affiliations, Google and Stanford University (instead of just Google), both Google and Stanford University would get additional 0.2/2=0.1 points.
Finally, the reason why we thought it was fair to combine ICML and NeurIPS publications into the same dataset is that they have similar perceived prestige among top AI researchers, similar institutional participation, and similar paper acceptance rates (21.8% for ICML and 20.0% for NeurIPS).
AI Research Rankings 2020
Top 50 Countries Leading in AI Research in 2020:
1. United States — 1677.8
2. China — 281.2
3. United Kingdom — 161.0
4. Canada — 114.5
5. France — 102.9
6. Germany — 91.5
7. Switzerland — 86.7
8. South Korea — 76.6
9. Japan — 57.8
10. Israel — 57.7
11. Australia — 47.6
12. Singapore — 30.1
13. India — 22.7
14. Italy — 19.5
15. Russia — 19.2
16. Sweden — 16.0
17. Netherlands — 15.1
18. Austria — 11.0
19. Denmark — 10.3
20. Saudi Arabia — 10.2
21. Finland — 9.4
22. Belgium — 8.9
23. Portugal — 6.6
24. Taiwan — 5.9
25. Spain — 5.4
26. Poland — 4.0
27. Vietnam — 2.9
28. Brazil — 2.8
29. Greece — 2.8
30. South Africa — 2.5
31. UAE — 2.2
32. Czechia — 1.8
33. Iran — 1.7
34. Chile — 1.3
35. Norway — 1.1
36. Romania — 1.0
37. Turkey — 1.0
38. Pakistan — 0.9
39. Hungary — 0.7
40. North Macedonia — 0.3
41. Luxembourg — 0.3
42. Egypt — 0.3
43. Barbados — 0.3
44. Thailand — 0.3
45. Cyprus — 0.3
46. Qatar — 0.2
47. Malaysia — 0.2
Top 100 Global Organizations (Industry & Academia) Leading in AI Research in 2020:
1. Google (USA) — 220.1
2. Stanford University (USA) — 106.1
3. MIT (USA) — 99.6
4. UC Berkeley (USA) — 86.7
5. Carnegie Mellon University (USA) — 71.3
6. Microsoft (USA) — 66.5
7. University of Oxford (UK) — 51.9
8. Facebook (USA) — 48.5
9. Tsinghua University (China) — 46.8
10. Princeton University (USA) — 45.0
11. UT Austin (USA) — 40.1
12. ETH (Switzerland) — 39.0
13. EPFL (Switzerland) — 36.5
14. Harvard University (USA) — 36.0
15. Cornell University (USA) — 35.6
16. Columbia University (USA) — 33.3
17. New York University (USA) — 33.2
18. UCLA (USA) — 33.0
19. University of Illinois at Urbana-Champaign (USA) — 32.8
20. KAIST (South Korea) — 31.8
21. IBM (USA) — 29.7
22. University of Cambridge (UK) — 29.3
23. UC San Diego (USA) — 27.8
24. Peking University (China) — 27.0
25. University of Pennsylvania (USA) — 26.5
26. Duke University (USA) — 24.9
27. National University of Singapore (Singapore) — 24.4
28. Georgia Institute of Technology (USA) — 23.9
29. University of Maryland (USA) — 22.7
30. University College London (UK) — 22.4
31. University of Washington (USA) — 22.3
32. University of Toronto (Canada) — 21.5
33. Technion (Israel) — 19.9
34. University of Michigan (USA) — 18.8
35. INRIA (France) — 18.6
36. UMass Amherst (USA) — 17.4
37. University of Wisconsin-Madison (USA) — 16.7
38. University of Southern California (USA) — 16.4
39. Texas A&M University (USA) — 15.6
40. Mila (Canada) — 15.6
41. Purdue University (USA) — 15.3
42. Boston University (USA) — 15.2
43. Shanghai Jiao Tong University (China) — 14.8
44. Seoul National University (South Korea) — 14.3
45. Huawei (China) — 14.3
46. NVIDIA (USA) — 13.9
47. Amazon (USA) — 13.5
48. RIKEN (Japan) — 13.5
49. University of Minnesota (USA) — 12.8
50. Johns Hopkins University (USA) — 12.4
51. McGill University (Canada) — 12.4
52. Tel Aviv University (Israel) — 12.3
53. Imperial College London (UK) — 12.2
54. University of Sydney (Australia) — 12.2
55. University of Chicago (USA) — 12.1
56. California Institute of Technology (USA) — 11.9
57. University of Tuebingen (Germany) — 11.9
58. University of Science and Technology of China (China) — 11.7
59. Northeastern University (USA) — 11.5
60. Samsung (South Korea) — 11.1
61. Rutgers University (USA) — 11.1
62. Rice University (USA) — 10.5
63. University of Tokyo (Japan) — 10.5
64. Alibaba (China) — 10.5
65. Nanjing University (China) — 10.4
66. Yale University (USA) — 10.2
67. University of Alberta (Canada) — 10.2
68. ENS Paris (France) — 10.1
69. KAUST (Saudi Arabia) — 10.0
70. University of British Columbia (Canada) — 9.9
71. Northwestern University (USA) — 9.8
72. Nanyang Technological University (China) — 9.7
73. Chinese University of Hong Kong (China) — 9.3
74. Tencent (China) — 9.2
75. Toyota Technological Institute at Chicago (USA) — 9.0
76. CNRS (France) — 9.0
77. University of Edinburgh (UK) — 8.9
78. Weizmann Institute (Israel) — 8.8
79. Hong Kong University of Science and Technology (China) — 8.2
80. UC Santa Barbara (USA) — 7.9
81. University of Montreal (Canada) — 7.9
82. University of Technology Sydney (Australia) — 7.9
83. University of Amsterdam (Netherlands) — 7.9
84. TU Munich (Germany) — 7.8
85. Yandex (Russia) — 7.5
86. Apple (USA) — 7.4
87. OpenAI (USA) — 7.3
88. Baidu (China) — 7.3
89. MPI Intelligent Systems (Germany) — 7.2
90. UC Davis (USA) — 7.2
91. Criteo (France) — 7.1
92. Ohio State University (USA) — 6.9
93. KTH Royal Institute of Technology (Sweden) — 6.8
94. UC Irvine (USA) — 6.8
95. Uber (USA) — 6.8
96. Intel (USA) — 6.7
97. Aalto University (Finland) — 6.5
98. University of North Carolina (USA) — 6.5
99. Hebrew University (Israel) — 6.5
100. Zhejiang University (China) — 6.1
Top 100 American Universities Leading in AI Research in 2020:
1. Stanford University — 106.1
2. MIT — 99.6
3. UC Berkeley — 86.7
4. Carnegie Mellon University — 71.3
5. Princeton University — 45.0
6. UT Austin — 40.1
7. Harvard University — 36.0
8. Cornell University — 35.6
9. Columbia University — 33.3
10. New York University — 33.2
11. UC Los Angeles — 33.0
12. University of Illinois at Urbana-Champaign — 32.8
13. UC San Diego — 27.8
14. University of Pennsylvania — 26.5
15. Duke University — 24.9
16. Georgia Institute of Technology — 23.9
17. University of Maryland — 22.7
18. University of Washington — 22.3
19. University of Michigan — 18.8
20. UMass Amherst — 17.4
21. University of Wisconsin-Madison — 16.7
22. University of Southern California — 16.4
23. Texas A&M University — 15.6
24. Purdue University — 15.3
25. Boston University — 15.2
26. University of Minnesota — 12.8
27. Johns Hopkins University — 12.4
28. University of Chicago — 12.1
29. California Institute of Technology — 11.9
30. Northeastern University — 11.5
31. Rutgers University — 11.1
32. Rice University — 10.5
33. Yale University — 10.2
34. Northwestern University — 9.8
35. Toyota Technological Institute at Chicago — 9.0
36. UC Santa Barbara — 7.9
37. UC Davis — 7.2
38. Ohio State University — 6.9
39. UC Irvine — 6.8
40. University of North Carolina — 6.5
41. University of Pittsburgh — 5.7
42. University of Utah — 5.4
43. Indiana University — 5.1
44. SUNY Stony Brook — 4.5
45. University of Washington Madison — 4.3
46. University of Virginia — 4.2
47. SUNY Buffalo — 4.1
48. University of Illinois at Chicago — 4.0
49. University of Iowa — 3.8
50. Pennsylvania State University — 3.8
51. Rensselaer Polytechnic Institute — 3.7
52. Brown University — 3.4
53. Allen Institute — 3.3
54. Institute for Advanced Study — 3.2
55. University of Florida — 3.2
56. North Carolina State University — 2.9
57. Virginia Tech — 2.8
58. Michigan State University — 2.8
59. Oregon State University — 2.7
60. Stevens Institute of Technology — 2.2
61. Lawrence Livermore National Laboratory — 2.2
62. Washington University in St. Louis — 2.2
63. Dartmouth College — 2.1
64. University of Arizona — 2.0
65. University of Notre Dame — 2.0
66. UT Dallas — 2.0
67. UC Merced — 1.9
68. University of Delaware — 1.8
69. UC Santa Cruz — 1.8
70. Rochester Institute of Technology — 1.7
71. UC Riverside — 1.7
72. Arizona State University — 1.6
73. University of Oregon — 1.5
74. UT Arlington — 1.5
75. New Jersey Institute of Technology — 1.3
76. Binghamton University — 1.3
77. Los Alamos National Laboratory — 1.2
78. Auburn University — 1.1
79. Lehigh University — 1.1
80. SUNY Albany — 1.0
81. Virginia Commonwealth University — 1.0
82. University of South Florida — 1.0
83. University of Connecticut — 1.0
84. Minnesota State University — 1.0
85. Carleton University — 1.0
86. Albert Einstein College of Medicine — 1.0
87. Vanderbilt University — 1.0
88. University of North Carolina at Charlotte — 1.0
89. Worcester Polytechnic Institute — 1.0
90. College of William and Mary — 1.0
91. Temple University — 0.8
92. Florida State University — 0.8
93. US Army Research Laboratory — 0.8
94. University of Arkansas — 0.8
95. Louisiana State University — 0.8
96. Brigham Young University — 0.8
97. University of Toledo — 0.7
98. Iowa State University — 0.7
99. Georgia State University — 0.7
100. University of Rochester — 0.6
Top 100 Global Universities Leading in AI Research in 2020:
1. Stanford University (USA) — 106.1
2. MIT (USA) — 99.6
3. UC Berkeley (USA) — 86.7
4. Carnegie Mellon University (USA) — 71.3
5. University of Oxford (UK) — 51.9
6. Tsinghua University (China) — 46.8
7. Princeton University (USA) — 45.0
8. UT Austin (USA) — 40.1
9. ETH (Switzerland) — 39.0
10. EPFL (Switzerland) — 36.5
11. Harvard University (USA) — 36.0
12. Cornell University (USA) — 35.6
13. Columbia University (USA) — 33.3
14. New York University (USA) — 33.2
15. UC Los Angeles (USA) — 33.0
16. University of Illinois at Urbana-Champaign (USA) — 32.8
17. KAIST (South Korea) — 31.8
18. University of Cambridge (UK) — 29.3
19. UC San Diego (USA) — 27.8
20. Peking University (China) — 27.0
21. University of Pennsylvania (USA) — 26.5
22. Duke University (USA) — 24.9
23. National University of Singapore (Singapore) — 24.4
24. Georgia Institute of Technology (USA) — 23.9
25. University of Maryland (USA) — 22.7
26. University College London (UK) — 22.4
27. University of Washington (USA) — 22.3
28. University of Toronto (Canada) — 21.5
29. Technion (Israel) — 19.9
30. University of Michigan (USA) — 18.8
31. INRIA (France) — 18.6
32. UMass Amherst (USA) — 17.4
33. University of Wisconsin-Madison (USA) — 16.7
34. University of Southern California (USA) — 16.4
35. Texas A&M University (USA) — 15.6
36. Mila (Canada) — 15.6
37. Purdue University (USA) — 15.3
38. Boston University (USA) — 15.2
39. Shanghai Jiao Tong University (China) — 14.8
40. Seoul National University (South Korea) — 14.3
41. RIKEN (Japan) — 13.5
42. University of Minnesota (USA) — 12.8
43. Johns Hopkins University (USA) — 12.4
44. McGill University (Canada) — 12.4
45. Tel Aviv University (Israel) — 12.3
46. Imperial College London (UK) — 12.2
47. University of Sydney (Australia) — 12.2
48. University of Chicago (USA) — 12.1
49. California Institute of Technology (USA) — 11.9
50. University of Tuebingen (Germany) — 11.9
51. University of Science and Technology of China (China) — 11.7
52. Northeastern University (USA) — 11.5
53. Rutgers University (USA) — 11.1
54. Rice University (USA) — 10.5
55. University of Tokyo (Japan) — 10.5
56. Nanjing University (China) — 10.4
57. Yale University (USA) — 10.2
58. University of Alberta (Canada) — 10.2
59. ENS Paris (France) — 10.1
60. KAUST (Saudi Arabia) — 10.0
61. University of British Columbia (Canada) — 9.9
62. Northwestern University (USA) — 9.8
63. Nanyang Technological University (China) — 9.7
64. Chinese University of Hong Kong (China) — 9.3
65. Toyota Technological Institute at Chicago (USA) — 9.0
66. CNRS (France) — 9.0
67. University of Edinburgh (UK) — 8.9
68. Weizmann Institute (Israel) — 8.8
69. Hong Kong University of Science and Technology (China) — 8.2
70. UC Santa Barbara (USA) — 7.9
71. University of Montreal (Canada) — 7.9
72. University of Technology Sydney (Australia) — 7.9
73. University of Amsterdam (Netherlands) — 7.9
74. TU Munich (Germany) — 7.8
75. MPI Intelligent Systems (Germany) — 7.2
76. UC Davis (USA) — 7.2
77. Ohio State University (USA) — 6.9
78. KTH Royal Institute of Technology (Sweden) — 6.8
79. UC Irvine (USA) — 6.8
80. Aalto University (Finland) — 6.5
81. University of North Carolina (USA) — 6.5
82. Hebrew University (Israel) — 6.5
83. Zhejiang University (China) — 6.1
84. University of Waterloo (Canada) — 5.9
85. University of Pittsburgh (USA) — 5.7
86. University of Utah (USA) — 5.4
87. IST Austria (Austria) — 5.4
88. IIS (India) — 5.3
89. Politecnico di Milano (Italy) — 5.2
90. Indiana University (USA) — 5.1
91. University of Melbourne (Australia) — 4.9
92. Xidian University (China) — 4.9
93. Ecole Polytechnique (France) — 4.7
94. Australian National University (Australia) — 4.5
95. Vector Institute (Canada) — 4.5
96. MPI Informatics (Germany) — 4.5
97. SUNY Stony Brook (USA) — 4.5
98. Skolkovo Institute of Science and Technology (Russia) — 4.5
99. ENSAE ParisTech (France) — 4.4
100. University of Washington Madison (USA) — 4.3
Top 100 Global Companies Leading in AI Research in 2020:
1. Google (USA) — 220.1
2. Microsoft (USA) — 66.5
3. Facebook (USA) — 48.5
4. IBM (USA) — 29.7
5. Huawei (China) — 14.3
6. NVIDIA (USA) — 13.9
7. Amazon (USA) — 13.5
8. Samsung (South Korea) — 11.1
9. Alibaba (China) — 10.5
10. Tencent (China) — 9.2
11. Yandex (Russia) — 7.5
12. Apple (USA) — 7.4
13. OpenAI (USA) — 7.3
14. Baidu (China) — 7.3
15. Criteo (France) — 7.1
16. Uber (USA) — 6.8
17. Intel (USA) — 6.7
18. Salesforce (USA) — 5.7
19. Adobe (USA) — 5.5
20. Qualcomm (USA) — 5.4
21. Bosch (Germany) — 5.1
22. NEC (Japan) — 4.0
23. SenseTime (China) — 3.8
24. JD (China) — 3.8
25. Flatiron Institute (USA) — 3.3
26. Bytedance (China) — 3.3
27. Element AI (Canada) — 3.3
28. Naver (South Korea) — 3.2
29. NTT (Japan) — 3.2
30. AITRICS (South Korea) — 3.0
31. Data61 (Australia) — 2.9
32. VinAI (Vietnam) — 2.5
33. Borealis AI (Canada) — 2.5
34. Mitsubishi (Japan) — 2.3
35. Kakao (South Korea) — 2.2
36. LinkedIn (USA) — 2.2
37. Preferred Networks (Japan) — 1.9
38. Hitachi (Japan) — 1.9
39. Autodesk (USA) — 1.8
40. Covariant AI (USA) — 1.8
41. Idiap (Switzerland) — 1.7
42. Shannon.AI (China) — 1.6
43. Neural Magic (USA) — 1.6
44. 4Paradigm (China) — 1.5
45. Spotify (Sweden) — 1.5
46. PROWLER.io (UK) — 1.5
47. Layer6 (Canada) — 1.4
48. JP Morgan (USA) — 1.3
49. Walmart (USA) — 1.3
50. reciTAL (France) — 1.2
51. Ant Group (China) — 1.2
52. Siemens (Germany) — 1.2
53. RealityEngines.AI (USA) — 1.2
54. Megvii (China) — 1.2
55. Pinterest (USA) — 1.1
56. Didi Chuxing (China) — 1.1
57. Speechmatics (UK) — 1.0
58. Vicarious AI (USA) — 1.0
59. SAS (USA) — 1.0
60. Netflix (USA) — 1.0
61. Fujitsu (Japan) — 1.0
62. Graphcore (UK) — 1.0
63. ESTsoft (South Korea) — 1.0
64. LightOn (France) — 0.9
65. Netease (China) — 0.9
66. Cyberagent (Japan) — 0.8
67. PROPHESEE (France) — 0.8
68. Guangzhou Huya Technology Co. (China) — 0.8
69. Sony (Japan) — 0.8
70. Latent Space (USA) — 0.8
71. NNAISENSE (Switzerland) — 0.7
72. AI21 (Israel) — 0.7
73. indust.ai (USA) — 0.7
74. Hugging Face (USA) — 0.7
75. InstaDeep (UK) — 0.7
76. Euclidean Technologies (USA) — 0.7
77. D-Wave (Canada) — 0.7
78. Volkswagen (Germany) — 0.6
79. ASAPP (USA) — 0.6
80. Abacus.AI (USA) — 0.6
81. Argo AI (USA) — 0.5
82. Ant Financial (China) — 0.5
83. Shenzhen SmartMore Technology (China) — 0.5
84. Yahoo! (USA) — 0.5
85. Toshiba (Japan) — 0.5
86. RJ Research Consulting (USA) — 0.5
87. Clova (South Korea) — 0.5
88. FiveAI (UK) — 0.5
89. Horizon Robotics (China) — 0.5
90. Twitter (USA) — 0.5
91. Radiance Technologies (USA) — 0.4
92. Invenia (Canada) — 0.4
93. iFLYTEK (China) — 0.4
94. TAL Education (China) — 0.4
95. XaiPient (USA) — 0.4
96. Honeywell (USA) — 0.4
97. Accenture (USA) — 0.4
98. RealAI (China) — 0.4
99. Petuum (USA) — 0.4
100. Tooploox (Poland) — 0.3
Further Analysis
Top 30 Regions Leading in AI Research in 2020:
1. USA — 1677.8
2. Europe* (EEA** + Switzerland + United Kingdom***) — 556.2
3. China — 281.2
4. Canada — 114.5
5. South Korea — 76.6
6. Japan — 57.8
7. Israel — 57.7
8. Australia — 47.6
9. Singapore — 30.1
10. India — 22.7
11. Russia — 19.2
12. Saudi Arabia — 10.2
13. Taiwan — 5.9
14. Vietnam — 2.9
15. Brazil — 2.8
16. South Africa — 2.5
17. UAE — 2.2
18. Iran — 1.7
19. Chile — 1.3
20. Turkey — 1.0
21. Pakistan — 0.9
22. Egypt — 0.3
23. North Macedonia — 0.3
24. Thailand — 0.3
25. Barbados — 0.3
26. Qatar — 0.2
27. Malaysia — 0.2
*The idea here is to group European countries that often have a common vision on AI research, but not necessarily any explicit coordination.
**Countries that belong to the EEA include Austria, Belgium, Bulgaria, Croatia, Republic of Cyprus, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, Spain, Sweden, Iceland, Liechtenstein, and Norway (source).
***This is not a political statement, but rather a clarification. Following the completion of its withdrawal from the EU, the UK could seek to continue to be a member of the EEA (source).
Top 50 Countries Leading in AI Research in 2020 by Per Capita Publication Index*:
1. Switzerland — 10.113
2. Israel — 6.378
3. Singapore — 5.269
4. United States — 5.126
5. Canada — 3.046
6. United Kingdom — 2.409
7. Australia — 1.876
8. Denmark — 1.769
9. Finland — 1.701
10. Sweden — 1.555
11. France — 1.535
12. South Korea — 1.482
13. Austria — 1.244
14. Germany — 1.100
15. Netherlands — 0.872
16. Barbados — 0.871
17. Belgium — 0.776
18. Portugal — 0.647
19. Luxembourg — 0.538
20. Japan — 0.458
21. Italy — 0.323
22. Saudi Arabia — 0.297
23. Greece — 0.260
24. Taiwan — 0.247
25. United Arab Emirates — 0.228
26. Cyprus — 0.209
27. Norway — 0.207
28. China — 0.198
29. Czechia — 0.166
30. North Macedonia — 0.160
31. Russia — 0.133
32. Spain — 0.114
33. Poland — 0.105
34. Qatar — 0.071
35. Hungary — 0.068
36. Chile — 0.066
37. Romania — 0.052
38. South Africa — 0.043
39. Vietnam — 0.030
40. Iran — 0.020
41. India — 0.017
42. Brazil — 0.013
43. Turkey — 0.012
44. Malaysia — 0.006
45. Pakistan — 0.004
46. Thailand — 0.004
47. Egypt — 0.003
*Publication Index divided by country population in millions according to the World Bank (source).
Academia vs. Industry — Share of Total Publication Index:
Academia — 78.9%
Industry — 21.1%
Word Cloud of Paper Titles:
Measuring Competition in AI Research (the Herfindahl Index):
The Herfindahl index (also known as Herfindahl–Hirschman Index) is a measure of the size of participants in relation to the industry and an indicator of the amount of competition among them:
Interpretation:
- An H below 100 indicates a highly competitive industry.
- An H below 1,500 indicates an unconcentrated industry.
- An H between 1,500 to 2,500 indicates moderate concentration.
- An H above 2,500 indicates a high concentration.
We can calculate the Herfindahl index two ways: by country and by organization. The former illustrates whether AI research is “monopolized” by any country and the latter shows “monopolization” by any organization.
- Herfindahl index by country: H=3,366, which indicates a highly-concentrated industry. In 2019, H=3,434, so over last year AI research got more competitive at the country level.
- Herfindahl index by organization: H=142, which indicates a non-concentrated industry. In 2019, H=146, so over last year AI research got slightly more competitive at the organization level.
Changes in AI Rankings Between 2019 and 2020
Changes in Top 5 Countries Leading in AI Research:
2019:
1. United States — 1260.2
2. China — 184.5
3. United Kingdom — 126.1
4. France — 94.3
5. Canada — 80.3
2020:
1. United States — 1677.8
2. China — 281.2
3. United Kingdom — 161.0
4. Canada — 114.5
5. France — 102.9
Changes in Top 5 Global Organizations Leading in AI Research:
2019:
1. Google (USA) — 167.3
2. Stanford University (USA) — 82.3
3. MIT (USA) — 69.8
4. Carnegie Mellon University (USA) — 67.7
5. UC Berkeley (USA) — 54.0
2020:
1. Google (USA) — 220.1
2. Stanford University (USA) — 106.1
3. MIT (USA) — 99.6
4. UC Berkeley (USA) — 86.7
5. Carnegie Mellon University (USA) — 71.3
Changes in Top 5 American Universities Leading in AI Research:
2019:
1. Stanford University — 82.3
2. MIT — 69.8
3. Carnegie Mellon University — 67.7
4. UC Berkeley — 54.0
5. Princeton University — 31.5
2020:
1. Stanford University — 106.1
2. MIT — 99.6
3. UC Berkeley — 86.7
4. Carnegie Mellon University — 71.3
5. Princeton University — 45.0
Changes in Top 5 Global Universities Leading in AI Research:
2019:
1. Stanford University (USA) — 82.3
2. MIT (USA) — 69.8
3. Carnegie Mellon University (USA) — 67.7
4. UC Berkeley (USA) — 54.0
5. University of Oxford (UK) — 37.7
2020:
1. Stanford University (USA) — 106.1
2. MIT (USA) — 99.6
3. UC Berkeley (USA) — 86.7
4. Carnegie Mellon University (USA) — 71.3
5. University of Oxford (UK) — 51.9
Changes in Top 5 Global Companies Leading in AI Research:
2019:
1. Google (USA) — 167.3
2. Microsoft (USA) — 51.9
3. Facebook (USA) — 33.1
4. IBM (USA) — 25.8
5. Amazon (USA) — 14.3
2020:
1. Google (USA) — 220.1
2. Microsoft (USA) — 66.5
3. Facebook (USA) — 48.5
4. IBM (USA) — 29.7
5. Huawei (China) — 14.3
As you can see, the top 5 lists are quite stable, with the exceptions of Huawei inching out Amazon and UC Berkeley leapfrogging CMU in 2020. However, just like in Lewis Carroll’s Red Queen’s race, top organizations need to publish significantly more papers each year just to maintain the lead:
“My dear, here we must run as fast as we can, just to stay in place. And if you wish to go anywhere you must run twice as fast as that.”
Can the United States Stay Ahead of China?
A heated debate is going on today on the state of the strategic race between the United States and China to dominate in AI. We tend to side with a more balanced perspective, but before we begin our analysis, a bit of history is in order (some of this will look familiar to the regular readers of our AI research rankings).
Two major events happened in AI in 2016: first, Google’s AlphaGo became the first computer program to beat a 9-dan Go professional, Lee Sedol, without handicaps; second, President Obama’s administration released a strategy on future directions and considerations for AI called Preparing for the Future of Artificial Intelligence. In China, these two events created a “Sputnik moment” which helped convince the Chinese government to prioritize and dramatically increase funding for artificial intelligence (see Kai-Fu Lee’s AI Superpowers).
In response, in 2017 the Communist Party of China set 2030 as the deadline for an ambitious AI goal: it called for China to reach the top tier of AI economies by 2020, achieve major new breakthroughs by 2025, and become the global leader in AI by 2030. The strategy became known as the New Generation Artificial Intelligence Development Plan, and it has spurred many policies and billions of dollars of investment in research and development from ministries, provincial governments, and private companies.
Certain think tanks, such as CNAS, argued that China’s AI strategy reflected the key principles from the Obama administration report — now it was China adopting them, instead of the United States. This copying strategy isn’t new: to quote Peter Thiel’s Zero to One, “The Chinese have been straightforwardly copying everything that has worked in the developed world: 19th-century railroads, 20th-century air conditioning, and even entire cities. They might skip a few steps along the way — going straight to wireless without installing landlines, for instance — but they’re copying all the same.”
As a result of this concerted effort by China, the American advantage in AI has been disappearing quickly: in 2017 the United States had an 11x lead over China (source); by 2019 the United States was down to a 7x lead (source); in 2020 the United States is left with a 6x lead (see above). Furthermore, this analysis by the Allen Institute for Artificial Intelligence found that China steadily increased its share of authorship of the top 10% most-cited papers.
One might say that it is not looking good for American competitiveness in AI in the next decade. However, we believe that the outcome will depend on the interplay of the advancement of three key ingredients of modern AI: algorithms, hardware, and training data, and it takes getting all three right in order to dominate the field.
We believe that the United States will have a strong lead in AI algorithms over the next few years, grounded in several decades of the advancement of computer science at world-class universities, such as MIT, Stanford, CMU, and UC Berkeley. In addition, the openness of the companies, such as Google and Facebook, to publishing internal research at AI conferences created a thriving ecosystem for top AI researchers, who now move seamlessly between academia and industry.
In addition, the United States is the home of Silicon Valley in its original silicon-focused definition, which has been at the forefront of hardware innovation. We think that it will be extremely difficult for China to catch up to the United States in advanced microprocessor technology over the next five to ten years, especially given the protections by vast patent portfolios held by Intel, AMD, and NVIDIA.
However, the American advantage is questionable when it comes to the availability of training data. Access to data is part of the broader privacy vs. public good debate, where the United States tends to choose the former, and China — the latter. In China today AI scans faces from hundreds of millions of street cameras, reads billions of WeChat messages, and analyzes millions of health records — all following the data-as-a-public-good argument. This training data availability, combined with China’s 1.4B population, creates an enormous strategic advantage for China.
Hard-pressed to draw a conclusion, we still think that the first two factors (algorithms and hardware) will outweigh the last one (availability of data), and the United States will maintain its lead in AI over the next few years. However, and this is the major takeaway from this analysis, the United States needs to wake up to the urgent need to innovate in AI and allocate significant public and private funds to educate undergraduate and graduate students in AI and empower AI research at top American research universities.
In August 2020 the White House announced a $1 billion research push for AI and quantum computing in response to many policy advisors who worried that America was falling behind in AI and quantum research compared to rivals like China, and warned that these technologies were instrumental not only for economic development but also national security. However, we believe that this number should be closer to the $1 trillion that Mark Cuban proposed to invest in AI and robotics back in 2016. Otherwise, the United States risks losing its strategic advantage in AI to China, just like it did in high-speed rail and space exploration.
Dataset
Please note that even data science conferences still don’t release publication data in any sort of Python-friendly form 🤷♂️, so our analysis ended up being quite manual (i.e. first parse HTML, then fix typos in organization names, standardize them, split lines with multiple organizations per author, summarize with a pivot table, etc.). If you find any bugs, please email us.
Further Reading
If you liked this post, you might also be interested in our analysis of ICML 2020 and NeurIPS 2020, where we show AI rankings at each conference separately. If you’d like to check out last year’s AI Research Rankings 2019, they are here.
About the author: My name is Gleb Chuvpilo, and I’m the Managing Partner at Thundermark Capital, a Venture Capital firm that invests in Deep Tech startups. I have a Master’s degree from the MIT Computer Science and Artificial Intelligence Lab and an MBA in Finance and Strategic Management from The Wharton School at the University of Pennsylvania.
