AI Research Rankings 2019: Insights from NeurIPS and ICML, Leading AI Conferences

UPDATE: We have published an update to this analysis for 2020 here. Enjoy!

Introduction

Welcome to the long-awaited refresh of our annual AI Research Rankings, 2019 edition (here is the first pilot of the rankings we published last year). This time we analyzed publications at the two most prestigious AI research conferences, Neural Information Processing Systems (NeurIPS, or NIPS) and International Conference on Machine Learning (ICML). Using conference proceedings (NeurIPS 2019 and ICML 2019), we went into each of the 2,200 accepted papers and compiled a list of authors and their affiliated organizations, and then calculated the Publication Index for each organization (see “Methodology” section below). The most intuitive way to think of the Publication Index is from the point of view of full paper equivalents: Google’s Publication Index of 167.3 can be interpreted as if Google published 167.3 full papers at the two leading AI conferences in 2019.

Methodology

The methodology of our Publication Index is inspired by the Nature Index:

AI Research Rankings 2019

Top 40 Global Organizations (Industry & Academia) Leading in AI Research in 2019 (with Publication Indices):

AI Research Rankings 2019—Top 40 Global Organizations (Industry & Academia) Leading in AI Research in 2019
AI Research Rankings 2019 — Top 20 Regions Leading in AI Research in 2019
AI Research Rankings 2019 — Top 20 Countries Leading in AI Research in 2019
AI Research Rankings 2019 — Top 20 American Universities Leading in AI Research in 2019
AI Research Rankings 2019 — Top 20 Global Universities Leading in AI Research in 2019
AI Research Rankings 2019 — Top 20 Companies Leading in AI Research in 2019

Further Analysis

Academia vs. Industry — Share of Total Publication Index:

AI Research Rankings 2019 — Academia vs. Industry
AI Research Rankings 2019 — Top 150 Words in 2200 Paper Titles at NeurIPS 2019 and ICML 2019
AI Research Rankings 2019 — Top 30 Countries by Per Capita Publication Index
AI Research Rankings 2019 — Treemap of Top 40 Organizations Leading in AI Research
The Herfindahl index calculation: si is the market share (percents are used as whole numbers, as in 75 instead of 0.75), and N is the number of participants.
  • An H below 1,500 indicates an unconcentrated industry.
  • An H between 1,500 to 2,500 indicates moderate concentration.
  • An H above 2,500 indicates high concentration.

Discussion: Who’s Ahead in AI?

A heated debate is going on today on the state of the strategic race between the United States and China to dominate in AI. We tend to side with a more balanced perspective, but before we begin our analysis, a bit of history is in order:

  • In China, these two events created a “Sputnik moment” which helped convince the Chinese government to prioritize and dramatically increase funding for artificial intelligence (see Kai-Fu Lee’s AI Superpowers).
  • In response, in July 2017 the Communist Party of China set 2030 as the deadline for an ambitious AI goal: it called for China to reach the top tier of AI economies by 2020, achieve major new breakthroughs by 2025, and become the global leader in AI by 2030. The strategy became known as the New Generation Artificial Intelligence Development Plan, and it has spurred many policies and billions of dollars of investment in research and development from ministries, provincial governments, and private companies.
  • Certain think tanks, such as CNAS, have argued that China’s AI strategy reflected the key principles from the Obama administration report — now it is China adopting them, instead of the United States.
  • This copying strategy isn’t new: to quote Peter Thiel’s Zero to One, “The Chinese have been straightforwardly copying everything that has worked in the developed world: 19th-century railroads, 20th-century air conditioning, and even entire cities. They might skip a few steps along the way — going straight to wireless without installing landlines, for instance — but they’re copying all the same.”
  • 2017 is precisely the year when we started tracking the state of AI research, so we established China’s baseline summarized in the following chart showing that the United States had an 11x lead in the total Publication Index over China:
Top 10 Countries Leading in AI in 2017: the USA had an 11x lead over China

Dataset

Please note that conferences don’t release publication data in a standard form, so our analysis ended up being quite manual (HTML parsing, Python transformations, a lot of manual name standardization, and a few unknown affiliations). If you find any bugs, please email us, and we’ll be happy to fix them.

Serial Entrepreneur & Investor | AI @ MIT & MBA @ Wharton | Peter Thiel, Y Combinator, Palantir, Goldman Sachs