2019 recent trends in GPU price per FLOPS

*Published 25 March, 2020*

We estimate that in recent years, GPU prices have fallen at rates that would yield an order of magnitude over roughly:

* 17 years for single-precision FLOPS
* 10 years for half-precision FLOPS
* 5 years for half-precision fused multiply-add FLOPS

Details
=======

GPUs (graphics processing units) are specialized electronic circuits originally used for computer graphics.[1](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-1-2316 "“A graphics processing unit (GPU) is a specialized electronic circuit designed to rapidly manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display device … Modern GPUs are very efficient at manipulating computer graphics and image processing … The term was popularized by Nvidia in 1999, who marketed the GeForce 256 as “the world’s first GPU”. It was presented as a “single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines”.”
“Graphics Processing Unit.” Wikipedia. Wikimedia Foundation, March 24, 2020. https://en.wikipedia.org/w/index.php?title=Graphics_processing_unit&oldid=947270104.") In recent years, they have been popularly used for machine learning applications.[2](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-2-2316 "Fraenkel, Bernard. “Council Post: For Machine Learning, It’s All About GPUs.” Forbes. Forbes Magazine, December 8, 2017. https://www.forbes.com/sites/forbestechcouncil/2017/12/01/for-machine-learning-its-all-about-gpus/#5ed90c227699.") One measure of GPU performance is FLOPS, the number of operations on floating-point numbers a GPU can perform in a second.[3](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-3-2316 "“In computing, floating point operations per second (FLOPS, flops or flop/s) is a measure of computer performance, useful in fields of scientific computations that require floating-point calculations. For such cases it is a more accurate measure than measuring instructions per second.”
“FLOPS.” Wikipedia. Wikimedia Foundation, March 24, 2020. https://en.wikipedia.org/w/index.php?title=FLOPS&oldid=947177339") This page looks at the trends in GPU price / FLOPS of theoretical peak performance over the past 13 years. It does not include the cost of operating the GPUs, and it does not consider GPUs rented through cloud computing.

Theoretical peak performance
----------------------------

‘Theoretical peak performance’ numbers appear to be determined by adding together the theoretical performances of the processing components of the GPU, which are calculated by multiplying the clock speed of the component by the number of instructions it can perform per cycle.[4](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-4-2316 "From this discussion on Nvidia’s forums about theoretical GFLOPS: “GPU theoretical flops calculation is similar conceptually. It will vary by GPU just as the CPU calculation varies by CPU architecture and model. To use K40m as an example: http://www.nvidia.com/content/PDF/kepler/Tesla-K40-PCIe-Passive-Board-Spec-BD-06902-001_v05.pdf

there are 15 SMs (2880/192), each with 64 DP ALUs that are capable of retiring one DP FMA instruction per cycle (== 2 DP Flops per cycle).

15 x 64 x 2 * 745MHz = 1.43 TFlops/sec

which is the stated perf:

http://www.nvidia.com/content/tesla/pdf/NVIDIA-Tesla-Kepler-Family-Datasheet.pdf “

Person. “Comparing CPU and GPU Theoretical GFLOPS.” NVIDIA Developer Forums, May 21, 2014. https://forums.developer.nvidia.com/t/comparing-cpu-and-gpu-theoretical-gflops/33335.") These numbers are given by the developer and may not reflect actual performance on a given application.[5](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-5-2316 "From this blog post on the performance of TensorCores, a component of new Nvidia GPUs specialized for deep learning: “The problem is it’s totally unclear how to approach the peak performance of 120 TFLOPS, and as far as I know, no one could achieve so significant speedup on real tasks. Let me know if you aware of good cases.”
Sapunov, Grigory. “Hardware for Deep Learning. Part 3: GPU.” Medium. Intento, January 20, 2020. https://blog.inten.to/hardware-for-deep-learning-part-3-gpu-8906c1644664.")
Metrics
-------

We collected data on multiple slightly different measures of GPU price and FLOPS performance.

### Price metrics

GPU prices are divided into release prices, which reflect the manufacturer suggested retail prices that GPUs are originally sold at, and active prices, which are the prices at which GPUs are actually sold at over time, often by resellers.

We expect that active prices better represent prices available to hardware users, but collect release prices also, as supporting evidence.

### FLOPS performance metrics

Several varieties of ‘FLOPS’ can be distinguished based on the specifics of the operations they involve. Here we are interested in single-precision FLOPS, half-precision FLOPS, and half-precision fused-multiply add FLOPS.

‘Single-precision’ and ‘half-precision’ refer to the number of bits used to specify a floating point number.[6](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-6-2316 "Gupta, Geetika. “Difference Between Single-, Double-, Multi-, Mixed-Precision: NVIDIA Blog.” The Official NVIDIA Blog, November 21, 2019. https://blogs.nvidia.com/blog/2019/11/15/whats-the-difference-between-single-double-multi-and-mixed-precision-computing/.") Using more bits to specify a number achieves greater precision at the cost of more computational steps per calculation. Our data suggests that GPUs have largely been improving in single-precision performance in recent decades,[7](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-7-2316 "See our 2017 analysis, footnote 4, which notes that single-precision price performance seems to be improving while double-precision price performance is not") and half-precision performance appears to be increasingly popular because it is adequate for deep learning.[8](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-8-2316 "“With the growing importance of deep learning and energy-saving approximate computing, half precision floating point arithmetic (FP16) is fast gaining popularity. Nvidia’s recent Pascal architecture was the first GPU that offered FP16 support.”
N. Ho and W. Wong, “Exploiting half precision arithmetic in Nvidia GPUs,” 2017 IEEE High Performance Extreme Computing Conference (HPEC), Waltham, MA, 2017, pp. 1-7.")
Nvidia, the main provider of chips for machine learning applications,[9](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-9-2316 "“In a recent paper, Google revealed that its TPU can be up to 30x faster than a GPU for inference (the TPU can’t do training of neural networks). As the main provider of chips for machine learning applications, Nvidia took some issue with that, arguing that some of its existing inference chips were already highly competitive to the TPU.”
Armasu, Lucian. “On Tensors, Tensorflow, And Nvidia’s Latest ‘Tensor Cores’.” Tom’s Hardware. Tom’s Hardware, May 11, 2017. https://www.tomshardware.com/news/nvidia-tensor-core-tesla-v100,34384.html.") recently released a series of GPUs featuring Tensor Cores,[10](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-10-2316 "“Tensor Cores in NVIDIA Volta GPU Architecture.” NVIDIA. Accessed May 2, 2020. https://www.nvidia.com/en-us/data-center/tensorcore/.
") which claim to deliver “groundbreaking AI performance”. Tensor Core performance is measured in FLOPS, but they perform exclusively certain kinds of floating-point operations known as fused multiply-adds (FMAs).[11](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-11-2316 "“Volta is equipped with 640 Tensor Cores, each performing 64 floating-point fused-multiply-add (FMA) operations per clock. That delivers up to 125 TFLOPS for training and inference applications.”
“Tensor Cores in NVIDIA Volta GPU Architecture.” NVIDIA. Accessed March 25, 2020. https://www.nvidia.com/en-us/data-center/tensorcore/.") Performance on these operations is important for certain kinds of deep learning performance,[12](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-12-2316 "“A useful operation in computer linear algebra is multiply-add: calculating the sum of a value c with a product of other values a x b to produce c + a x b. Typically, thousands of such products may be summed in a single accumulator for a model such as ResNet-50, with many millions of independent accumulations when running a model in deployment, and quadrillions of these for training models.”
Johnson, Jeff. “Making Floating Point Math Highly Efficient for AI Hardware.” Facebook AI Blog, November 8, 2018. https://ai.facebook.com/blog/making-floating-point-math-highly-efficient-for-ai-hardware/.") so we track ‘GPU price / FMA FLOPS’ as well as ‘GPU price / FLOPS’.

In addition to purely half-precision computations, Tensor Cores are capable of performing mixed-precision computations, where part of the computation is done in half-precision and part in single-precision.[13](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-13-2316 "See Figure 2:
Gupta, Geetika. “Using Tensor Cores for Mixed-Precision Scientific Computing.” NVIDIA Developer Blog, April 19, 2019. https://devblogs.nvidia.com/tensor-cores-mixed-precision-scientific-computing/.") Since explicitly mixed-precision-optimized hardware is quite recent, we don’t look at the trend in mixed-precision price performance, and only look at the trend in half-precision price performance.

#### Precision tradeoffs

Any GPU that performs multiple kinds of computations (single-precision, half-precision, half-precision fused multiply add) trades off performance on one for performance on the other, because there is limited space on the chip, and transistors must be allocated to either one type of computation or the other.[14](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-14-2316 "Three different individuals told us about this constraint, including one Nvidia employee.") All current GPUs that perform half-precision or TensorCore fused-multiply-add computations also do single-precision computations, so they are splitting their transistor budget. For this reason, our impression is that half-precision FLOPS could be much cheaper now if entire GPUs were allocated to each one alone, rather than split between them.

Release date prices
-------------------

We collected data on theoretical peak performance (FLOPS), release date, and price from several sources, including Wikipedia.[15](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-15-2316 "See the ‘Source’ column in this spreadsheet, tab ‘GPU Data’. We largely used TechPowerUp, Wikipedia’s List of Nvidia GPUs, List of AMD GPUs, and this document listing GPU performance.") (Data is available in [this spreadsheet](https://docs.google.com/spreadsheets/d/1ZZm5Wgr3BDRtloTZGylWzYTaVr5VqjiwOiRNu5Pz_q8/edit?usp=sharing)). We found GPUs by looking at Wikipedia’s existing large lists[16](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-16-2316 "See Wikipedia’s List of Nvidia GPUs and List of AMD GPUs.") and by Googling “popular GPUs” and “popular deep learning GPUs”. We included any hardware that was labeled as a ‘GPU’. We adjusted prices for inflation based on the consumer price index.[17](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-17-2316 "“CPI Home.” U.S. Bureau of Labor Statistics. U.S. Bureau of Labor Statistics. Accessed May 2, 2020. https://www.bls.gov/cpi/.")
We were unable to find price and performance data for many popular GPUs and suspect that we are missing many from our list. In our search, we did not find any GPUs that beat our 2017 minimum of $0.03 (release price) / single-precision GFLOPS. We put out a $20 bounty on a popular Facebook group to find a cheaper GPU / FLOPS, and the bounty went unclaimed, so we are reasonably confident in this minimum.[18](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-18-2316 "The Facebook group is for posting and claiming bounties and has around 750 people, many with interests in computers. The bounty has been up for two months, as of March 13 2020.")
### GPU price / single-precision FLOPS

Figure 1 shows our collected dataset for GPU price / single-precision FLOPS over time.[19](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-19-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for SP’ for the chart generation.")
![](https://lh6.googleusercontent.com/eakDTr43veWzFljIgZr2zzJ8--TzbWpIbelaGFeM7clFeQHdhcodTZfBAw5aUxkkJ5lNd3h4g7m8X6AsHwEm_kU-5gZnnESi26Mnf43eMCcD0W8EnpJDrPqBhN9OXT5W7UnQR9om)**Figure 1: Real GPU price / single-precision FLOPS over time. The vertical axis is log-scale. Price is measured in 2019 dollars.**
To find a clear trend for the prices of the cheapest GPUs / FLOPS, we looked at the running minimum prices every 10 days.[20](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-20-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for SP Minimums’ for the plotting. We used this script on the data from the ‘Cleaned GPU Data for SP’ to calculate the minimums and then import them into a new sheet of the spreadsheet.")
![](https://aiimpacts.org/wp-content/uploads/2020/05/image-1-1024x633.png)**Figure 2: Ten-day minimums in real GPU price / single-precision FLOPS over time. The vertical axis is log-scale. Price is measured in 2019 dollars. The blue line shows the trendline ignoring data before late 2007. (We believe the apparent steep decline prior to late 2007 is an artefact of a lack of data for that time period.)**
The cheapest GPU price / FLOPS hardware using release date pricing has not decreased since 2017. However there was a similar period of stagnation between early 2009 and 2011, so this may not represent a slowing of the trend in the long run.

Based on the figures above, the running minimums seem to follow a roughly exponential trend. If we do not include the initial point in 2007, (which we suspect is not in fact the cheapest hardware at the time), we get that the cheapest GPU price / single-precision FLOPS fell by around 17% per year, for a factor of ten in  ~12.5 years.[21](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-21-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for SP Minimums’ for the calculation.")
### GPU price / half-precision FLOPS

Figure 3 shows GPU price / half-precision FLOPS for all the GPUs in our search above for which we could find half-precision theoretical performance.[22](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-22-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP’ for the chart generation.")
![](https://lh4.googleusercontent.com/mAmmM2htFRBW13ObHMQSBBDbP0DiVsRqDZyvrvGXYS6iueDBNNPYABeJqDM2DWqi1o3S21IcMXbLCQeeWUR1xFSYDKapZW2vzr4_pFaXvpjN98DvZAwmLCx_3dQm21NWa7xlWKrh)**Figure 3: Real GPU price / half-precision FLOPS over time. The vertical axis is log-scale. Price is measured in 2019 dollars.**
Again, we looked at the running minimums of this graph every 10 days, shown in Figure 4 below.[23](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-23-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP Minimums’ for the plotting. We used this script on the data from the ‘Cleaned GPU Data for HP’ to calculate the minimums and then import them into a new sheet of the spreadsheet.")
![](https://lh6.googleusercontent.com/ZoVY4R_fEQVgrS4n-v8EkOorER7M-Wup_nAFUujdcRTwRrj3MyK5ADkcKgPremWa0uF_gMHUZYoZ6uq735_XyZa6d_voJMhOF7wyOEq23goNuCzjoFaKc-pHYhsXgMc0inmXHUNs)**Figure 4: Minimums in real GPU price / half-precision FLOPS over time. The vertical axis is log-scale. Price is measured in 2019 dollars.**
If we assume an exponential trend with noise,[24](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-24-2316 "Where ambiguous, we assume these trends are exponential rather than linear, because our understanding is that that is much more common historically in computing hardware price trends.") cheapest GPU price / half-precision FLOPS fell by around 26% per year, which would yield a factor of ten after ~8 years.[25](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-25-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP Minimums’ for the calculation.")
### GPU price / half-precision FMA FLOPS

Figure 5 shows GPU price / half-precision FMA FLOPS for all the GPUs in our search above for which we could find half-precision FMA theoretical performance.[26](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-26-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP + Tensor Cores’ for the chart generation.") (Note that this includes all of our half-precision data above, since those FLOPS could be used for fused-multiply adds in particular). GPUs with TensorCores are marked in red.

![](https://lh6.googleusercontent.com/Reu_O__PF1WmEYmT6RD8JIkotqoO-dO9HTqmoQBVf_lgj0CtTvl4TNm6E1kPJ5AirX0T5j3g1QOv6hwQLjpWct__by_g1lj3LFpSgd1emAKg3FyyDWP-gRyW5sTs4Ostp9JGkcRA)**Figure 5: Real GPU price / half-precision FMA FLOPS over time. Price is measured in 2019 dollars.**
Figure 6 shows the running minimums of GPU price / HP FMA FLOPS.[27](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-27-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP + Tensor Cores Minimums’ for the plotting. We used this script on the data from the ‘Cleaned GPU Data for HP + Tensor Cores’ to calculate the minimums and then import them into a new sheet of the spreadsheet.")
![](https://lh6.googleusercontent.com/SyYEb2HGDZCw8fPF15iIUwmU3U5r9UTKpYlaRXBFCMcrTBJhCf7VTeqwC6B82G2hmANUKYTiTLibBmsYm5XGAiV655z7GCdFX0jDeilXyX2cyHlcHe9wWwXRfQrPE2_gA3iTxzf3)**Figure 6: Minimums in real GPU price / half-precision FMA FLOPS over time. Price is measured in 2019 dollars.**
GPU price / Half-Precision FMA FLOPS appears to be following an exponential trend over the last four years, falling by around 46% per year, for a factor of ten in ~4 years.[28](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-28-2316 "See this spreadsheet, tab ‘Cleaned GPU Data for HP + Tensor Cores Minimums’ for the calculation.")
Active Prices
-------------

GPU prices often go down from the time of release, and some popular GPUs are older ones that have gone down in price.[29](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-29-2316 "For example, one of the GPUs recommended for deep learning in this Reddit thread is the GTX 1060 (6GB), which has been around since 2016.") Given this, it makes sense to look at active price data for the same GPU over time.

### Data Sources

We collected data on peak theoretical performance in FLOPS from [TechPowerUp](https://www.techpowerup.com/)[30](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-30-2316 "We scraped data from individual TechPowerUp pages using this script. Our full scraped TechPowerUp dataset can be found here.") and combined it with active GPU price data to get GPU price / FLOPS over time.[31](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-31-2316 "We chose to automatically scrape theoretical peak performance numbers from TechPowerUp instead of using the ones we manually collected above because there were several GPUs in the active pricing datasets that we hadn’t collected data for manually, and it was easier to scrape the entire site than just the subset of GPUs we needed.") Our primary source of historical pricing data was Passmark, though we also found a less trustworthy dataset on Kaggle which we used to check our analysis. We adjusted prices for inflation based on the consumer price index.[32](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-32-2316 "“CPI Home.” U.S. Bureau of Labor Statistics. U.S. Bureau of Labor Statistics. Accessed May 2, 2020. https://www.bls.gov/cpi/.")
#### Passmark

We scraped pricing data[33](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-33-2316 "We used this script.") on GPUs between 2011 and early 2020 from Passmark.[34](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-34-2316 "“PassMark – GeForce GTX 660 – Price Performance Comparison.” Accessed March 24, 2020. https://www.videocardbenchmark.net/gpu.php?gpu=GeForce+GTX+660&id=2152.") Where necessary, we renamed GPUs from Passmark to be consistent with TechPowerUp.[35](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-35-2316 "In most cases where renaming was necessary, the same GPU had multiple clear names, e.g. the “Radeon HD 7970 / R9 280X” in PassMark was just called the “Radeon HD 7970” in TechPowerUp. In a few cases, Passmark listed some GPUs which TechPowerUp listed separately as one GPU, e.g. “Radeon R9 290X / 390X” seemed to ambiguously refer to the Radeon R9 290X or Radeon R9 390X. In these cases, we conservatively assume that the GPU refers to the less powerful / earlier GPU. In one exceptional case, we assumed that the “Radeon R9 Fury + Fury X” referred to the Radeon Fury X in PassMark. The ambiguously named GPUs were not in the minimum data we calculated, so probably did not have a strong effect on the final result.") The Passmark data consists of 38,138 price points for 352 GPUs. We guess that these represent most popular GPUs.

Looking at the ‘current prices’ listed on individual Passmark GPU pages, prices appear to be sourced from Amazon, Newegg, and Ebay. Passmark’s listed pricing data does not correspond to regular intervals. We don’t know if prices were pulled at irregular intervals, or if Passmark pulls prices regularly and then only lists major changes as price points. When we see a price point, we treat it as though the GPU is that price only at that time point, not indefinitely into the future.

The data contains several blips where a GPU is briefly sold very unusually cheaply. A random checking of some of these suggests to us that these correspond to single or small numbers of GPUs for sale, which we are not interested in tracking, because we are trying to predict AI progress, which presumably isn’t influenced by temporary discounts on tiny batches of GPUs.

#### Kaggle

[This Kaggle dataset](https://www.kaggle.com/raczeq/ethereum-effect-pc-parts) contains scraped data of GPU prices from price comparison sites PriceSpy.co.uk, PCPartPicker.com, Geizhals.eu from the years 2013 – 2018. The Kaggle dataset has 319,147 price points for 284 GPUs. Unfortunately, at least some of the data is clearly wrong, potentially because price comparison sites include pricing data from untrustworthy merchants.[36](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-36-2316 "For example, the Kaggle dataset includes extremely cheap FirePro S7150s sold in 2014, even though the FirePro S7150 only came out in 2016. One of the sellers of these cheap GPUs were ‘Club 3D’, which also appeared to sell several other erroneously cheap GPUs.") As such, we don’t use the Kaggle data directly in our analysis, but do use it as a check on our Passmark data. The data that we get from Passmark roughly appears to be a subset of the Kaggle data from 2013 – 2018,[37](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-37-2316 "See this plot of Passmark single-precision GPU price / FLOPS compared to the combined Passmark and Kaggle single-precision GPU price / FLOPS, and this plot of Passmark half-precision GPU price / FLOPS compared to the combined Passmark and Kaggle half-precision $ / FLOPS. In both cases the 2013 – 2018 Passmark data appears to roughly be a subset of the Kaggle data.") which is what we would expect if the price comparison engines picked up prices from the merchants Passmark looks at.

#### Limitations

There are a number of reasons why we think this analysis may in fact not reflect GPU price trends:

* We effectively have just one source of pricing data, Passmark.
* Passmark appears to only look at Amazon, Newegg, and Ebay for pricing data.
* We are not sure, but we suspect that Passmark only looks at the U.S. versions of Amazon, Newegg, and Ebay, and pricing may be significantly different in other parts of the world (though we guess it wouldn’t be different enough to change the general trend much).
* As mentioned above, we are not sure if Passmark pulls price data regularly and only lists major price changes, or pulls price data irregularly. If the former is true, our data may be overrepresenting periods where the price changes dramatically.
* None of the price data we found includes quantities of GPUs which were available at that price, which means some prices may be for only a very limited number of GPUs.
* We don’t know how much the prices from these datasets reflect the prices that a company pays when buying GPUs in bulk, which we may be more interested in tracking.

A better version of this analysis might start with more complete data from price comparison engines (along the lines of the Kaggle dataset) and then filter out clearly erroneous pricing information in some principled way.

### Data

The original scraped datasets with cards renamed to match TechPowerUp can be found [here](https://drive.google.com/drive/folders/1cCjG_sUUePxbh5fN9ViOPX6GW9D2GyPJ?usp=sharing). GPU price / FLOPS data is graphed on a log scale in the figures below. Price points for the same GPU are marked in the same color. We adjusted prices for inflation using the consumer price index.[38](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-38-2316 "“CPI Home.” U.S. Bureau of Labor Statistics. U.S. Bureau of Labor Statistics. Accessed May 2, 2020. https://www.bls.gov/cpi/.") All points below are in 2019 dollars.

To try to filter out noisy prices that didn’t last or were only available in small numbers, we took out the lowest 5% of data in every several day period[39](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-39-2316 "We set this period to be 10 days long when looking at single-precision data, and 30 days long when looking at half-precision data, since half-precision data was significantly more sparse.") to get the 95th percentile cheapest hardware. We then found linear and exponential trendlines of best fit through the available hardware with the lowest GPU price / FLOPS every several days.[40](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-40-2316 "This calculation can be found in this spreadsheet.")
#### GPU price / single-precision FLOPS

Figures 7-10 show the raw data, 95th percentile data, and trendlines for single-precision GPU price / FLOPS for the Passmark dataset. [This folder](https://drive.google.com/open?id=1-PEl2kSORRH78Qa4huRF-t_g_m1QOTDs) contains plots of all our datasets, including the Kaggle dataset and combined Passmark + Kaggle dataset.[41](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-41-2316 "We used a Python plotting library to generate our plots, the script can be found here. All of our resulting plots can be found here. ‘single’ vs. ‘half’ refers to whether its $ / FLOPS data for single or half-precision FLOPS, ‘passmark’, ‘kaggle’, and ‘combined’ refer to which dataset is being plotted and ‘raw’ vs. ‘95’ refer to whether we’re plotting all the data or the 95th percentile data.")
![](https://aiimpacts.org/wp-content/uploads/2020/04/single_passmark_raw_crop-1-1024x525.png)
**Figure 7: GPU price / single-precision FLOPS over time, taken from our Passmark dataset.[42](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-42-2316 "The dataset we used for this plot can be found here. This a processed version of our scraped dataset, with prices / FLOPS adjusted for inflation. The script we used to process and plot can be found here.") Price is measured in 2019 dollars. [This picture](https://drive.google.com/open?id=194Tqrcix2XdytbT-WbgcRbFpeHIHqsao) shows that the Kaggle data does appear to be a superset of the Passmark data from 2013 – 2018, giving us some evidence that the Passmark data is correct. The vertical axis is log-scale.**

![](https://aiimpacts.org/wp-content/uploads/2020/04/single_passmark_95_crop-1024x525.png)
**Figure 8: The top 95% of data every 10 days for GPU price / single-precision FLOPS over time, taken from the Passmark dataset we plotted above. (Figure 7 with the cheapest 5% removed.) The vertical axis is log-scale.[43](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-43-2316 "The script to calculate the 95th percentile and generate this plot can be found here.")**

![](https://aiimpacts.org/wp-content/uploads/2020/04/single_passmark_95_zoom_crop-1024x525.png)
**Figure 9: The same data as Figure 8, with the vertical axis zoomed-in.**
![](https://aiimpacts.org/wp-content/uploads/2020/06/image-1024x633.png)**Figure 10: The minimum data points from the top 95% of the Passmark dataset, taken every 10 days. We fit linear and exponential trendlines through the data. The vertical axis is log-scale.[44](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-44-2316 "See here, tab ‘Passmark SP Minimums’ to see our calculation of the minimums over time. We used this script to generate the minimums, then imported them into this spreadsheet.")**
##### Analysis

The cheapest 95th percentile data every 10 days appears to fit relatively well to both a linear and exponential trendline. However we assume that progress will follow an exponential, because previous progress has [followed an exponential](https://aiimpacts.org/recent-trend-in-the-cost-of-computing/).

In the Passmark dataset, the exponential trendline suggested that from 2011 to 2020, 95th-percentile GPU price / single-precision FLOPS fell by around 13% per year, for a factor of ten in ~17 years,[45](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-45-2316 "You can see our calculations for this here, sheet ‘Passmark SP Minimums’. Each sheet has a cell ‘Rate to move an order of magnitude’ which has our calculation for how many years we need to move an order of magnitude. In the (untrustworthy) Kaggle dataset alone, its rate would yield an order of magnitude of decrease every ~12 years, and the rate in the combined dataset  would yield an order of magnitude of decrease every ~16 years.") bootstrap[46](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-46-2316 "Orloff, Jeremy, and Jonathan Bloom. “Bootstrap Confidence Intervals.” MIT OpenCourseWare, 2014. https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf.") 95% confidence interval 16.3 to 18.1 years.[47](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-47-2316 "We used this script to generate bootstrap confidence intervals for our datasets.") We believe the rise in price / FLOPS in 2017 corresponds to a rise in GPU prices due to increased demand from cryptocurrency miners.[48](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-48-2316 "We think this is the case because we’ve observed this dip in other GPU analyses we’ve done, and because the timing lines up: the first table in this article shows how GPU prices were increasing starting 2017 and continued to increase through 2018, and the chart here shows how GPU prices increased in 2017.") If we instead look at the trend from 2011 through 2016, before the cryptocurrency rise, we instead get that 95th-percentile GPU price / single-precision FLOPS price fell by around 13% per year, for a factor of ten in ~16 years.[49](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-49-2316 "You can see our calculations for this here, sheet ‘Passmark SP Minimums’, next to ‘Exponential trendline from 2015 to 2016. The trendline calculated is technically the linear fit through the log of the data.")
This is slower than the order of magnitude every ~12.5 years we found when looking at release prices. If we restrict the release price data to 2011 – 2019, we get an order of magnitude decrease every ~13.5 years instead,[50](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-50-2316 "See our calculation here, tab ‘Cleaned GPU Data for SP Minimums’, next to the cell marked “Exponential trendline from 2011 to 2019.”") so part of the discrepancy can be explained because of the different start times of the datasets. To get some assurance that our active price data wasn’t erroneous, we spot checked the best active price at the start of 2011, which was somewhat lower than the best release price at the same time, and confirmed that its given price was consistent with surrounding pricing data.[51](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-51-2316 "At the start of 2011, the minimum release price / FLOPS (see tab, ‘Cleaned GPU Data for SP Minimums’)  is .000135 $ / FLOPS, whereas the minimum active price / FLOPS (see tab, ‘Passmark SP Minimums’) is around .0001 $ / FLOPS. The initial GPU price / FLOPS minimum (see sheet ‘Passmark SP Minimums’) corresponds to the Radeon HD 5850 which had a price of $184.9 in 3/2011 and a release price of $259. Looking at the general trend in Passmark suggests that the Radeon HD 5850 did indeed rapidly decline from its $259 release price to consistently below $200 prices.") We think active prices are likely to be closer to the prices at which people actually bought GPUs, so we guess that ~17 years / order of magnitude decrease is a more accurate estimate of the trend we care about.

#### GPU price / half-precision FLOPS

Figures 11-14 show the raw data, 95th percentile data, and trendlines for half-precision GPU price / FLOPS for the Passmark dataset. [This folder](https://drive.google.com/open?id=1-PEl2kSORRH78Qa4huRF-t_g_m1QOTDs) contains plots of the Kaggle dataset and combined Passmark + Kaggle dataset.

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_raw_crop-1024x525.png)
 **Figure 11: GPU price / half-precision FLOPS over time, taken from our Passmark dataset. Price is measured in 2019 dollars.[52](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-52-2316 " The dataset we used for this plot can be found here. This is a processed version of our scraped dataset, with prices / FLOPS adjusted for inflation. The script we used to process and plot can be found here.") This picture shows that the Kaggle data does appear to be a superset of the Passmark data from 2013 – 2018, giving us some evidence that the Passmark data is reasonable. The vertical axis is log-scale.**

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_95_crop-1024x525.png)
**Figure 12: The top 95% of data every 30 days for GPU price / half-precision FLOPS over time, taken from the Passmark dataset we plotted above. (Figure 11 with the cheapest 5% removed.) The vertical axis is log-scale.[53](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-53-2316 "The script to calculate the 95th percentile and generate this plot can be found here.")**

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_95_zoom_crop-1024x525.png)
**Figure 13: The same data as Figure 12, with the vertical axis zoomed-in.**
![](https://aiimpacts.org/wp-content/uploads/2020/06/image-1-1024x633.png)**Figure 14: The minimum data points from the top 95% of the Passmark dataset, taken every 30 days. We fit linear and exponential trendlines through the data. The vertical axis is log-scale.[54](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-54-2316 "See here, tab ‘Passmark HP Minimums’ to see our calculation of the minimums over time. We used this script to generate the minimums, then imported them into this spreadsheet.")**
##### Analysis

If we assume the trend is exponential, the Passmark trend seems to suggest that from 2015 to 2020, 95th-percentile GPU price / half-precision FLOPS of GPUs has fallen by around 21% per year, for a factor of ten over ~10 years,[55](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-55-2316 "See the sheet marked ‘Passmark HP minimums’ in this spreadsheet. The trendline calculated is technically the linear fit through the log of the data.") bootstrap[56](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-56-2316 "Orloff, Jeremy, and Jonathan Bloom. “Bootstrap Confidence Intervals.” MIT OpenCourseWare, 2014. https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf.") 95% confidence interval 8.8 to 11 years.[57](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-57-2316 "We used this script to generate bootstrap confidence intervals for our datasets.") This is fairly close to the ~8 years / order of magnitude decrease we found when looking at release price data, but we treat active prices as a more accurate estimate of the actual prices at which people bought GPUs. As in our previous dataset, there is a noticeable rise in 2017, which we think is due to GPU prices increasing as a result of cryptocurrency miners. If we look at the trend from 2015 through 2016, before this rise, we get that 95th-percentile GPU price / half-precision FLOPS has fallen by around 14% per year, which would yield a factor of ten over ~8 years.[58](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-58-2316 "See the sheet marked ‘Passmark HP minimums’ in this spreadsheet.")
#### GPU price / half-precision FMA FLOPS

Figures 15-18 show the raw data, 95th percentile data, and trendlines for half-precision GPU price / FMA FLOPS for the Passmark dataset. GPUs with Tensor Cores are marked in black. [This folder](https://drive.google.com/open?id=1-PEl2kSORRH78Qa4huRF-t_g_m1QOTDs) contains plots of the Kaggle dataset and combined Passmark + Kaggle dataset.

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_tensor_raw_crop-1024x525.png)
**Figure 15: GPU price / half-precision FMA FLOPS over time, taken from our Passmark dataset.[59](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-59-2316 "The dataset we used for this plot can be found here. This a processed version of our scraped dataset, with prices / FLOPS adjusted for inflation. The script we used to process and plot can be found here.") price is measured in 2019 dollars. This picture shows that the Kaggle data does appear to be a superset of the Passmark data from 2013 – 2018, giving us some evidence that the Passmark data is correct. The vertical axis is log-scale.**

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_tensor_95_crop-1024x520.png)
**Figure 16: The top 95% of data every 30 days for GPU price / half-precision FMA FLOPS over time, taken from the Passmark dataset we plotted above.[60](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-60-2316 "The script to calculate the 95th percentile and generate this plot can be found here.") (Figure 15 with the cheapest 5% removed.)**

![](https://aiimpacts.org/wp-content/uploads/2020/04/half_passmark_tensor_95_zoom_crop-1024x520.png)
**Figure 17: The same data as Figure 16, with the vertical axis zoomed-in.**
![](https://aiimpacts.org/wp-content/uploads/2020/06/image-2-1024x633.png)**Figure 18: The minimum data points from the top 95% of the Passmark dataset, taken every 30 days. We fit linear and exponential trendlines through the data.[61](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-61-2316 "See here, tab ‘Passmark HP FMA Minimums’ to see our calculation of the minimums over time. We used this script to generate the minimums, then imported them into this spreadsheet.")**
##### Analysis

If we assume the trend is exponential, the Passmark trend seems to suggest the 95th-percentile GPU price / half-precision FMA FLOPS of GPUs has fallen by around 40% per year, which would yield a factor of ten in ~4.5 years,[62](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-62-2316 "See the sheet marked ‘Passmark HP FMA minimums’ in this spreadsheet. The trendline calculated is technically the linear fit through the log of the data.") with a bootstrap[63](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-63-2316 "Orloff, Jeremy, and Jonathan Bloom. “Bootstrap Confidence Intervals.” MIT OpenCourseWare, 2014. https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf.") 95% confidence interval 4 to 5.2 years.[64](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-64-2316 "We used this script to generate bootstrap confidence intervals for our datasets.") This is fairly close to the ~4 years / order of magnitude decrease we found when looking at release price data, but we think active prices are a more accurate estimate of the actual prices at which people bought GPUs.

The figures above suggest that certain GPUs with Tensor Cores were a significant (~half an order of magnitude) improvement over existing GPU price / half-precision FMA FLOPS.

Conclusion
==========

We summarize our results in the table below.

|  |  |  |  |
| --- | --- | --- | --- |
|  | **Release Prices** | **95th-percentile Active Prices** | **95th-percentile Active Prices** **(pre-crypto price rise)** |
|  | *11/2007 – 1/2020* | *3/2011 – 1/2020* | *3/2011 – 12/2016*  |
| **$ / single-precision FLOPS** | 12.5 | 17 | 16 |
|  | *9/2014 – 1/2020* | *1/2015 – 1/2020* | *1/2015 – 12/2016*  |
| **$ / half-precision FLOPS** | 8 | 10 | 8 |
| **$ / half-precision FMA FLOPS** | 4 | 4.5 | — |

Release price data seems to generally support the trends we found in active prices, with the notable exception of trends in GPU price / single-precision FLOPS, which cannot be explained solely by the different start dates.[65](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/#easy-footnote-bottom-65-2316 "See our analysis in this section above.") We think the best estimate of the overall trend for prices at which people recently bought GPUs is the 95th-percentile active price data from 2011 – 2020, since release price data does not account for existing GPUs becoming cheaper over time. The pre-crypto trends are similar to the overall trends, suggesting that the trends we are seeing are not anomalous due to cryptocurrency.

Given that, we guess that GPU prices as a whole have fallen at rates that would yield an order of magnitude over roughly:

* 17 years for single-precision FLOPS
* 10 years for half-precision FLOPS
* 5 years for half-precision fused multiply-add FLOPS

Half-precision FLOPS seem to have become cheaper substantially faster than single-precision in recent years. This may be a “catching up” effect as more of the space on GPUs was allocated to half-precision computing, rather than reflecting more fundamental technological progress.

*Primary author: Asya Bergal*

Notes
=====

This post analyzes the the trends in cost per FLOP for GPUs. There are a bunch of details in how to do this analysis, but they end up finding that this cost goes down by an order of magnitude over 17 years for single-precision FLOPS (halving time: 5 years), 10 years for half-precision FLOPS (halving time: 3 years), and 5 years for half-precision fused multiply-add FLOPS (halving time: 1.5 years). However, the latter two categories have become more popular in recent years with the rise of deep learning, so their low halving times might be because some of the single-precision hardware was converted to half-precision hardware, rather than fundamental technological improvements.


Cortés, Pizarro, and Afonso as precedents for takeover

*Daniel Kokotajlo, 29 February 2020*

*Epistemic status: I am not a historian, nor have I investigated these case studies in detail. I admit I am still uncertain about how the conquistadors were able to colonize so much of the world so quickly. I think my ignorance is excusable because this is just a blog post; I welcome corrections from people who know more. If it generates sufficient interest I might do a deeper investigation. Even if I’m right, this is just one set of historical case-studies; it doesn’t prove anything about AI, even if it is suggestive. Finally, in describing these conquistadors as “successful,” I simply mean that they achieved their goals, not that what they achieved was good.*

### **Summary**

In the span of a few years, some minor European explorers (later known as the conquistadors) encountered, conquered, and enslaved several huge regions of the world. That they were able to do this is surprising; their technological advantage was not huge. (This was before the scientific and industrial revolutions.) From these cases, I think we learn that it is occasionally possible for a small force to quickly conquer large parts of the world, despite:

1. Having only a minuscule fraction of the world’s resources and power
2. Having technology + diplomatic and strategic cunning that is better but not *that* much better
3. Having very little data about the world when the conquest begins
4. Being disunited

Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.

 *EDIT: In light of good pushback from people (e.g.* [*Lucy.ea8*](https://forum.effectivealtruism.org/posts/MNPrXCsPpwTgygMxc/cortes-pizarro-and-afonso-as-precedents-for-takeover#xZ8h23jc9pmp8Mqhy) *and* [*Matthew Barnett)*](https://www.alignmentforum.org/posts/ivpKSjM4D6FbqF4pZ/cortes-pizarro-and-afonso-as-precedents-for-takeover#kNFNjBJjuzTd3irrR) *about the importance of disease, I think one should probably add a caveat to the above: “In times of chaos & disruption, at least.”*
*NEW EDIT: After reading three giant history books on the subject, I take back my previous edit. My original claims were correct.*

### **Three shocking true stories**

I highly recommend you read the wiki pages yourself; otherwise, here are my summaries:

#### **Cortés:**[**[wiki]**](https://en.wikipedia.org/wiki/Fall_of_Tenochtitlan)[**[wiki]**](https://en.wikipedia.org/wiki/Spanish_conquest_of_the_Aztec_Empire)

* April 1519: Hernán Cortés lands in Yucatan with ~500 men, 13 horses, and a few cannons. He destroys his ships so his men won’t be able to retreat. His goal is to conquer the Aztec empire of several million people.
* He makes his way towards the imperial capital, Tenochtitlán. Along the way he encounters various local groups, fighting some and allying with some. He is constantly outnumbered but his technology gives him an advantage in fights. His force grows in size, because even though he loses Spaniards he gains local allies who resent Aztec rule.
* Tenochtitlán is an island fortress (like Venice) with a population of over 200,000, making it one of the largest and richest cities *in the world* at the time. Cortés arrives in the city asking for an audience with the Emperor, who receives him warily.
* Cortés takes the emperor hostage within his own palace, indirectly ruling Tenochtitlán through him.
* Cortés learns that the Spanish governor has landed in Mexico with a force twice his size, intent on arresting him. (Cortés’ expedition was illegal!) Cortés leaves 200 men guarding the Emperor, marches to the coast with the rest, surprises and defeats the new Spaniards in battle, and incorporates the survivors into his army.
* July 1520: Back at the capital, the locals are starting to rebel against his men. Cortés marches back to the capital, uniting his forces just in time to be besieged in the imperial palace. They murder the emperor and fight their way out of the city overnight, taking heavy losses.
* They shelter in another city (Tlaxcala) that was thinking about rebelling against the Aztecs. Cortés allies with the Tlaxcalans and launches a general uprising against the Aztecs. Not everyone sides with him; many city-states remain loyal to Tenochtitlan. Some try to stay neutral. Some join him at first, and then abandon him later. Smallpox sweeps through the land, killing many on all sides and causing general chaos.
* May 1521: The final assault on Tenochtitlán. By this point, Cortés has about 1,000 Spanish troops and 80,000 – 200,000 allied native warriors. He had 16 cannons and 13 boats. The Aztecs have 80,000 – 300,000 warriors and 400 boats. Cortés and his allies win.
* Later, the Spanish would betray their native allies and assert hegemony over the entire region, in violation of the treaties they had signed.

#### **Pizarro**[**[wiki]**](https://en.wikipedia.org/wiki/Francisco_Pizarro)[**[wiki]**](https://en.wikipedia.org/wiki/Spanish_conquest_of_Peru)

* 1532: Francisco Pizarro arrives in Inca territory with 168 Spanish soldiers. His goal is to conquer the Inca empire, which was much bigger than the Aztec empire.
* The Inca empire is in the middle of a civil war and a devastating plague.
* Pizarro makes it to the Emperor right after the Emperor defeats his brother. Pizarro is allowed to approach because he promises that he comes in peace and will be able to provide useful information and gifts.
* At the meeting, Pizarro ambushes the Emperor, killing his retinue with a volley of gunfire and taking him hostage. The remainder of the Emperor’s forces in the area back away, probably confused and scared by the novel weapons and hesitant to keep fighting for fear of risking the Emperor’s life.
* Over the next months, Pizarro is able to leverage his control over the Emperor to stay alive and order the Incans around; eventually he murders the Emperor and makes an alliance with local forces (some of the Inca generals) to take over the capital city of Cuzco.
* The Spanish continue to rule via puppets, primarily Manco Inca, who is their puppet ruler while they crush various rebellions and consolidate their control over the empire. Manco Inca escapes and launches a rebellion of his own, which is partly successful: He utterly wipes out four columns of Spanish reinforcements, but is unable to retake the capital. With the morale and loyalty of his followers dwindling, Manco Inca eventually gives up and retreats, leaving the Spanish still in control.
* Then the Spanish ended up fighting *each other* for a while, while *also* putting down more local rebellions. After a few decades Spanish dominance of the region is complete. (1572).

#### **Afonso**[**[wiki]**](https://en.wikipedia.org/wiki/Afonso_de_Albuquerque)[**[wiki]**](https://en.wikipedia.org/wiki/Capture_of_Malacca_(1511))[**[wiki]**](https://en.wikipedia.org/wiki/Portuguese_conquest_of_Goa)

* 1506: Afonso helps the Portuguese king come up with a shockingly ambitious plan. *Eight years* prior, the first Europeans had rounded the coast of Africa and made it to the Indian Ocean. The Indian Ocean contained most of the world’s trade at the time, since it linked up the world’s biggest and wealthiest regions. See [this map of world population (timestamp 3:45)](https://www.youtube.com/watch?v=PUwmA3Q0_OE). Remember, this is prior to the Industrial and Scientific Revolutions; Europe is just coming out of the Middle Ages and does not have an obvious technological advantage over India or China or the Middle East, and has an obvious economic *disadvantage*. And Portugal is a just tiny state on the edge of the Iberian peninsula.
* The plan is: Not only will we go into the Indian Ocean and participate in the trading there — cutting out all the middlemen who are currently involved in the trade between that region and Europe — we will *conquer strategic ports around the region so that no one else can trade there!*
* Long story short, Afonso goes on to complete this plan by 1513. (!!!)

Some comparisons and contrasts:

* Afonso had more European soldiers at his disposal than Cortes or Pizarro, but not many more — usually he had about a thousand or so. He did have more reinforcements and support from home.
* Like them, he was usually significantly outnumbered in battles. Like them, the empires he warred against were vastly wealthier and more populous than his forces.
* Like them, Afonso was often able to exploit local conflicts to gain local allies, which were crucial to his success.
* Unlike them, his goal wasn’t to conquer the empires entirely, just to get and hold strategic ports.
* Unlike them, he was fighting empires that were technologically advanced; for example, in several battles his enemies had more cannons and gunpowder than he did.
* That said, it does seem that Portuguese technology was qualitatively better in some respects (ships, armor, and cannons, I’d say.) Not dramatically better, though.
* While Afonso’s was a naval campaign, he did fight many land battles, usually marine assaults on port cities, or defenses of said cities against counterattacks. So superior European naval technology is not by itself enough to explain his victory, though it certainly was important.
* Plague and civil war were not involved in Afonso’s success.

### **What explains these devastating conquests?**

#### **Wrong answer: I cherry-picked my case studies.**

History is full of incredibly successful conquerors: Alexander the Great, Ghenghis Khan, etc. Perhaps some people are just really good at it, or really lucky, or both.

However: Three incredibly successful conquerors from the same tiny region and time period, conquering three separate empires? Followed up by dozens of less successful but still very successful conquerors from the same region and time period? Surely this is not a coincidence. Moreover, it’s not like the conquistadors had many failed attempts and a few successes. The Aztec and Inca empires were the two biggest empires in the Americas, and there weren’t any other Indian Oceans for the Portuguese to fail at conquering.

Fun fact: I had not heard of Afonso before I started writing this post this morning. Following the [Rule of Three](https://en.wikipedia.org/wiki/Rule_of_three_%28writing%29), I needed a third example and I predicted on the basis of Cortes and Pizarro that there would be other, similar stories happening in the world at around that time. That’s how I found Afonso.

#### **Right answer: Technology**

However, I don’t think this is the whole explanation. The technological advantage of the conquistadors was not overwhelming.

Whatever technological advantage the conquistadors had over the existing empires, it was the sort of technological advantage that one could acquire *before* the Scientific and Industrial revolutions. Technology didn’t change very fast back then, yet Portugal managed to get a lead over the Ottomans, Egyptians, Mughals, etc. that was sufficient to bring them victory. On paper, the Aztecs and Spanish were pretty similar: Both were medieval, feudal civilizations. I don’t know for sure, but I’d bet there were at least a few techniques and technologies the Aztecs had that the Spanish didn’t. And of course the technological similarities between the Portuguese and their enemies were much stronger; the Ottomans even had access to European mercenaries! Even in cases in which the conquistadors had technology that was completely novel — like steel armor, horses, and gunpowder were to the Aztecs and Incas — it wasn’t god-like. The armored soldiers were still killable; the gunpowder was more effective than arrows but limited in supply, etc.

(Contrary to popular legend, neither Cortés nor Pizarro were regarded as gods by the people they conquered. The Incas concluded pretty early on that the Spanish were mere men, and while the idea did float around the Aztecs for a bit the modern historical consensus is that most of them didn’t take it seriously.)

Ask yourself: Suppose Cortés had found 500 local warriors, gave them all his equipment, trained them to use it expertly, and left. Would those local men have taken over all of Mexico? I doubt it. And this is despite the fact that they would have had much better local knowledge than Cortés did! Same goes for Pizarro and Afonso. Perhaps if he had found 500 local warriors *led by an exceptional commander* it would work. But the explanation for the conquistador’s success can’t just be that they were all exceptional commanders; that would be positing too much innate talent to occur in one small region of the globe at one time.

#### **Right answer: Strategic and diplomatic cunning**

This is my non-expert guess about the missing factor that joins with technology to explain this pattern of conquistador success.

They didn’t just have technology; they had *effective* *strategy* and they had *effective diplomacy*. They made long-term plans that *worked* despite being breathtakingly ambitious. (And their short-term plans were usually pretty effective too, read the stories in detail to see this.) Despite not knowing the local culture or history, these conquistadors made surprisingly savvy diplomatic decisions. They knew when they could get away with breaking their word and when they couldn’t; they knew which outrages the locals would tolerate and which they wouldn’t; they knew how to convince locals to ally with them; they knew how to use words to escape militarily impossible situations… The locals, by contrast, often badly misjudged the conquistadors, e.g. not thinking Pizarro had the will (or the ability?) to kidnap the emperor, and thinking the emperor would be safe as long as they played along.

This raises the question, how did they get that advantage? My answer: they had *experience* with this sort of thing, whereas locals didn’t. Presumably Pizarro learned from Cortés’ experience; his strategy was pretty similar. (See also: [the prior conquest of the Canary Islands by the Spanish](https://en.wikipedia.org/wiki/Conquest_of_the_Canary_Islands)). In Afonso’s case, well, the Portuguese had been sailing around Africa, conquering ports and building forts for more than a hundred years.

### **Lessons I think we learn**

I think we learn that:

It is occasionally possible for a small force to quickly conquer large parts of the world, despite:

1. Having only a minuscule fraction of the world’s resources and power
2. Having technology + diplomatic and strategic cunning that is better but not *that* much better
3. Having very little data about the world when the conquest begins
4. Being disunited

Which all suggests that it isn’t as implausible that a small AI takes over the world in mildly favorable circumstances as is sometimes thought.

 *EDIT: In light of good pushback from people (e.g.* [*Lucy.ea8*](https://forum.effectivealtruism.org/posts/MNPrXCsPpwTgygMxc/cortes-pizarro-and-afonso-as-precedents-for-takeover#xZ8h23jc9pmp8Mqhy) *and*  [*Matthew Barnett)*](https://www.alignmentforum.org/posts/ivpKSjM4D6FbqF4pZ/cortes-pizarro-and-afonso-as-precedents-for-takeover#kNFNjBJjuzTd3irrR) *about the importance of disease, I think one should probably add a caveat to the above: “In times of chaos & disruption, at least.”*

#### **Having only a minuscule fraction of the world’s resources and power**

In all three examples, the conquest was more or less completed without support from home; while Spain/Portugal did send reinforcements, it wasn’t even close to the entire nation of Spain/Portugal fighting the war. So these conquests are examples of non-state entities conquering states, so to speak. (That said, their *claim* to represent a large state may have been crucial for Cortes and Pizarro getting audiences and respect initially.) Cortés landed with about a thousandth the troops of Tenochtitlan, which controlled a still larger empire of vassal states. Of course, his troops were better equipped, but on the other hand they were also cut off from resupply, whereas the Aztecs were in their home territory, able to draw on a large civilian population for new recruits and resupply.

The conquests succeeded in large part due to diplomacy. This has implications for AI takeover scenarios; rather than imagining a conflict of humans vs. robots, we could imagine humans vs. humans-with-AI-advisers, with the latter faction winning and somehow by the end of the conflict the AI advisers have managed to become *de facto* rulers, using the humans who obey them to put down rebellions by the humans who don’t.

#### **Having technology + diplomatic and strategic skill that is better but not** ***that*** **much better**

As previously mentioned, the conquistadors didn’t enjoy god-like technological superiority. In the case of Afonso the technology was pretty similar. Technology played an important role in their success, but it wasn’t enough on its own. Meanwhile, the conquistadors may have had more diplomatic and strategic cunning (or experience) than the enemies they conquered. But not that much more–they are only human, after all. And their enemies were pretty smart.

In the AI context, we don’t need to imagine god-like technology (e.g. swarms of self-replicating nanobots) to get an AI takeover. It might even be possible without any new physical technologies at all! Just superior software, e.g. piloting software for military drones, targeting software for anti-missile defenses, cyberwarfare capabilities, data analysis for military intelligence, and of course excellent propaganda and persuasion.

Nor do we need to imagine an AI so savvy and persuasive that it can persuade anyone of anything. We just need to imagine it about as cunning and experienced relative to its enemies as Cortés, Pizarro, and Afonso were relative to theirs. (Presumably no AI would be experienced with world takeover, but perhaps an intelligence advantage would give it the same benefits as an experience advantage.) And if I’m wrong about this explanation for the conquistador’s success–if they had no such advantage in cunning/experience–then the conclusion is even stronger.

Additionally, in a rapidly-changing world that is undergoing [slow takeoff](https://sideways-view.com/2018/02/24/takeoff-speeds/), where there are lesser AIs and AI-created technologies all over the place, most of which are successfully controlled by humans, AI takeover might still happen if one AI is better, but not that much better, than the others.

#### **Having very little data about the world when the conquest begins**

Cortés invaded Mexico knowing very little about it. After all, the Spanish had only realized the Americas existed two decades prior. He heard rumors of a big wealthy empire and he set out to conquer it, knowing little of the technology and tactics he would face. Two years later, he ruled the place.

Pizarro and Afonzo were in better epistemic positions, but still, they had to learn a lot of important details (like what the local power centers, norms, and conflicts were, and exactly what technology the locals had) on the fly. But they were good at learning these things and making it up as they went along, apparently.

We can expect superhuman AI to be good at learning. Even if it starts off knowing very little about the world — say, it figured out it was in a training environment and hacked its way out, having inferred a few general facts about its creators but not much else — if it is good at learning and reasoning, it might still be pretty dangerous.

#### **Being disunited**

Cortés invaded Mexico in defiance of his superiors and had to defeat the army they sent to arrest him. Pizarro ended up fighting a civil war against his fellow conquistadors in the middle of his conquest of Peru. Afonzo fought Greek mercenaries and some traitor Portuguese, conquered Malacca against the  orders of a rival conquistador in the area, and was ultimately demoted due to political maneuvers by rivals back home.

This astonishes me. Somehow these conquests were completed by people who were at the same time busy infighting and backstabbing each other!

Why was it that the conquistadors were able to split the locals into factions, ally with some to defeat the others, and end up on top? Why didn’t it happen the other way around: some ambitious local ruler talks to the conquistadors, exploits their internal divisions, allies with some to defeat the others, and ends up on top?

I think the answer is partly the “diplomatic and strategic cunning” mentioned earlier, but mostly other things. (The conquistadors were disunited, but presumably were united in the ways that mattered.) At any rate, I expect AIs to be pretty [good at coordinating too](https://www.alignmentforum.org/posts/gYaKZeBbSL4y2RLP3/strategic-implications-of-ais-ability-to-coordinate-at-low); they should be able to conquer the world just fine even while competing fiercely with each other. For more on this idea, see [this comment](https://www.lesswrong.com/posts/gYaKZeBbSL4y2RLP3/strategic-implications-of-ais-ability-to-coordinate-at-low?commentId=pdXk8xKtHQKAiu8Pf).

*By Daniel Kokotajlo*

**Acknowledgements**
--------------------

*Thanks to Katja Grace for feedback on a draft. All mistakes are my own, and should be pointed out to me via email at daniel@aiimpacts.org. Edit: Also, when I wrote this post I had forgotten that the basic idea for it probably came from [this comment by JoshuaFox](https://www.lesswrong.com/posts/vkjWGJrFWBnzHtxrw/superintelligence-7-decisive-strategic-advantage#sWjzvXJToQ3KfH7pB).*

(Front page image [from the Conquest of México series. Representing the 1521 Fall of Tenochtitlan, in the Spanish conquest of the Aztec Empire](https://en.wikipedia.org/wiki/Fall_of_Tenochtitlan#/media/File:The_Conquest_of_Tenochtitlan.jpg))

This post lists three historical examples of how small human groups conquered large parts of the world, and shows how they are arguably precedents for AI takeover scenarios. The first two historical examples are the conquests of American civilizations by Hernán Cortés and Francisco Pizarro in the early 16th century. The third example is the Portugese capture of key Indian Ocean trading ports, which happened at roughly the same time as the other conquests. Daniel argues that technological and strategic advantages were the likely causes of these European victories. However, since the European technological advantage was small in this period, we might expect that an AI coalition could similarly take over a large portion of the world, even without a large technological advantage.

In a [comment](https://www.alignmentforum.org/posts/ivpKSjM4D6FbqF4pZ/cortes-pizarro-and-afonso-as-precedents-for-takeover?commentId=kNFNjBJjuzTd3irrR), I dispute the claimed reasons for why Europeans conquered American civilizations. I think that a large body of historical literature supports the conclusion that American civilizations fell primarily because of their exposure to diseases which they lacked immunity to, rather than because of European military power. I also think that this helps explain why Portugal was "only" able to capture Indian Ocean trading ports during this time period, rather than whole civilizations. I think the primary insight here should instead be that pandemics can kill large groups of humans, and therefore it would be worth exploring the possibility that AI systems use pandemics as a mechanism to kill large numbers of biological humans.


Incomplete case studies of discontinuous progress

*Published 7 Feb 2020*

This is a list of potential cases of discontinuous technological progress that we have investigated partially or not at all.

List
----

In the course of [investigating cases of potentially discontinuous technological progress](http://aiimpacts.org/discontinuous-progress-investigation/), we have collected around fifty suggested instances that we have not investigated fully. This is a list of them and what we know about them.

### The Haber Process

This was previously listed as NSD, but that is tentatively revoked while we investigate a complication with the data.

##### Previous explanation

The Haber process was [the first energy efficient method](http://en.wikipedia.org/wiki/Haber_process) of producing ammonia, which is key to making fertilizer. The reason to expect that the Haber process might represent discontinuous technological progress is that previous processes were barely affordable, while the Haber process was hugely valuable—it is [credited](http://en.wikipedia.org/wiki/Haber_process#Economic_and_environmental_aspects) with fixing much of the nitrogen now in human bodies—and has been used on an industrial scale since [1913](http://en.wikipedia.org/wiki/Haber_process#History).

A likely place to look for discontinuities then is in the energy cost of fixing nitrogen. [Table 4](https://books.google.com/books?id=NDUXBQAAQBAJ&pg=PA74&dq=energy+efficiency+haber+birkeland+frank&hl=en&sa=X&ei=dG61VLm7IonroATIyoCADg&ved=0CC0Q6AEwAA#v=onepage&q=energy%20efficiency%20haber%20birkeland%20frank&f=false) in Grünewald’s Chemistry for the Future suggests that the invention of the Haber reduced the energy expense by around 60% per nitrogen bonded over a method developed eight years earlier. The previous step however appears to have represented at least a 50% improvement over the process of two years earlier (though the figure is hard to read). Later improvements to the Haber process appear to have been comparable. Thus it seems the Haber process was not an unusually large improvement in energy efficiency, but was probably instead the improvement that happened to take the process into the range of affordability.

Since it appears that energy was an important expense, and the Haber process was especially notable for being energy efficient, and yet did not represent a particular discontinuity in energy efficiency progress, it seems unlikely that the Haber process involved a discontinuity. Furthermore, it appears that the world moved to using the Haber process over other sources of fertilizer gradually, suggesting there was not a massive price differential, nor any sharp practical change as a result of the adoption of the process. In the 20’s the US [imported](http://pubs.usgs.gov/of/2004/1290/2004-1290.pdf)[![](chrome-extension://gmpljdlgcdkljlppaekciacdmdlhfeon/images/beside-link-icon.svg)](https://docs.google.com/viewer?url=http%3A%2F%2Fpubs.usgs.gov%2Fof%2F2004%2F1290%2F2004-1290.pdf&embedded=true&chrome=false&dov=1 "View this pdf file")[![](chrome-extension://gmpljdlgcdkljlppaekciacdmdlhfeon/images/beside-link-icon.svg)](https://docs.google.com/viewer?url=http%3A%2F%2Fpubs.usgs.gov%2Fof%2F2004%2F1290%2F2004-1290.pdf&embedded=true&chrome=false&dov=1 "View this pdf file")[![](chrome-extension://gmpljdlgcdkljlppaekciacdmdlhfeon/images/beside-link-icon.svg)](https://docs.google.com/viewer?url=http%3A%2F%2Fpubs.usgs.gov%2Fof%2F2004%2F1290%2F2004-1290.pdf&embedded=true&chrome=false&dov=1 "View this pdf file") much nitrogen from Chile. Alternative nitrogen source calcium cyanamide [reached](http://www.acs.org/content/acs/en/education/whatischemistry/landmarks/calciumcarbideacetylene.html) peak production in 1945, thirty years since the Haber process reached industrial scale production.

The amount of synthetic nitrogen fertilizer applied hasn’t abruptly changed since 1860 (see [p24](http://www.slideshare.net/ILRI/ilri-lse-seminardavidson2014)). Neither has the amount of food produced, for [a few](http://gardenearth.blogspot.com/2013/08/chicken-and-fertilizers.html) [foods](http://www.washingtonpost.com/blogs/wonkblog/wp/2012/08/16/a-brief-history-of-u-s-corn-in-one-chart/) at least.

In sum, it seems the Haber process has had a large effect, but it was produced by a moderate change in efficiency, and manifest over a long period.

### Aluminium

It is sometimes claimed that the discovery of the Hall–Héroult process in the 1880s brought the price of aluminium down precipitously. We found several pieces of quantitative data about this, but they seriously conflict. The most rigorous looking is a report from Patricia Plunkert at the US Geological Survey, from which we get the following data. However note that some of these figures may be off by orders of magnitude, according to other sources.

Plunkert [provides a table of historic aluminium prices](http://minerals.usgs.gov/minerals/pubs/commodity/aluminum/050798.pdf), according to which the nominal price fell from $8 per pound to $0.58 per pound sometime between 1887 and 1895 (during most of which time no records are available). This period probably captures the innovation of interest, as the Hall–Héroult process was patented in 1886 according to Plunkert, and the price only dropped by $1 per pound during the preceding fifteen years according to her table. Plunkert also says that the price was held artificially low to encourage consumers in the early 1900s, suggesting the same may have been true earlier, however this seems likely to be a small correction.

### The sewing machine

Early sewing machines apparently brought the time to produce clothing down by an order of magnitude (from 14 hours to 75 minutes for a man’s dress shirt by [one estimate](http://en.wikipedia.org/wiki/Sewing_machine#Social_impact)). However [it appears](http://www.sewalot.com/sewing_machine_history.htm) that the technology progressed more slowly, then was taken up by the public later – probably when it became cost-effective, at which time adoptees may have experienced a rapid reduction in sewing time (presumably at some expense). These impressions are from a very casual perusal of the evidence.

### Video compression

Blogger John McGowan [claims](https://mathblog.com/martin-fowlers-design-stamina-hypothesis-and-video-compression/) that video compression performance was constant at a ratio of around 250 for about seven years prior to 2003, then jumped to around 900.

[![Figure 1: video compression performance in the past two decades.](http://aiimpacts.org/wp-content/uploads/2020/02/VideoCompression-1-300x167.png)](http://aiimpacts.org/wp-content/uploads/2020/02/VideoCompression-1.png)Figure 1: video compression performance in the past two decades.
#### Information storage volume

According to the Performance Curves Database (PCDB), ‘information storage volume’ for both [handwriting](http://pcdb.santafe.edu/graph.php?curve=201) and [printing](http://pcdb.santafe.edu/graph.php?curve=205) has grown by a factor of three in recent years, after less than doubling in the hundred years previously. It is unclear what exactly is being measured here however.

### Undersea cable price

The bandwidth per cable length available for a dollar [apparently](http://pcdb.santafe.edu/graph.php?curve=214) grew by more than 1000 times in around 1880.

### Infrared detector sensitivity

We understand that infrared detector sensitivity is measured in terms of ‘Noise Equivalent Power’ (NEP), or the amount of power (energy per time) that needs to hit the sensor for the sensor’s output to have a signal:noise ratio of one. We investigated progress in infrared detection technology because according to Academic Press (1974), the helium-cooled germanium bolometer represented a four order of magnitude improvement in sensitivity over uncooled detectors.[1](https://aiimpacts.org/incomplete-case-studies-of-discontinuous-progress/#easy-footnote-bottom-1-2267 "‘Following Johnson’s work at shorter wavelengths, photometric systems were established at the University of Arizona for each of the infrared windows from 1 to 25μm. At 5, 10, and 22μm, the helium-cooled germanium bolometer was used. This detector provided four orders of magnitude improvement in sensitivity over uncooled detectors and was utilized at wavelengths out to 1000μm.’ – Academic press, 1974") However our own investigation suggests there were other innovations between uncooled detectors and the bolometer in question, and thus no abrupt improvement.

We list advances we know of [here](https://docs.google.com/spreadsheets/d/1EDmWBL2yk-aYZjNUoKDUG2wZwN6qKdNPOdKors-vShI/edit?usp=sharing), and summarize them in Figure 5. The 1947 point is uncooled. The 1969 point is nearly four orders of magnitude better. However we know of at least four other detectors with intermediate levels of sensitivity, and these are spread fairly evenly between the uncooled device and the most efficient cooled one listed.

We have not checked whether the progress between the uncooled detector and the first cooled detector was discontinuous, given previous rates. This is because we have no strong reason to suspect it is.

[![xxx](http://aiimpacts.org/wp-content/uploads/2015/02/image-11.png)](http://aiimpacts.org/wp-content/uploads/2015/02/image-11.png)Figure 2: Sensitivity of infrared detectors during the transition to liquid-helium-cooled devices.
### Genome sequencing – IIP

This appears to have seen at least a moderate discontinuity. An investigation is in progress.

It was suggested to us in particular that Next Generation Sequencing produced discontinuous progress in output per instrument run for DNA sequencing.

### Aircraft shot down per shell fired

We’ve seen it claimed that the [proximity fuse](https://en.wikipedia.org/wiki/Proximity_fuze) increased this metric by 2x or more. We don’t know what the trend was beforehand, however.

### Time to produce clothing

The Sewing machine was proposed as discontinuous in this metric. We have not investigated.

### Sensitivity of infrared detectors

Cryogenically cooled semiconductor sensors were proposed as discontinuous in this metric. We have not investigated.

Frames per Second
-----------------

It was suggested that something in high sensitivity, high precision metrology, e.g. trillion frame-per-second camera from MIT, would be discontinuous in this metric. We have not investigated.

Access to Information
---------------------

Smart phones were suggested as a discontinuity in this metric. We have not investigated.

Spread of minimally invasive surgery
------------------------------------

Laparoscopic cholecystectomy was suggested as a discontinuity in this metric. We have not investigated.

Max. submerged endurance, submerged runs
----------------------------------------

Nuclear-powered submarines may be a discontinuity in this metric. We have not investigated.

Clothmaking efficiency
----------------------

The Jacquard Loom and the Spinning Jenny were suggested as discontinuities in this metric. We have not investigated.

Personal armor protectiveness-to-weight ratio
---------------------------------------------

Kevlar was suggested as discontinuities in this metric. We have not investigated.

Lumens per watt
---------------

High pressure sodium lamps were suggested as discontinuities in this metric. We have not investigated.

Linear programming
------------------

The Simplex algorithm was suggested as discontinuities in this metric. We have not investigated.

Fourier transform speed
-----------------------

Fast fourier transform was suggested as discontinuities in this metric. We have not investigated.

Polynomial identity testing efficiency
--------------------------------------

Probabilistic testing methods were suggested as discontinuities in this metric. We have not investigated.

Audio compression efficiency
----------------------------

The MP3 format was suggested as discontinuities in this metric. We have not investigated.

Crop yields
-----------

[This amazing genetic modification](https://www.igb.illinois.edu/article/scientists-engineer-shortcut-photosynthetic-glitch-boost-crop-growth-40), if it works as claimed, may well be a discontinuity in this metric. We have not investigated.

*Thanks to Stephen Jordan, Bren Worth, Finan Adamson and others for suggesting potential discontinuities in this list.*


Effect of AlexNet on historic trends in image recognition

AlexNet did not represent a greater than 10-year discontinuity in fraction of images labeled incorrectly, or log or inverse of this error rate, relative to progress in the past two years of competition data.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

The annual ImageNet competition asks researchers to build programs to label images.[1](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/#easy-footnote-bottom-1-1382 "“Since 2010, the ImageNet project runs an annual software contest, the ImageNet Large Scale Visual Recognition Challenge (ILSVRC), where software programs compete to correctly classify and detect objects and scenes.” – “Imagenet”. 2019. En.Wikipedia.Org. Accessed June 20 2019. https://en.wikipedia.org/w/index.php?title=ImageNet&oldid=900080629.") It began in 2010, when every team labeled at least 25% of images wrong. The same was true in 2011, and would have been true in 2012, if not for AlexNet, a [convolutional neural network](https://en.wikipedia.org/wiki/Convolutional_neural_network) that mislabeled only 16.4% of images.[2](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/#easy-footnote-bottom-2-1382 "See our data section for sources for this data.")
### Trends

#### Percent of images mislabeled

##### Data

We collected data on the error rate (%) of the 2010 – 2012 ImageNet competitors from Table 6 of Russakovsky et al[3](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/#easy-footnote-bottom-3-1382 "Olga Russakovsky et al., “ImageNet Large Scale Visual Recognition Challenge,” International Journal of Computer Vision 115, no. 3 (December 1, 2015): 211–52, https://doi.org/10.1007/s11263-015-0816-y.") into [this spreadsheet](https://docs.google.com/spreadsheets/d/1HYdv4gLdtwkzYKeXaBJTXBbqeX_9onmw4aAaYVWVUfs/edit?usp=sharing). See Figure 1 below.

![](https://aiimpacts.org/wp-content/uploads/2020/02/ErrorRate.png)Figure 1: Error rate (%) of ImageNet competitors from 2010 – 2012
##### Discontinuity measurement

The ImageNet competition had only been going for two years when AlexNet entered, so the past trend is very short. Given this, the shape of the curve prior to AlexNet is entirely ambiguous. We treat the trend as linear for simplicity, but given that, it is better to choose a transformation of the data that we expect to be linear, given our understanding of the situation.

Two plausible transformations are the log of the error, and the reciprocal of the error rate.[4](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/#easy-footnote-bottom-4-1382 "Percentage of answers incorrect seems unlikely to change linearly over time, since we expect moving from 50% incorrect to 49% incorrect to be easier than halving a 2% error rate. Log of the error rate and inverse of the error rate seem to us more plausible.") These two transformations of the data are shown in Figures 2 and 3 below.

![](https://lh3.googleusercontent.com/D9XDAoRH9tDVn8usb1XzCUsOdNY_-cNssrBHe1sKeXMbj5fT1wNcojVsoY7ydgpkU_s1mC9vPHvNV2nSWZufkr48fnmKjBonMxyK06Tk1VNoUpgSZzzJIt19TT22u5gUQMV76hfV)Figure 2: Log base 2 / error rate of ImageNet competitors from 2010 – 2012

![](https://lh5.googleusercontent.com/a-EUqvmgOprtUu--eoK-ZAkdNTtYkE9Mo9Ir-e219QsAWeOQAcfwwqsWv_Xsi-zDJ8vGVkKbeBRShPL4IbCP6zt2GWzkj3YM4AMbpMfrD-0_tfyq2jasOp5ZXq_q7ooaRVT5lfDx)Figure 3: 1 / error rate of ImageNet competitors from 2010 – 2012
The best 2012 AlexNet competitor gives us discontinuous jumps of 3 years of progress at previous rates for the raw error rate, 4 years of progress at previous rates for log base 2 of the error rate, or 6 years of progress at previous rates for 1 / the error rate.[5](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/#easy-footnote-bottom-5-1382 "See our methodology page for more details our spreadsheet for calculation.") For the 6-year discontinuity, we tabulated a number of other potentially relevant metrics in the ‘Notable discontinuities under 10 years’ tab **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.

**Notes**
---------


Historic trends in transatlantic message speed

The speed of delivering a short message across the Atlantic Ocean saw at least three discontinuities of more than ten years before 1929, all of which also were more than one thousand years: a 1465-year discontinuity from Columbus’ second voyage in 1493, a 2085-year discontinuity from the first telegraph cable in 1858, and then a 1335-year discontinuity from the second telegraph cable in 1866.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

#### **Summary of historic developments**

All communications between Europe and North America were carried on ships until 1858, when the first telegraph messages were transmitted over cable between the UK and US.[1](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-1-1869 "” Exactly 150 years ago, August 1858, the world witnessed a historic event in the history of telecommunications: the successful transmission of telegraph messages across the Atlantic Ocean. Although the transatlantic cable carrying these messages failed after a few weeks of operation, and it wasn’t until 1866 that permanent transatlantic telegraph cable transmission became possible, the 1858 transmissions were heralded worldwide as a major achievement, introducing a new Age of Information. ”  Schwartz, Mischa. “History of Communications.” IEEE Communications Magazine, vol. 46, no. 8, 2008, pp. 26–29., doi:10.1109/mcom.2008.4597099. ") That first cable only lasted six weeks, and took more than sixteen hours to send a message from the Queen.[2](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-2-1869 "“The celebrations hit a sour note as the fireworks set fire to City Hall. Far worse news was to come, as the cable itself failed completely after six weeks. The cable never really worked well; the Queen’s message had taken 16-1/2 hours to transmit.”  Schwartz, Mischa. “History of Communications.” IEEE Communications Magazine, vol. 46, no. 8, 2008, pp. 26–29., doi:10.1109/mcom.2008.4597099. ")
A permanent cable wasn’t laid until eight years later.[3](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-3-1869 "“…and it wasn’t until 1866 that permanent transatlantic telegraph cable transmission became possible”

 Schwartz, Mischa. “History of Communications.” IEEE Communications Magazine, vol. 46, no. 8, 2008, pp. 26–29., doi:10.1109/mcom.2008.4597099. ") Better telegraph cables were laid a further thirty and sixty years later. We do not investigate developments after 1929.

![](https://lh3.googleusercontent.com/6SFwnXQLdGXDZhhsykqyd4lU_glcTRVhNvr93xsUmFuBek77Id4tFd2qaFD1hSVDFB_-HMlkR045N7OhuQuiPwQGBJ9H-0bqw8stn8Q6g4tf-3QYebeBuRt8CGYQtACzLpVtLfYp)Figure 1: Undersea communications cables became common in the long run: map of undersea communications cables in 2007.[4](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-4-1869 "From Wikimedia Commons:
Rarelibra [Public domain] ")
### **Trends**

#### Transatlantic message speed, 140 character message

We looked at historic times to send messages across the Atlantic Ocean.

Message speed can depend on the length of the message. Where this was relevant, we somewhat arbitrarily chose to investigate for a 140 character message. We measure fastest speeds of real historic systems that could send 140 character messages across the Atlantic Ocean. We do not require that a 140 character message was actually sent by the method in question.

We generally use whatever route was actually taken (or supposed in an estimate), and do not attempt to infer faster speeds possible had an optimal route been taken (though note that because we are measuring speed rather than time to cross the Ocean, route length is adjusted for to a first approximation).

We only investigated this metric from 1492-1493 and 1841-1928. We do not investigate 1493-1841 because our data is insufficiently complete to determine how continuous it was.[5](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-5-1869 "See historic trends in transatlantic passenger travel for discussion of this.")
##### Data

Our data for message speed came from a variety of online sources, and has not been thoroughly vetted. The full dataset with sources can be found [here.](https://docs.google.com/spreadsheets/d/11WV8JUIZeVNWKfggCmHaCEJrtG_YMiOdq18zjfwFUVk/edit#gid=1115870502)[6](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-6-1869 "The ‘Message’ tab contains data for the speed in knots of different modes of transport for carrying messages, along with the source for each data point.")
Because message delivery coincided with passenger travel until the first telegraph, data until then coincides with that used in our investigation into [historic trends in transatlantic passenger travel](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/).

The resulting trend is shown in Figures 2-3.

![](https://aiimpacts.org/wp-content/uploads/2020/02/MessageZoom-1024x768.png) Figure 2: Average speed for message transmission across the Atlantic in recent centuries (see Figure 3 for longer term trend)
![ Figure 3: Average speed for message transmission across the Atlantic. ](http://aiimpacts.org/wp-content/uploads/2020/02/Message-1024x768.png) Figure 3: Average speed for message transmission across the Atlantic.
##### Discontinuity Measurement

We measure discontinuities by comparing progress made at a particular time to the past trend. For this purpose, we treat the past trend at any given point as exponential or linear depending on apparent fit, and judge a new trend to have begun when the recent trend has diverged sufficiently from the longer term trend. See [our spreadsheet](https://docs.google.com/spreadsheets/d/11WV8JUIZeVNWKfggCmHaCEJrtG_YMiOdq18zjfwFUVk/edit?usp=sharing), tab ‘Message’ to view the trends, and [our methodology page](https://aiimpacts.org/methodology-for-discontinuity-investigation/#time-period-selection) for details on how to interpret our sheets and how we divide data into trends.

Given these judgments about past progress, there were three discontinuities of more than ten years, all of which were more than one thousand years: a 1465-year discontinuity from Columbus’ second voyage in 1493, then a 2085-year discontinuity from the first telegraph cable in 1858, and then a 1335-year discontinuity from the improved telegraph cable in 1866.[7](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-7-1869 "See our methodology page for more details, and our spreadsheet, tab ‘Message’ for our calculation.") In addition to the size of these discontinuity in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[8](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/#easy-footnote-bottom-8-1869 "See our methodology page for more details.").

##### Discussion of causes

 Transatlantic message speed is a narrower metric than overall message speed, precluding some technologies that can only deliver messages short distances or on land (e.g. the [semaphore telegraph](https://en.wikipedia.org/wiki/Telegraphy#Early_signalling), which relied on a series of towers within line of sight). We expected this would make discontinuity more likely.

Notes
-----