Likelihood of discontinuous progress around the development of AGI

We aren’t convinced by any of the arguments we’ve seen to expect large discontinuity in AI progress above the extremely low base rate for all technologies. However this topic is controversial, and many thinkers on the topic disagree with us, so we consider this an open question.

Details
-------

### Definitions

We say a technological discontinuity has occurred when a particular technological advance pushes some progress metric substantially above what would be expected based on extrapolating past progress. We measure the size of a discontinuity in terms of how many years of past progress would have been needed to produce the same improvement. We use judgment to decide how to extrapolate past progress.

For instance, in the following trend of progress in chess AI performance, we would say that there was a discontinuity in 2007, and it represented a bit over five years of progress at previous rates.

[![](http://aiimpacts.org/wp-content/uploads/2014/12/SSDF-progress-300x247.png)](http://aiimpacts.org/wp-content/uploads/2014/12/SSDF-progress.png)Figure 1: Machine chess progress, in particular SSDF records.
### Relevance

Discontinuity by some measure, on the path to AGI, lends itself to:

* A party gaining decisive strategic advantage
* A single important ‘deployment’ event
* Other very sudden and surprising events

Arguably, the first two require some large discontinuity. Thus the importance of planning for those outcomes rests on the likelihood of a discontinuity.

### Outline

We investigate this topic in two parts. First, with no particular knowledge of AGI as a technology, how likely should we expect a particular discontinuity to be? We take the answer to be quite low. Second, we review arguments that AGI is different from other technologies, and lends itself to discontinuity. We currently find these arguments uncompelling, but not decisively so.

### Default chance of large technological discontinuity

Discontinuities larger than around ten years of past progress in one advance seem to be rare in technological progress on natural and desirable metrics.[1](https://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/#easy-footnote-bottom-1-1086 "We expect discontinuities can be found in the following areas, but these do not seem interesting:

convoluted metrics designed to capture discontinuities, and not likely to be considered a natural measure of success

metrics that with additional details that aren’t necessarily desirable, e.g. we expect ‘Xootr scooter miles/$’ to be more discontinuous than ‘scooter miles/$’, and for that to be more discontinuous than ‘transport miles/$’. Similarly, the day an update to OSX is launched, the number of users of that particular operating system presumably grows rapidly, but the number of users of operating systems does not, and nor does any natural operating system performance metric that we know of.

metrics that nobody is trying to perform well on e.g. number of eight-foot tall unicorns made of marshmallow might undergo a sudden increase if some particular rich person becomes interested. This is uninteresting to us in part because it doesn’t represent a change in the underlying technology, and in part because by its nature if anybody cared how many such unicorns there were the metric wouldn’t have been discontinuous.

") [We have verified](http://aiimpacts.org/cases-of-discontinuous-technological-progress/) around five examples, and know of several other likely cases, though have not completed this [investigation](http://aiimpacts.org/discontinuous-progress-investigation/).

This does not include the discontinuities when metrics initially go from zero to a positive number. For instance, the metric ‘number of Xootr scooters in the world’ presumably went from zero to one on the first day of production, though this metric had seen no progress before. So on our measure of discontinuity size, this was infinitely many years of progress in one step. It is rarer for a broader metric (e.g. ‘scooters’) to go from zero to one, but it must still happen occasionally. We do not mean to ignore this type of phenomenon, but just to deal with it separately, since discontinuity in an ongoing progress curve seems to be quite rare, whereas the beginning of positive progress on a metric presumably occurs once for every metric that begins at zero.

### Proposed reasons to expect a discontinuity near AGI

We have said that the base rate of large technological discontinuities appears to be low. We now review and examine arguments that AGI is unusually likely to either produce a discontinuity, or follow one.

#### Hominid variation

##### Argument

Humans are vastly more successful in certain ways than other hominids, yet in evolutionary time, the distance between them is small. This suggests that evolution induced discontinuous progress in returns, if not in intelligence itself, somewhere approaching human-level intelligence. If evolution experienced this, this suggests that artificial intelligence research may do also.

##### Counterarguments

Evolution does not appear to have been optimizing for the activities that humans are good at. In terms of what evolution was optimizing for—short term ability to physically and socially prosper—it is quite unclear that humans are disproportionately better than earlier hominids, compared to differences between earlier hominids and less optimized animals.

This counterargument implies that if one were to optimize for human-like success that it would be possible (and likely at some point) to make agents with brains the size of chimps, that were humanlike but somewhat worse in their capabilities.

It is also unclear to us that humans succeed so well on something like ‘intelligence’ from vastly superior individual intelligence, rather than vastly superior group intelligence, via innovations such as language.

##### Status

See status section of ‘brain scaling’ argument—they are sufficiently related to be combined.

#### Brain scaling

##### Argument

If we speculate that the main relevant difference between us and our hominid relatives is brain size, then we can argue that brain size in particular appears to produce very fast increase in impressiveness. So if we develop AI systems with certain performance, we should expect to make disproportionately better systems by using more hardware.

Variation in brain size among humans also appears to be correlated with intelligence, and differences in intelligence in the human range anecdotally correspond to large differences in some important capabilities, such as ability to develop new physics theories. So if the relatively minor differences in human brain size cause the differences in intelligence, this would suggest that larger differences in ‘brain size’ possible with computing hardware could lead to quite large differences in some important skills, such as ability to successfully theorize about the world.

##### Counterarguments

This is closely related to the ‘hominids saw discontinuous progress from intelligence’ argument (see above), so shares the counterargument that evolution does not appear to have been optimizing for those things that humans excel at.

Furthermore, if using more hardware generally gets outsized gains, one might expect that by the time we have any particular level of AGI performance, we are already using extremely large amounts of hardware. If this is a cheap axis on which to do better, it would be surprising if we left that value on the table until the end. So the only reason to expect a discontinuity here would seem to be if the opportunity to use more hardware and get outsized results started existing suddenly. We know of no particular reason to expect this, unless there was a discontinuity in something else, for instance if there was indeed one algorithm. That type of discontinuity is discussed elsewhere.

##### Status

This argument seems weak to us currently, but further research could resolve these questions in directions that would make it compelling:

1. Are individual humans radically superior to apes on particular measures of cognitive ability? What are those measures, and how plausible is it that evolution was (perhaps indirectly) optimizing for them?
2. How likely is improvement in individual cognitive ability to account for humans’ radical success over apes? (For instance, compared to new ability to share innovations across the population)
3. How does human brain size relate to intelligence or particular important skills, quantitatively?

#### Intelligence explosion

##### Argument

Intelligence works to improve many things, but the main source of intelligence—human brains—is arguably not currently improved substantially itself. The development of artificial intelligence will change that, by allowing the most intelligent entities to directly contribute to their increased intelligence. This will create a feedback loop where each step of progress in artificial intelligence makes the next steps easier and so faster. Such a feedback loop could potentially go fast. This would not strictly be a discontinuity, in that each step would only be a little faster than the last, however if the growth rate became sufficiently fast sufficiently quickly, it would be much like a discontinuity from our perspective.

This feedback loop is usually imagined to kick in after AI was ‘human-level’ on at least some tasks (such as AI development), but that may be well before it is recognized as human-level in general.

##### Counterarguments

Positive feedback loops are common in the world, and very rarely move fast enough and far enough to become a dominant dynamic in the world. So we need strong reason to expect an intelligence-of-AGI feedback loop to be exceptional, beyond the observation of a potential feedback effect. We do not know of such arguments. However, this is a widely discussed topic, so contenders probably exist.

##### Status

The counterargument currently seems to carry to us. However, we think the argument could be strong if:

1. Strong reason is found to expect an unusually fast and persistent feedback loop
2. Quantitative models of current relationships between the variables hypothesized to form a feedback loop suggest very fast intelligence growth.

#### One algorithm

##### Argument

If a technology is simple, we might expect to discover it in a small number of steps, perhaps one. If it is also valuable, we might expect each of those those steps to represent huge progress in some interesting metric. The argument here is that intelligence—or something close enough to what we mean by intelligence—is like this.

Some reasons to suspect that intelligence is conceptually simple:

1. Evolution discovered it relatively quickly (though see earlier sections on hominid evolution and brain size)
2. It seems conceivable to some that the mental moves required to play Go are basically all the ones you need for a large class of intellectual activities, at least with the addition of some way to navigate the large number of different contexts that are about that hard to think about.
3. It seems plausible that the deep learning techniques we have are already such a thing, and we just need more hardware and adjustment to make it work.

We expect that this list is missing some motivations for this view.

This argument can go in two ways. If we do not have the one algorithm yet, then we might expect that when we find it there will be a discontinuity. If we already have it, yet don’t have AGI, then what remains to be done is probably accruing sufficient hardware to run it. This might traditionally be expected to lead to continuous progress, but it is also sometimes argued to lead to discontinuous progress. Discontinuity in hardware is discussed in a forthcoming section, so here we focus on discontinuity from acquiring the insight.

##### Counterarguments

Society has invented many simple things before. So even if they are unusually likely to produce discontinuities, the likelihood would still seem to be low, since discontinuities are rare. To the extent that you believe that society has not invented such simple things before, you need a stronger reason to expect that intelligence is such a thing, and we do not know of strong reasons.

##### Status

We currently find this argument weak, but would find it compelling if:

1. Strong reason emerged to expect intelligence to be a particularly simple innovation, among all innovations, and
2. We learned that very simple innovations do tend to cause discontinuities, and we have merely failed to properly explore possible past discontinuities well.

#### Deployment scaling

##### Argument

Having put in the research effort to develop one advanced AI system, it is very little further effort to make a large number of them. Thus at the point that a single AGI system is developed, there could shortly thereafter be a huge number of AGI systems. This would seem to represent a discontinuity in something like global intelligence, and also in ‘global intelligence outside of human brains’.

##### Counterarguments

It is the nature of almost every product that large quantities of upfront effort are required to build a single unit, at which point many units can be made more cheaply. This is especially true of other software products, where a single unit can be copied without the need to build production facilities. So this is far from a unique feature of AGI, and if it produced discontinuities, we would see them everywhere.

The reason that the ability to quickly scale the number of instances of a product does not generally lead to a discontinuity seems to be that before the item’s development, there was usually another item that was slightly worse, which there were also many copies of. For instance, if we write a new audio player, and it works well, it can be scaled up across many machines. However on each machine it is only replacing a slightly inferior audio player. Furthermore, because everyone already has a slightly inferior audio player, adoption is gradual.

If we developed human-level AGI software right now, it would be much better than what computing hardware is already being used for, and because of that and other considerations, it may spread to a lot of hardware fast. So we would expect a discontinuity. However if we developed human-level AGI software now, this would already be a discontinuity in other metrics.  In general, it seems to us that this scaling ability may amplify the effects of existing discontinuities, but it is hard to see how it could introduce one.

##### Status

This argument seems to us to rest on a confusion. We do not think it holds unless there is already some other large discontinuity.

#### Train vs. test

##### Argument

There are at least three different processes you can think of computation being used for in the development of AGI:

1. Searching the space of possible software for AGI designs
2. Training specific AGI systems
3. Running a specific AGI system

We tend to think of the first requiring much more computation than the second, which requires much more computation than the third. This suggests that when we succeed at the first, hardware will have to be so readily available that we can quickly do a huge amount of training, and having done the training, we can run vast numbers of AI systems. This seems to suggest a discontinuity in number of well-trained instances of AGI systems.

##### Counterarguments

This seems to be a more specific version of the deployment scaling argument above, and so seems to fall prey to the same counterargument: that whenever you find a good AGI design and scale up the number of systems running it, you are only replacing somewhat worse software, so this is not a discontinuity in any metric of high level capability. For there to be a discontinuity, it seems we would need a case where step 1 returns an AGI design that is much better than usual, or returns a first AGI design that is already very powerful compared to other ways of solving the high level problems that it is directed at. These sources of discontinuity are both discussed elsewhere.

##### Status

This seems to be a variant of the deployment scaling argument, and so similarly weak. However it is more plausible to us here that we have failed to understand some stronger form of the argument, that others find compelling.

#### Starting high

##### Argument

As discussed in Default chance of technological discontinuity above, the first item in a trend always represents a ‘large’ discontinuity in the number of such items—the number has been zero forever, and now it is one, which is more progress at once than in all of history. When we said that discontinuities were rare, we explicitly excluded discontinuities such as these, which appear to be common but uninteresting. So perhaps a natural place to find an AGI-related discontinuity is here.

Discontinuity from zero to one in the number of AGI systems would not be interesting on its own. However perhaps there are other metrics where AGI will represent the first step of progress. For instance, the first roller coaster caused ‘number of roller coasters’ to go from zero to one, but also ‘most joy caused by a roller coaster’ to go from zero to some positive number. And once we admit that inventing some new technology T could cause some initial discontinuity in more interesting trends than ‘number of units of T’, we do not know very much about where or how large such first steps may be (since we were ignoring such discontinuities). So the suggestion is that AGI might see something like this: important metrics going from zero to high numbers, where ‘high’ is relative to social impact (our previous measure of discontinuities being unhelpful in the case where progress was previously flat). Call this ‘starting high’.

Having escaped the presumption of a generally low base rate of interesting discontinuities then, there remain the questions of what the base rate should be for new technologies ‘starting high’ on interesting metrics, and whether we should expect AGI in particular to see this even above the new base rate.

One argument that the base rate should be high is that new technologies have a plethora of characteristics, and empirically some of them seem to start ‘high’, in an intuitive sense. For instance, even if the first plane did not go very far, or fast, or high, it did start out 40 foot wide—we didn’t go through flying machines that were only an atom’s breadth, or only slightly larger than albatrosses. ‘Flying machine breadth’ is not a very socially impactful metric, but if non-socially impactful metrics can start high, then why not think that socially impactful metrics can?

Either way, we might then argue that AGI is especially likely to produce high starting points on socially impactful metrics:

1. AGI will be the first entry in an especially novel class, so it is unusually likely to begin progress on metrics we haven’t seen progress on before.
2. Fully developed AGI would represent progress far above our current capabilities on a variety of metrics. If fully developed AGI is unusually impactful, this suggests that the minimal functional AGI is also unusually impactful.
3. AGI is unusually non-amenable to coming in functional yet low quality varieties. (We do not understand the basis for this argument well enough to relay it. One possibility is that the insights involved in AGI are sufficiently few or simple that having a partial version naturally gets you a fully functional version quickly. This is discussed in another section. Another is that AGI development will see continuous progress in some sense, but go through non-useful precursors—much like ‘half of a word processor’ is not a useful precursor to a word processor. This is also discussed separately elsewhere.)

##### Counterarguments

While it is true that new technologies sometimes represent discontinuities in metrics other than ‘number of X’, these seem very unlikely to be important metrics. If they were important, there would generally be some broader metric that had previously been measured that would also be discontinuously improved. For instance, it is hard to imagine AGI suddenly possessing some new trait Z to a degree that would revolutionize the economy, without this producing discontinuous change in previously measured things like ‘ability to turn resources into financial profit’.

Which is to say, it is not a coincidence that the examples of high starting points we can think of tend to be unimportant. If they were important, it would be strange if we had not previously made any progress on achieving similar good outcomes in other ways, and if the new technology didn’t produce a discontinuity in any such broader goal metrics, then it would not be interesting. (However it could still be the case that we have so far systematically failed to look properly at very broad metrics as they are affected by fairly unusual new technologies, in our search for discontinuities.)

We know of no general trend where technologies that are very important when well developed cause larger discontinuities when they are first begun. We have also not searched for this specifically, but isn’t an immediately apparent trend in the discontinuities we know of.[2](https://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/#easy-footnote-bottom-2-1086 "Roughly, nuclear weapons seem arguably ‘very important’, but high temperature superconductors, rocket cars, and somewhat faster aeroplanes do not. These are the greater than ten year discontinuities we have documented.")
Supposing that an ideal AGI is much smarter than a human, then humans clearly demonstrate the possibility of functional lesser AGIs. (Which is not to say that ‘via human-like deviations from perfection’ is a particularly likely trajectory.) Humans also seem to demonstrate the possibility of developing human-level intelligence, then not immediately tending to have far superior intelligence. This is usually explained by hardware limitations, but seems worth noting nonetheless.

##### Status

We currently think it is unlikely that any technology ‘starts high’ on any interesting metric, and either way AGI does not seem especially likely to do so. We would find this argument more compelling if:

1. We were shown to be confused about the inference that ‘starting high’ on an interesting metric should necessarily produce discontinuity in broader, previously improved metrics. For instance, if a counterexample were found.
2. Historical data was found to suggest that ultimately valuable technologies tended to ‘start high’ or produce discontinuities.
3. Further arguments emerged to expect AGI to be especially likely to ‘start high’.

#### Awesome AlphaZero

##### Argument

We have heard the claim that AlphaZero represented a discontinuity, at least in the ability to play a variety of games using the same system, and maybe on other metrics. If so, this would suggest that similar technologies were unusually likely to produce discontinuities.

##### Counterarguments

Supposing that AlphaZero did represent discontinuity on playing multiple games using the same system, there remains a question of whether that is a metric of sufficient interest to anyone that effort has been put into it. We have not investigated this.

Whether or not this case represents a large discontinuity, if it is the only one among recent progress on a large number of fronts, it is not clear that this raises the expectation of discontinuities in AI very much, and in particular does not seem to suggest discontinuity should be expected in any other specific place.

##### Status

We have not investigated the claims this argument is premised on, or examined other AI progress especially closely for discontinuities. If it turned out that:—

1. Recent AI progress represented at least one substantial discontinuity
2. The relevant metric had been a focus of some prior effort

—then we would consider more discontinuities around AGI more likely. Beyond raising the observed base rate of such phenomena, this would also suggest to us that the popular expectation of AI discontinuity is well founded, even if we have failed to find good explicit arguments for it. So this may raise our expectation of discontinuous change more than the change in base rate would suggest.

#### Uneven skills

##### Argument

There are many axes of ability, and to replace a human at a particular task, a system needs to have a minimal ability on all of them. At which point, it should be substantially superhuman on some axes. For instance, by the time that self-driving cars are safe enough in all circumstances to be allowed to drive, they might be very much better than humans at noticing bicycles. So there might be a discontinuity in bicycle safety at the point that self-driving cars become common.

If this were right, it would also suggest a similar effect near overall human-level AGI: that the first time we can automate a variety of tasks as well as a human, we will be able to automate them much better than a human on some axes. So those axes may see discontinuity.

[![](http://aiimpacts.org/wp-content/uploads/2014/12/human-level-AI-300x230.png)](http://aiimpacts.org/wp-content/uploads/2014/12/human-level-AI.png)**Figure 2**: Illustration of the phenomenon in which the first entirely ‘human-level’ system is substantially superhuman on most axes. (Image from [Superintelligence Reading Group](http://lesswrong.com/lw/kw4/superintelligence_reading_group/))
##### Counterarguments

This phenomenon appears to be weakly related to AGI in particular, and should show up whenever one type of process replaces another one. Replacement of one type of system with another type does not seem very rare. So if it is true that large discontinuities are rare, then this type of dynamic does not produce large discontinuities often.

This argument benefits from the assumption that you have to ‘replace’ a human in one go, which seems usually false. New technology can usually be used to complement humans, admittedly with inefficiency relative to a more integrated system. For instance, one reason there will not be a discontinuity in navigation ability of cars on the road is that humans already use computers for their navigation. However this counterargument doesn’t apply to all axes. For instance, if computers operate quite fast but cannot drive safely, you can’t lend their speed to humans and somehow still use the human steering faculties.

##### Status

This argument seems successfully countered, unless:

1. We have missed some trend of discontinuities of this type elsewhere. If there were such a trend, it seems possible that we would have systematically failed to notice them because they look somewhat like ‘initial entry goes from zero to one’, or are in hard to measure metrics.
2. There are further reasons to expect this phenomenon to apply in the AGI case specifically, especially without being substantially mitigated by human-complementary technologies

#### Payoff thresholds

##### Argument

Even if we expect progress on technical axes to change continuously, there seems no reason this shouldn’t translate to discontinuous change in a related axis such as ‘economic value’ or ‘ability to take over the world’. For instance, continuous improvement in boat technology might still lead to a threshold in whether or not you can reach the next island.

##### Counterarguments

There actually seems to be a stronger theoretical reason to expect continuous progress in metrics that we care about directly, such as economic value, than on more technical metrics. This is that if making particular progress is expected to bring outsized gains, more effort should be directed toward the project until that effort has offset a large part of the gains. For instance, suppose that for some reason guns were a hundred times more useful if they could shoot bullets at just over 500m/s than just under, and otherwise increasing speed was slightly useful. Then if you can make a 500m/s gun, you can earn a hundred times more for it than a lesser gun. So if guns can currently shoot 100m/s, gun manufacturers will be willing to put effort into developing the 500m/s one up to the point that it is costing nearly a hundred times more to produce it. Then if they manage to produce it, they do not see a huge jump in value of making guns, and the buyer doesn’t see a huge jump in value of gun – their gun is much better but also much more expensive. However at the same time, there may have been a large jump in ‘speed of gun’.

This is a theoretical argument, and we do not know how well supported it is. (It appears to be at least supported by [the case of nuclear weapons](http://aiimpacts.org/discontinuity-from-nuclear-weapons/), which were discontinuous in the technical metric ‘explosive power per unit mass’, but not obviously so in terms of cost-effectiveness).

In practice, we don’t know of many large discontinuities in metrics of either technical performance or things that we care about more directly.

Even if discontinuities in value metrics were common, this argument would need to be paired with another for why AGI in particular should be expected to bring about a large discontinuity.

##### Status

This argument currently seems weak. In the surprising event that that discontinuities are actually common in metrics more closely related to value (rather than generally rare, and especially so that case), the argument would only be mildly stronger, since it doesn’t explain why AGI is especially susceptible to this.

#### Human-competition threshold

##### Argument

There need not be any threshold in nature that makes a particular level of intelligence much better. In a world dominated by humans, going from being not quite competitive with humans to slightly better could represent a step change in value generated.

This is a variant of the above argument from payoff thresholds that explains why AGI is especially likely to see discontinuity, even if it is otherwise rare: it is not often that something as ubiquitous and important as humans is overtaken by another technology.

##### Counterarguments

Humans have a variety of levels of skill, so even ‘competing with humans’ is not a threshold. You might think that humans are in some real sense very close to one another, but [this seems unlikely](http://aiimpacts.org/is-the-range-of-human-intelligence-small/) to us.

Even competing with a particular human is unlikely to have threshold behavior—without for instance a requirement that someone hire one or the other—because there are a variety of tasks, and the human-vs.-machine differential will vary by task.

##### Status

This seems weak. We would reconsider if further evidence suggested that the human range is very narrow, in some ubiquitously important dimension of cognitive ability.

 Footnotes
----------


Electrical efficiency of computing

Computer performance per watt has probably doubled every 1.5 years between 1945 and 2000. Since then the trend slowed. By 2015, performance per watt appeared to be doubling every 2.5 years.

Details
-------

In 2011 Jon Koomey reported that computation per kWh had doubled every roughly 1.5 years since around 1950, as shown in figure 1 (taken from him).[1](https://aiimpacts.org/electrical-efficiency-of-computing/#easy-footnote-bottom-1-1096 "“What most folks don’t know, however, is that the electrical efficiencyof computing (the number of computations that can be completed per kilowatt-hour of electricity) has doubled about every one and a half years since the dawn of the computer age ” – Jon Koomey, 19 Dec 2011, http://www.koomey.com/post/14466436072") Wikipedia calls this trend ‘[Koomey’s Law](https://en.wikipedia.org/wiki/Koomey%27s_law)‘. In 2015 Koomey and Naffziger reported in IEEE Spectrum that Koomey’s law began to slow down in around 2000 and by 2015, electrical efficiency was taking 2.5 years to double.[2](https://aiimpacts.org/electrical-efficiency-of-computing/#easy-footnote-bottom-2-1096 "“This trend started well before the first microprocessor, way back in the mid-1940s. But it began to come to an end around 2000. Growth in both peak-output efficiency and performance started to slow, weighed down by the physical limitations of shrinking transistors. ” – Koomey and Naffziger, 31 March 2015, https://spectrum.ieee.org/computing/hardware/moores-law-might-be-slowing-down-but-not-energy-efficiency")
We have not investigated beyond this, except to note that there is not obvious controversy on the topic. We do not know the details of the methods involved in this research, for instance how ‘computations’ are measured.

[![](http://aiimpacts.org/wp-content/uploads/2018/02/Koomeys_law_graph_made_by_Koomey.jpg)](http://aiimpacts.org/wp-content/uploads/2018/02/Koomeys_law_graph_made_by_Koomey.jpg)**Figure 1.** Computations per kWh over recent history. Taken from Dr Jon Koomey, <http://www.koomey.com/post/14466436072>, [CC BY-SA 3.0](https://creativecommons.org/licenses/by-sa/3.0 "Creative Commons Attribution-Share Alike 3.0")


Nordhaus hardware price performance dataset

This page contains the data from Appendix 2 of William Nordhaus’ [*The progress of computing*](https://web.archive.org/web/20160222082744/http://www.econ.yale.edu/~nordhaus/homepage/prog_083001a.pdf) in usable formats.

Notes
-----

This data was collected from Appendix 2 of [*The progress of computing*](https://web.archive.org/web/20160222082744/http://www.econ.yale.edu/~nordhaus/homepage/prog_083001a.pdf), using [Tabula](http://tabula.technology/) (a program for turning tables in pdfs into other table formats). We have not checked its accuracy beyond a graph of the resulting data looking visually similar to a graph of the original.

**Important:** we [previously noted](https://aiimpacts.org/trends-in-the-cost-of-computing/#Nordhaus) that this data appears to be orders of magnitude different from other sources, and haven’t had time to look into this discrepancy.

Data
----

[Here](https://docs.google.com/spreadsheets/d/1b6v_AXGjIYEc22MNXWmv0AcbUH6oJnqMHThrFcqUXoo/edit?usp=sharing) is a Google sheet of the data. See ‘Nordhaus via Tabula’ page.


2018 price of performance by Tensor Processing Units

Tensor Processing Units (TPUs) perform around 1 GFLOPS/$, when purchased as cloud computing.

Details
-------

In February 2018, Google Cloud Platform blog says their TPUs can perform up to 180 TFLOPS, and currently cost $6.50/hour.[1](https://aiimpacts.org/2018-price-of-performance-by-tensor-processing-units/#easy-footnote-bottom-1-1092 "“Built with four custom ASICs, each Cloud TPU packs up to 180 teraflops of floating-point performance…”

“Cloud TPUs are available in limited quantities today and usage is billed by the second at the rate of $6.50 USD / Cloud TPU / hour.”

Google Cloud Platform Blog, https://cloudplatform.googleblog.com/2018/02/Cloud-TPU-machine-learning-accelerators-now-available-in-beta.html [Accessed Feb 13 2018]") This gives us $171,000 to rent one TPU continually for a roughly three year lifecycle[2](https://aiimpacts.org/2018-price-of-performance-by-tensor-processing-units/#easy-footnote-bottom-2-1092 "We do not know the lifecycle of TPUs, but usually assume a lifecycle of three years for converting per hour and per computer prices for computing hardware.") Which is 1.05 GFLOPS/$.

This service apparently began on February 12 2018.[3](https://aiimpacts.org/2018-price-of-performance-by-tensor-processing-units/#easy-footnote-bottom-3-1092 "“Starting today, Cloud TPUs are available in beta on Google Cloud Platform (GCP) to help machine learning (ML) experts train and run their ML models more quickly.”

Google Cloud Platform Blog, https://cloudplatform.googleblog.com/2018/02/Cloud-TPU-machine-learning-accelerators-now-available-in-beta.html [Dated Feb 12 2018, Accessed Feb 13 2018]") So this does not appear to be competitive with the cheapest GPUs, in terms of FLOPS/$, or even the cheapest cloud computing.


Examples of AI systems producing unconventional solutions

This page lists examples of AI systems producing solutions of an unexpected nature, whether due to goal misspecification or successful optimization.  This list is highly incomplete.

List
----

1. [CoastRunners’ burning boat](https://blog.openai.com/faulty-reward-functions/)
2. [Incomprehensible evolved logic gates](https://www.damninteresting.com/on-the-origin-of-circuits/)
3. [AlphaGo’s inhuman moves](https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol)
4. [Waze direction into fires](https://www.usatoday.com/story/tech/news/2017/12/07/california-fires-navigation-apps-like-waze-sent-commuters-into-flames-drivers/930904001/)


Historic trends in altitude

*Published 7 Feb 2020*

Altitude of objects attained by man-made means has seen six discontinuities of more than ten years of progress at previous rates since 1783, shown below.

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Height (m)** | **Discontinuity (years)** | **Entity** |
| 1784 | 4000 | 1032 | Balloon |
| 1803 | 7280 | 1693 | Balloon |
| 1918 | 42,300 | 227 | [Paris gun](https://en.wikipedia.org/wiki/Paris_Gun) |
| 1942 | 85,000 | 120 | [V-2 Rocket](https://en.wikipedia.org/wiki/List_of_V-2_test_launches) |
| 1944 | 174,600 | 11 | [V-2 Rocket](https://en.wikipedia.org/wiki/List_of_V-2_test_launches) |
| 1957 | 864,000,000 | 35 | Pellets (after one day) |

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Trends

#### Altitude of objects attained by manmade means

We looked for records in height from the ground reached by any object via man-made technology.

‘Man-made technology’ is ambiguous, but we exclude for instance objects tied to birds and debris carried up by hurricanes. We include debris launched unintentionally via gunpowder explosion, and rocks launched via human arms.

We measure ‘altitude’ from the ground at the launch site. This excludes mountain climbing, but also early flight attempts that involve jumping from towers and traveling downward slowly.[1](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-1-1087 "For instance Hezârfen Ahmed Çelebi was reported to travel over 3km while losing 77m of altitude between a tower and a square in 1632.

See ‘site details’, “Hezârfen Ahmed Çelebi.” In Wikipedia, September 25, 2019. https://en.wikipedia.org/w/index.php?title=Hez%C3%A2rfen_Ahmed_%C3%87elebi&oldid=917757610.") It also excludes early parachutes, which were mentioned in fiction thousands of years ago.[2](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-2-1087 "“The earliest fictional account of a parachute type of device was made some 4,000 years ago when the Chinese noticed that air resistance would slow a person’s fall from a height.”

“Parachute.” In Wikipedia, November 21, 2019. https://en.wikipedia.org/w/index.php?title=Parachute&oldid=927299715.")

Measured finely enough, there are never discontinuities in altitude, since objects travel continuously.[3](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-3-1087 "As an object passes a previous record, first it is equal to it, then it moves continually past and onwards.") This prohibits finding discontinuities in continuously measured altitude, but doesn’t interfere with the dataset being relevant evidence to us. We are interested in discontinuities because they tell us about how much surprising progress can happen in a short time, and how much progress can come from a single innovation. So to make use of this data, we need to find alternate ways of measuring it that fulfill these purposes.

For the purpose of knowing about progress in short periods, we can choose a short period of interest, and measure jumps in progress made at that scale. For the purpose of knowing about progress made by single innovations, we can assign the maximum altitude reached to the time that the relevant innovation was made, for instance.[4](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-4-1087 "In many trends, if the full benefits of an innovation are not manifest until later, it is hard to connect it to that innovation, since further innovations are made, and contribute to ongoing progress. However with altitude, once an object is flying away from the Earth, further innovation in rocket design on Earth will not affect it, so we can measure this.")

We could measure both of these trends, but currently only measure a version of the former. For short periods of travel, we assign the maximum altitude reached to the date given (our understanding is that most of the entries took place over less than one day). For travel that appears to have taken more than a day, we record any altitudes we have particular information about, and otherwise estimate records on roughly an annual basis, including a record for the peak altitude (and possibly more than a year apart to allow for the final record to have the maximum altitude). This is ad hoc, but for the current purpose, converting what we have to a more consistent standard does not seem worth it. Instead, we consider these the effects of these choices when measuring discontinuities. They do not appear to matter, except to make modest differences to the size of the pellet discontinuity, discussed below (section, ‘Discontinuity measurement’).

##### Data

We collected data from various sources, and added them to [this spreadsheet](https://docs.google.com/spreadsheets/d/1YDhaYQNNEGyBqpQGTd1D8vG1W8WCc6DXZtb8jYn2Gho/edit?usp=sharing), tab ‘Manned and unmanned’. This data is shown in Figures 1-3 below. We have not thoroughly verified this data.

Record altitudes might plausibly be reached by a diversity of objects for a diversity of purposes, so collecting such data is especially dependent on imagination for the landscape of these.[5](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-5-1087 "For instance, the earliest objects we found that probably traveled as far away as the sun were small pellets fired into the sky for this purpose, which we did not immediately think to search for, and might easily have missed if we had only considered space probes") For this reason, this data is especially likely to be incomplete.

We also intentionally left the data less complete than usual in places where completeness seemed costly and unlikely to affect conclusions about discontinuities. The following section discusses our collection of data for different periods in history and details of our reasoning about it.

###### Detailed overview of data

Here we describe the history of progress in altitude reached and the nature of the data we collected during different times. See [the spreadsheet](https://docs.google.com/spreadsheets/d/1YDhaYQNNEGyBqpQGTd1D8vG1W8WCc6DXZtb8jYn2Gho/edit#gid=0) for all uncited sources.

Chimps throw rocks, so we infer that humans have probably also done this from the beginning.[6](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-6-1087 "“Recent research demonstrates that the bonds of kinship will not keep a chimp from piling up stones and hurling them at zoo visitors if they get too close. A new study of wild chimps at four sites in West Africa now shows that they also like to throw stones at trees.”

   BalterFeb. 29, Michael, 2016, and 5:00 Am. “Why Do Some Chimps Throw Rocks at Trees?” Science | AAAS, February 26, 2016. https://www.sciencemag.org/news/2016/02/why-do-some-chimps-throw-rocks-trees.") A good rock throw can apparently reach around 25m. Between then and the late 1700s, humanity developed archery, sky lanterns, kites, gunpowder, other projectile weapons, rockets, and primitive wings[7](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-7-1087 "See examples of early flying attempts here and here"), among probably other things. However records before the late 1700s are hard or impossible to find, so we do not begin the search for discontinuities until a slew of hot air balloon records beginning in 1783s. We collected some earlier records in order to have a rough trend to compare later advances to, but we are likely missing many entries, and the entries we have are quite uncertain. (It is more important to have relatively complete data for measuring discontinuities than it is for estimating a trend.)

The highest altitude probably attained before the late 1700s that we know of was reached by debris in a large gunpowder building explosion in 1280, which we estimate traveled around 2.5km into the air. Whether to treat this as a ‘man-made technology’ is ambiguous, given that it was not intentional, but we choose to ignore intention.[8](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-8-1087 "If we did not want to include unintentional object launches, this explosion does still suggest that intentionally launching debris that far using gunpowder was possible at the time, though it is unclear to us whether it was possible to do in a more controlled fashion, such that it might have been useful to anyone, and therefore for this possibility to imply much about what happened.")
Kites may also have traveled quite high, quite early. It appears that they have been around for at least two thousand years.[9](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-9-1087 "Stephanie Hall discusses some evidence that they were around by 1200 BC, here.") and were used [in ancient warfare](https://en.wikipedia.org/wiki/Kite#Military_applications) and even occasionally for [lifting people](https://en.wikipedia.org/wiki/Man-lifting_kite). We find it hard to rule out the possibility that early kites could travel one or two thousand meters into the air: modern kites frequently fly at 2km altitudes, silk has been available for thousands of years, and modern silk at least appears to be about as strong as nylon.[10](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-10-1087 "See spreadsheet for more detailed reasoning as well as citations.") Thus if we are wrong about the gunpowder factory explosion, it is still plausible that two thousand meter altitudes were achieved by kites.

Over a period of three and a half months from August 1783, manned hot air balloons were invented,[11](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-11-1087 "Though unmanned hot air propelled vessels are older: sky lanterns appear to have existed for thousands of years, and we saw one claim that hot-air aerostats existed in the 9th Century (which may just mean more sky lanterns, except that that wouldn’t make sense in context because sky lanterns are older.) ‘…the Middle Ages from the ninth century at least were familiar with hot-air aerostats used as military signals’

 White, Lynn. “Eilmer of Malmesbury, an Eleventh Century Aviator: A Case Study of Technological Innovation, Its Context and Tradition.” Technology and Culture 2, no. 2 (1961): 97–111. https://doi.org/10.2307/3101411. p98") and taken from an initial maximum altitude of 24m up to a maximum altitude of 2700m. While this was important progress in manned travel[12](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-12-1087 "Though perhaps in the convenience, reliability and safety of it before the height of it—Marco Polo describes observing a man-carrying kite that ‘might go up until it may no longer be seen’ by 1295.

Translation from Latin, found in only two manuscripts of Marco Polo’s work, but considered likely to be genuine: “The men of the ship will have wicker framework, that is a grate of switches, and to each corner and side of that framework will be tied a cord, so that there are eight cords and all of these are tied at the other end to a long rope. Next they will find some fool or drunkard and lash him to the frame, since no one in his right mind or with his wits about him would expose himself to that peril. This is done when the wind is high, then they raise the framework into the teeth of the wind and the wind lifts up the framework and carries it aloft, and the men hold it by the long rope. If the kite tips the men on the ground haul on the rope to straighten it, then pay the rope out again so by this means it might go up until it could no longer be seen, if only the rope were long enough.”

Taken from the first of these, but a very similar translation available at the second:

Beachcombing’s Bizarre History Blog. “Manned Kite Flight in Medieval China,” May 12, 2011. http://www.strangehistory.net/2011/05/13/manned-kite-flight-in-medieval-china/.

White, Lynn. “Eilmer of Malmesbury, an Eleventh Century Aviator: A Case Study of Technological Innovation, Its Context and Tradition.” Technology and Culture 2, no. 2 (1961): 97–111. https://doi.org/10.2307/3101411."), most of these hot air balloons were still lower than the gunpowder explosion and perhaps kites. Nonetheless, there are enough records from around this time, that we begin our search for discontinuities here.

The first time that humanity sent any object clearly higher than ancient kites or explosion debris was December 1783, when the first hydrogen balloon flight ascended to 2,700m. This was not much more than we (very roughly) estimate that those earlier objects traveled. However the hot air balloon trend continued its steep incline, and in 1784 a balloon reached 4000m, which is over a thousand years of discontinuity given our estimates (if we estimated the rate of progress as an order of magnitude higher or lower, the discontinuity would remain large, so the uncertainties involved are not critical.)

The next hot air balloon that we have records for ascended nearly twice as high—7280m—in 1803, representing another over a thousand years of discontinuity. We did not thoroughly search for records between these times. However if that progress actually accrued incrementally over the twenty years between these records, then still every year would have seen an extra 85 years of progress at the previous rate, so there must have been at least one year that saw at least that much progress, and it seems likely that in fact at least one year saw over a hundred years of progress. Thus there was very likely a large discontinuity at that time, regardless of the trend between 1784 and 1803.

We collected all entries from Wikipedia’s [Flight altitude record](https://en.wikipedia.org/wiki/Flight_altitude_record) page, which claims to cover ‘highest aeronautical flights conducted in the atmosphere, set since the age of ballooning’.[13](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-13-1087 "Such records were previously found to contain numerous discrepancies with different sources, so it seems likely that there are still some errors, however the known past errors were within ten percent, so this seems unlikely to be an important issue. See examples of discrepancies at “Talk:Flight Altitude Record.” In Wikipedia, February 28, 2018. https://en.wikipedia.org/w/index.php?title=Talk:Flight_altitude_record&oldid=828133827.") It is not entirely clear to us what ‘aeronautical flights’ covers, but seemingly at least hot air balloons and planes. The list includes some unmanned balloons, but it isn’t clear whether they are claiming to cover all of them. They also include two cannon projectiles, but not [38 cm SK L/45 “Max”](https://en.wikipedia.org/wiki/38_cm_SK_L/45_%22Max%22), which appears to be a record relative to anything they have, and cannon projectiles are probably not ‘flights’, so we think they are not claiming to have exhaustively covered those. Thus between the late 1700s, and the first flights beyond the atmosphere, the main things this data seems likely to be missing is military projectiles, and any other non-flight atmospheric-level objects.

We searched separately for military projectiles during this period. Wikipedia claims, without citation, that the 1918 Paris gun represented the greatest height reached by a human-made projectile until the first successful V-2 flight test in October 1942[14](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-14-1087 "“The gun was capable of firing a 106-kilogram (234 lb)[1]:120 shell to a range of 130 kilometers (81 mi) and a maximum altitude of 42.3 kilometers (26.3 mi)[1]:120—the greatest height reached by a human-made projectile until the first successful V-2 flight test in October 1942.”“Paris Gun.” In Wikipedia, August 19, 2019. https://en.wikipedia.org/w/index.php?title=Paris_Gun&oldid=911481367.    "), which matches what we could find. We searched for military records prior to the Paris gun, and found only one other, “Max” mentioned above, a 38cm German naval gun from 1914.

We expect there are no much higher military records we are missing during this time but that we could easily have missed some similar ones. As shown in Figure 1, the trend of military records we are aware of is fairly linear, and that line is substantially below the balloon record trend until around 1900. So it would be surprising if there were earlier military records that beat balloon records, and less surprising if we were missing something between 1900 and 1918. It seems unlikely however that we could have missed enough data that the Paris Gun did not represent at least a moderate discontinuity.[15](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-15-1087 "The Paris Gun was a 227 year discontinuity according to our calculations below, so had there been incremental progress over the four years since the previous record (assuming that was the same), each year would still have seen over fifty years of surprising progress (at least ignoring readjustment of expectations after seeing this twice in a row). Nonetheless, it is possible that there was enough incremental progress between 1893 and 1918 that we did not find that the Paris Gun is not a substantial discontinuity. It seems fairly unlikely to us that we would not have found any of it.")
We could not think of other types of objects that might have gone higher than aeronautical flights and military projectiles between the record 1803 balloon and V-2 rockets reaching ‘the edge of space’ from 1942. Thus the data in this period seems likely to be relatively complete, or primarily missing less important military projectiles.

The German V-2 rockets are considered the first man-made objects to travel to space (though the modern definition of space is higher)[16](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-16-1087 "“The V-2 rocket also became the first man-made object to travel into space by crossing the Kármán line with the vertical launch of MW 18014 on 20 June 1944.[5]“

   “V-2 Rocket.” In Wikipedia, November 15, 2019. https://en.wikipedia.org/w/index.php?title=V-2_rocket&oldid=926267063.    ") so they are presumably the highest thing at that time (1942). They are also considered the first projectile record since the Paris gun, supporting this. Wikipedia has an extensive [list of V-2 test launches](https://en.wikipedia.org/wiki/List_of_V-2_test_launches) and their outcomes, from which we infer than three of them represent altitude records.[17](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-17-1087 "A fairly early one was fired undesirably steeply, and so became the first rocket to reach space, as defined at the time. Thus it is less surprising that so few were records.")
The two gun records we know of were both German WWI guns, and the V2 rockets that followed were German WWII weapons, apparently developed in an attempt to replace the Paris Gun when it was banned under the Versailles Treaty.[18](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-18-1087 "“In the 1930s, the German Army became interested in rockets for long-range artillery as a replacement for the Paris Gun—which was specifically banned under the Versailles Treaty. This work would eventually led to the V-2 rocket that was used in World War II.” –    “Paris Gun.” In Wikipedia, August 19, 2019. https://en.wikipedia.org/w/index.php?title=Paris_Gun&oldid=911481367. ") So all altitude records between the balloons of the 1800s and the space rockets of the 50s appear to be German military efforts.

Between the last record V-2 rocket in 1946 and 1957, we found a series of rockets that traveled to increasing altitudes. We are not confident that there were no other record rocket altitudes in this time. However the rockets we know of appear to have been important ones, so it seems unlikely that other rockets at the time were radically more powerful, and there does not appear to have been surprising progress over that entire period considered together, so there could not have been much surprising progress in any particular year of it, unless the final record should be substantially higher than we think. We are quite unsure about the final record (the R-7 Semyorka), however it doesn’t seem as though it could have gone higher than 3000km, which would only add a further four years of surprising progress to be distributed in the period.

In October 1957, at least one centimeter-sized pellet was apparently launched into solar orbit, using shaped charges and a rocket. As far as we know, this was the first time an object escaped Earth’s gravity to orbit the sun.[19](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-19-1087 "For instance, Fritz Zwicky, the scientist responsible, writes in an article called, “The first shots into interplanetary space”, that “Small man-made projectiles were launched away from the earth for the first time, never to return.” We think he means that this was the first time anything was launched away from the Earth, never to return, rather than just the record for ‘shots’ or ‘small man-made projectiles’.

   Zwicky, Fritz. “The First Shots Into Interplanetary Space.” Engineering and Science 21 (January 1, 1958): 20–23. ") This episode does not appear to be mentioned often, but we haven’t found anyone disputing its being the first time a man-made object entered solar orbit, or offering an alternate object.

Because the pellets launched were just pellets, with no sophisticated monitoring equipment, it is harder to know what orbit they ended up in, and therefore exactly how long it took to reach their furthest distance from Earth, or what it was. Based on their speed and direction, we estimate they should still have been moving at around 10km/s as they escaped Earth’s gravity. Within a day we estimate that they should have traveled more than six hundred times further away than anything earlier that we know of. Then conservatively they should have reached the other side of the sun, at a distance from it comparable to that of Earth, in around 1.5 years. However this is all quite uncertain.

At around this time, reaching maximum altitudes goes from taking on the order of days to on the order of years. As discussed at the start of section ‘Altitude of objects attained by manmade means’ above, from here on we record new altitudes every year or so for objects traveling at increasing altitudes over more than a year.

In the years between 1959 and 1973, various objects entered heliocentric orbit.[20](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-20-1087 "Wikipedia lists them.

  “List of Artificial Objects in Heliocentric Orbit.” In Wikipedia, November 21, 2019. https://en.wikipedia.org/w/index.php?title=List_of_artificial_objects_in_heliocentric_orbit&oldid=927304642.

") It is possible that some of them reached greater altitudes than the pellets, via being in different orbits around the sun. Calculating records here is difficult, because reaching maximal distance from Earth takes years,[21](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-21-1087 "For instance, we roughly estimate that Luna 1 took five years to reach its maximum distance from Earth.") and how far an object is from Earth at any time depends on how their (eccentric) orbits relate to Earth’s, in 3D space. Often, the relevant information isn’t available.

Among artificial objects in heliocentric orbit listed by Wikipedia[22](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-22-1087 "Though known to not be complete: “This list does not include objects that are escaping from the Solar System, upper stages from robotic missions (only the S-IVB upper stages from Apollo missions with astronauts are listed), or objects in the Sun–Earth Lagrange points.”

“List of Artificial Objects in Heliocentric Orbit.” In Wikipedia, November 21, 2019. https://en.wikipedia.org/w/index.php?title=List_of_artificial_objects_in_heliocentric_orbit&oldid=927304642.   ") none are listed as having orbits where they travel more than 1.6 times further from the Sun than Earth does[23](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-23-1087 "Zond 3 was the furthest we found."), though many are missing such data. This is probably less far than the pellets, though further away than our conservative estimate for the pellets. For an object to reach this maximal distance from the Earth, it would need to be at this furthest part of its orbit, while being on the opposite side of the Sun from Earth, on the same plane as Earth.

Given all of this, it seems implausible that anything went ten times as far from the Sun as Earth by 1960, but even this would not have represented a discontinuity of even ten years. Given this and the difficulty of calculating records, we haven’t investigated this period of solar orbiters thoroughly.

In 1973 Pioneer 10 became the first of five space probes to begin a journey outside the solar system. In 1998 it was overtaken by Voyager 1. We know that no other probes were the furthest object during that time, however have not checked whether various other objects exiting the solar system (largely [stages of multi-stage rockets that launched the aforementioned probes](https://en.wikipedia.org/wiki/List_of_artificial_objects_leaving_the_Solar_System#Propulsion_stages)) might have gone further.

Figure 1 shows all of the altitude data we collected, including entries that turned out not to be records. Figures 2 and 3 show the best current altitude record over time.

![](https://aiimpacts.org/wp-content/uploads/2020/02/AllRecentTmp-1024x768.png) Figure 1: Post-1750 altitudes of various objects, including many non-records. Whether we collected data for non-records is inconsistent, so this is not a complete picture of progress within object types. It should however contain most aircraft and balloon records since 1783. See image in detail [here](http://aiimpacts.org/wp-content/uploads/2018/02/Altitudes-since-1750-3.png).
[To see detail:](http://aiimpacts.org/wp-content/uploads/2018/02/Altitudes-since-1750-3.png)[Download](http://aiimpacts.org/wp-content/uploads/2018/02/Altitudes-since-1750-3.png)

![](http://aiimpacts.org/wp-content/uploads/2018/02/AltitudeRecordZoom-1024x791.png)Figure 2: record altitudes known to us since 1750
![](https://aiimpacts.org/wp-content/uploads/2019/11/AltitudeRecord-1024x791.png)Figure 3: Record altitudes known to us since 6000 BC (early ones estimated imprecisely)
##### **Discontinuity measurement**

For measuring discontinuities, we treat the past trend at a given point as linear or exponential and as starting from earlier or later dates depending on what fits well at that time.[24](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-24-1087 "See our methodology page for more details. The trends are colored differently in column D of ‘calculations’ tab in the spreadsheet.") Relative to these previous rates, this altitude trend contains six discontinuities of greater than ten years, with four of them being greater than 100 years:[25](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-25-1087 "See our methodology page for more details, and our spreadsheet, tab ‘Calculations’ for the calculation.")

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Height (m)** | **Discontinuity (years)** | **Entity** |
| 1784 | 4000 | 1032 | Balloon |
| 1803 | 7280 | 1693 | Balloon |
| 1918 | 42,300 | 227 | [Paris gun](https://en.wikipedia.org/wiki/Paris_Gun) |
| 1942 | 85,000 | 120 | [V-2 Rocket](https://en.wikipedia.org/wiki/List_of_V-2_test_launches) |
| 1944 | 174,600 | 11 | [V-2 Rocket](https://en.wikipedia.org/wiki/List_of_V-2_test_launches) |
| 1957 | 864,000,000 | 35 | Pellets (after one day) |

The 1957 pellets would be a 66 year discontinuity if we counted all of their ultimate estimated altitude as one jump on the day after their launch, so exactly how one decides to treat altitudes that grow over years is unlikely to prevent these pellets representing a discontinuity of between ten and a hundred years.

In addition to the size of these discontinuities in years, we have tabulated a number of other potentially relevant metrics **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.[26](https://aiimpacts.org/discontinuity-in-altitude-records/#easy-footnote-bottom-26-1087 "See our methodology page for more details.")

*Primary authors: Katja Grace, Rick Korzekwa*

*Thanks to Stephen Jordan and others for suggesting a potential discontinuity in altitude records.*

Notes
-----


2015 FLOPS prices

In April 2015, the lowest GFLOPS prices we could find were approximately $3/GFLOPS. However recent records of hardware performance from 2015 and earlier imply substantially lower prices, suggesting that something confusing has happened with these sources of data. We have not resolved this.

Recent data
-----------

We have not finished exploring the apparent discrepancies between 2015 prices for performance and current records of 2015 prices for performance. However in the data described in our [2017 assessment](http://aiimpacts.org/recent-trend-in-the-cost-of-computing/) of recent price trends (key figure [here](http://aiimpacts.org/wp-content/uploads/2017/10/chart-43.png)), prices appear to have been below $1 since 2008.[1](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-1-1035 "Other examples of apparently very cheap hardware from 2015 or earlier can be seen at the bottom of the sheet called ‘Oct 2017 Update – misc – incomplete, misleading’ in this spreadsheet.") The measurements are not entirely comparable, but we would not expect the differences to produce such a large price difference.

2015 research
-------------

*The rest of this page is largely taken from our page written in 2015.*

In April 2015, the lowest recorded GFLOPS prices we knew of were approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this was $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seemed to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices were at least $11/GFLOPS, and computing as a service cost more like $160/GFLOPS.

### Background

We have written about [long term trends](http://aiimpacts.org/trends-in-the-cost-of-computing/ "Trends in the cost of computing") in the costs of computing hardware. We were interested in evaluating the current prices more thoroughly, both to validate the long term trend data, and because current hardware prices are particularly important to know about.

### Details

We separately investigated CPUs, GPUs, computing as a service, and supercomputers. In all categories, we collected some contemporary instances which we judged heuristically as especially likely to be cost-effective. We did not find any definitive source on the most cost-effective in any category, or in general, so our examples are probably not the very cheapest.  Nevertheless, these figures give a crude sense for the cost of computation in the contemporary market. Our full dataset of CPUs, GPUs and supercomputers is [here](https://docs.google.com/spreadsheets/d/1yqX2cENwkOxC26wV_sBOvV0NxHzzfmL6tU7StzrFXRc/edit?usp=sharing), and contains data on twenty two machines. Our data on computing as a service is all included in this page.

#### Included costs

For CPUs and GPUs, we list the price of the CPU and/or GPU (GPUs were always used with a CPU, so we include the cost for both), but not other computer components. We compared prices between [one complete rack server](https://drive.google.com/file/d/1xTA4LoooCuHhCWLhkJZ2ZbOWDxso3a0UILO2JxLn5CweadJVJUziux7CMiumtITXTOjXfttY5zNHzCee/view?usp=sharing) and the set of four [processors](http://www.ebay.com/itm/like/351337480917?lpid=82&chn=ps) inside it, and found the complete server was around 36% more expensive ($30,000 vs. $22,000). We expect this is representative at this scale, but diminishes with scale.

For computing services, we list the cheapest price for renting the instance for a long period, with no additional features. We do not include spot prices.

For supercomputers, we list costs cited, which don’t tend to come with elaboration. We expect that they only include upfront costs, and that most of the costs are for hardware.

We have not included the costs of energy or other ongoing expenses in any prices. Non-energy costs are hard to find, and we suspect a relatively small and consistent fraction of costs. Energy costs appear to be around 10% of hardware costs. For instance, the Intel Xeon E5-2699 uses 527.8 watts and costs $5,190.[2](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-2-1035 "The processor can be bought here for $5,190 as of April 1 2015. Its energy consumption is 527.8 watts under load, or 90.9 watts idle.") Over three years, with $0.05/kWh this is $694, or 13% of the hardware cost. Titan also uses 13% of its hardware costs in energy over three years.[3](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-3-1035 "Titan cost about $4000 dollars per hour amortized over 3 years, and consumes about 10M watts, at a cost of $500 per hour (assuming $0.05 per kWh), which is also 13% of its hardware cost.") We might add these costs later for a more precise estimate.

#### FLOPS measurements

To our knowledge we report only empirical performance figures from benchmark tests, rather than theoretical maximums. We sometimes use figures for [LINPACK](http://en.wikipedia.org/wiki/LINPACK_benchmarks) and sometimes for [DGEMM](http://matthewrocklin.com/blog/work/2012/10/29/Matrix-Computations/) benchmarks, depending on which are available. [Geekbench](http://browser.primatelabs.com/) in particular does not use the common LINPACK, but [LINPACK relies heavily on DGEMM](http://en.wikipedia.org/wiki/Basic_Linear_Algebra_Subprograms), suggesting DGEMM is fairly comparable. We guess they differ by around 10%.[4](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-4-1035 "This presentation (page ‘Results on a single node’) reports Linpack performance of 95% and 89% of DGEMM performance for their hardware in two tests.")

### Prices

#### Central processing units (CPUs)

We found prices and performance data for five contemporary CPUs, including three different instances of one of them. They ranged from $11-354/GFLOPS with most prices below $100/GFLOPS.[5](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-5-1035 "Muehlhauser and Rieber extended Koh and Magee’s data on MIPS available per dollar to 2014 (data available here). Their 2014 datapoint is for Intel Core i5-4430 and is 607 MIPS/$, and is roughly in line with their figures for recent years. According to Geekbench, it achieves 15.7 GFLOPS on the DGEMM benchmark. According to PCworld, it initially cost $175, and we have not found cheaper prices than this. This implies $11.14/GFLOPS. However Muehlhauser and Rieber seem to report a price of $48, which would make it $3.06/GFLOPS, highly competitive with GPUs and supercomputers. They also cite CPUworld, so we suspect this is an error. Either way, this CPU would not be substantially cheaper than the best GPUs, so does not alter our results.") The cheapest of these CPUs still looks several times more expensive than some GPUs and supercomputers, so we did not investigate these numbers in great depth, or search far for cheaper CPUs.

#### Graphics processing units (GPUs)

We found performance data for six recent combinations of CPUs and GPUs (with much overlap between CPUs and GPUs between combinations. They ranged from $3.22/GFLOPS to $4.17/GFLOPS.

Note that graphics cards are typically significantly restricted in the kinds of applications they can run efficiently; this performance is achieved for highly regular computations that can be carried out in parallel throughout a GPU (of the sort that are required for rendering scenes, but which have also proved useful in scientific computing).

#### Computing as service

Another way to purchase FLOPS is via virtual computers.

Amazon [Elastic Cloud Compute](http://en.wikipedia.org/wiki/Amazon_Elastic_Compute_Cloud) (EC2) is a major seller of virtual computing. Based on their [current pricing](https://aws.amazon.com/ec2/pricing/), renting a [c4.8xlarge](https://aws.amazon.com/ec2/instance-types/) instance costs about $1.17 / hour.[6](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-6-1035 "The effective hourly rate, if you purchase 3 years of computing, and pay upfront, is $1.1653 per hour.") This is their largest instance optimized for computing performance (rather than e.g. memory). A c4.8xlarge instance [delivers](http://browser.primatelabs.com/geekbench3/1694602) around 97.5 GFLOPS.[7](https://aiimpacts.org/2015-flops-prices/#easy-footnote-bottom-7-1035 " Geekbench Browser allows users to measure performance in FLOPS using a variety of tasks. 97.5 is the multi-core DGEMM score a user reported for c4.8xlarge. We use a multi-core score because the cost cited is for purchasing all of the cores. On other tasks, Geekbench reports scores from 46 to 199 GFLOPS. We do not know how reliable Geekbench reports are.") This implies that a GFLOPShour costs $0.012. If we suppose this is an alternative to buying computer hardware, then the relevant time horizon is about three years. Over three years, renting this hardware will cost $316/GFLOPS, i.e. around two orders of magnitude more than buying GFLOPS in the form of GPUs.

Other sources of virtual computing seem to be similarly priced. An [informal comparison](http://www.infoworld.com/d/cloud-computing/ultimate-cloud-speed-tests-amazon-vs-google-vs-windows-azure-237169?page=0,2) of computing providers suggests that on a set of “real-world java benchmarks” three providers are quite closely comparable, with all between just above Amazon’s price and just under half Amazon’s price for completing the benchmarks, across different instance sizes. This analysis also suggests Amazon is a relatively costly provider, and suggests a cheap price for virtual computing is closer to $0.006/GFLOPShour or $160/GFLOPS over three years.

Even with this optimistic estimate, virtual computing appears to cost something like fifty times more than GPUs. This high price is presumably partly because there are non-hardware costs which we have not accounted for in the prices of buying hardware, but are naturally included in the cost of renting it. However it is unlikely that these additional costs make up a factor of fifty.

#### Supercomputing

The Titan supercomputer [purportedly](http://en.wikipedia.org/wiki/Titan_(supercomputer)) cost about $97M to produce, or about $4,000 dollars per hour amortized over 3 years. It performs 17,590,000 GFLOPS which comes to $5.51/GFLOPS. This makes it around the same price as the cheapest GPUs. It is made of a combination of GPUs and CPUs, so this similarity is unsurprising.

The other six built supercomputers we looked at were more expensive, ranging up to $95/GFLOPS. Another cost-effective supercomputer, the L-CSC, was being built at the time it was most recently reported on, and while it should be completed now we could not find more data on it. Extrapolating from the figures before it was finished, when completed it should cost $2.39/GFLOPS, and thus be the cheapest source of FLOPS we are aware of.

### Summary

The lowest recorded GFLOPS prices we know of are approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this is $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seem to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices are at least $11/GFLOPS, and computing as a service costs more like $160/GFLOPS.


Effect of marginal hardware on artificial general intelligence

We do not know how AGI will scale with marginal hardware. Several sources of evidence may shed light on this question.

Details
-------

### Background

Suppose that at some point in the future, general artificial intelligence can be run on some quantity of hardware, *h*, producing some measurable performance *p*. We would like to know how running approximately the same algorithms using additional hardware (increasing *h*) affects *p*.

This is important, because if performance scales superlinearly, then at the time we can run a human-level intelligence on hardware that costs as much as a human, we can run an intelligence that performs more than twice as well as a human on twice as much hardware, and so already have superhuman efficiency at converting hardware into performance. And perhaps go on, for instance producing an entity which is 64 times as costly as a human yet almost incomparably better at thinking.

This might mean that the first ‘human-level’ effectiveness at converting dollars of hardware into performance would be earlier, when perhaps a mass of hardware costing one thousand times as much as a human can be used to produce something which performs a thousand times as well as a human. However it might be that the first time we have software to produce something roughly human-like, it does so at human-level with much smaller amounts of hardware. In which case, immediately scaling up the hardware might produce a substantially superhuman intelligence. This is one reason some people expect fast progress from sub-human to superhuman intelligence.

Whether gains are superexponential or sublinear depends on the metrics of performance. For instance, imagine hypothetically that doubling hardware generally produces a twenty point IQ increase (logarithmic gains in IQ), but twenty IQ points above the smartest human is enough to conquer areas of science that thousands of scientists have puzzled over ineffectually forever (much better than exponential gains in some metric of discovery or economic value). So the question must be how marginal hardware affects metrics that matter to us.

### Considerations

We have not investigated this question, but the following sources of evidence seem promising.

#### Evidence from existing algorithms

We do not yet have any kind of artificial general intelligence. However we can look at how performance scales with hardware in other kinds of software applications, especially narrow AI.

#### Evidence from human brain scaling

Among humans, brain size and intelligence are related. The exact relationship has been studied, but we have not reviewed the literature.

#### Evidence from between-animal brain scaling

Between animals, brain size relative to body size is related to intelligence. The exact relationship has probably been studied, but we have not reviewed the literature.

#### Evidence from new types of computation being possible with additional hardware

Some gains with hardware could come not from better performance on a particular task, but from being able to perform new tasks that were previously infeasible with so little hardware. We do not know how big an issue this is, but examining past experience with increasing hardware availability (e.g. by talking to researchers) seems promising.


Human-level hardware timeline

We estimate that ‘human-level hardware’— hardware able to perform as many computations per second as a human brain, at a similar cost to a human brain—has a 30% chance of having already occurred, a 45% third chance of occurring by 2040, and a 25% chance of occurring later. We are not confident about these estimates.

Support
-------

### Background

Say that computing hardware has reached ‘human-level’ performance when machines can perform as many computations as a human brain performs (under some natural interpretation of the brain as performing computations), at no greater cost than that of running a human brain.

We are interested in when hardware reaches this level of performance, because then if we have software that is also at least ‘human-level’,[1](https://aiimpacts.org/human-level-hardware-timeline/#easy-footnote-bottom-1-1070 "Here we say software is ‘human-level’ if it uses hardware as efficiently as the human brain to produce any behavior that a human can produce.") we will have ‘human-level’ AI overall: AI that can perform the tasks that a human brain performs, as efficiently as a human brain.

We may have human-level AI before having hardware and software that are both at least as good as the human brain—intuitively, if one of them is better than that, then the other may be worse. So the implications of human-level hardware alone are not straightforward. (We may also have disruptive change before we have human-level AI—the event of human-level AI is an upper bound for when some large changes in society can be expected to occur.)

It is unclear to us how important the contributions from hardware and software progress are, respectively, to overall AI progress. If hardware progress is much more important than software progress, then human-level hardware should approximately co-occur with human-level AI.

### Calculation

To roughly forecast when computing hardware will reach ‘human-level’, we can combine estimates of how much computing the human brain performs per dollar, current hardware performance per dollar, and the rate of improvement in hardware performance.

Let:

***T*** = time until human-level hardware performance per dollar.

***H*** = human-level hardware performance per dollar = 0.4-13\*1010 FLOPS/$.[2](https://aiimpacts.org/human-level-hardware-timeline/#easy-footnote-bottom-2-1070 "We estimate that the human brain performs around 1—34 x 1016 FLOPS, based on using a communication benchmark (TEPS) as a proxy for computing. This range does not represent our uncertainty about this method’s reliability.

Humans earn very roughly $100/hour. This means that purchasing computing hardware that costs as much as a human per hour, and lasted for around three years (as computing hardware often does), would cost $2.6M upfront.

So we should consider hardware to be competitive with human brains when it performs somewhere between 0.4-13*1010 FLOPS/$.")

***C*** = current hardware performance per dollar = 0.3-30 \*109FLOPS/$[3](https://aiimpacts.org/human-level-hardware-timeline/#easy-footnote-bottom-3-1070 "In 2017, cheap hardware appears to perform around 0.3-30 *109 FLOPS/$.")

***R*** = 1 + growth rate of hardware performance per dollar = 1.16-1.78[4](https://aiimpacts.org/human-level-hardware-timeline/#easy-footnote-bottom-4-1070 "The price of hardware appears to be declining at around an order of magnitude every 10-16 years. However in the longer term, the rate has been an order of magnitude every four years.")

Then we have:

T =  logR(H/C)

=log1.16(0.4 x 1010/(30 x 109)) to log1.16(13 x 1010/(.3 x 109))

= -14 to 41 years

These are rough calculations, and the breadth of the intervals don’t necessarily mean a lot—the intervals were non-specific to begin with, and then we combined several of them.

If we do something similar (shown [here](https://www.getguesstimate.com/models/10042)), using more realistic distributions for each variable and calculating using the entire distributions rather than end points, we get -14 to 22 years using the narrower estimates for human-level hardware that we used above, or -31 to 99 years for a very wide set of estimates for [human-level hardware](http://aiimpacts.org/brain-performance-in-flops/). The chance that human-level hardware has already occurred is around 20-40%, according to these calculations.

Based on these calculations, we estimate a 30% chance we are already past human-level hardware (at human cost), a 45% chance it occurs by 2040, and a 25% chance it occurs later.[5](https://aiimpacts.org/human-level-hardware-timeline/#easy-footnote-bottom-5-1070 "This is based on weighting the narrower estimates for what constitutes human-level hardware at 60% and the broader ones at 40%.")

Implications
------------

These figures suggest that the period when we most expect human-level hardware has already begun, and we are a substantial part of the way through it. In the case that hardware progress matters a lot more than software progress, this means that we should expect to see human-level AI in the next several decades, or possibly in the past. This is some evidence against hardware progress being so important, but still overall makes human-level AI likely to be sooner than one might have thought without the evidence considered here.


Chance date bias

There is modest evidence that people consistently forecast events later when asked the probability that the event occurs by a certain year, rather than the year in which a certain probability of the event will have accrued.

Details
-------

In the [2016 ESPAI](http://aiimpacts.org/2016-expert-survey-on-progress-in-ai/) and its preparation, AI experts and Mechanical Turk workers both consistently gave later probability distributions for events relating to AI when asked to give the probability that the event would occur by a given year, rather than the year by which there was a certain probability. See more on [the ESPAI page](http://aiimpacts.org/2016-expert-survey-on-progress-in-ai/).

We do not know which framing produces more reliable answers. We have not seen this bias elsewhere.

The following figure shows an example for some key figures: the distributions with stars are consistently a little flatter than those with circles.

[![](http://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone.png)](http://aiimpacts.org/wp-content/uploads/2017/04/AI-forecasts-by-framing-and-milestone.png)**Figure 1.** Median answers to questions about probabilities by dates (‘fixed year’) and dates for probabilities (‘fixed probability’), for different occupations, all current occupations, and all tasks (HLMI).


GoCAS talk on AI Impacts findings

*By Katja Grace, 27 November 2017*

Here is a video summary of some highlights from AI Impacts research over the past years, from the [GoCAS Existential Risk](https://www.chalmers.se/en/centres/GoCAS/Events/Existential-risk-to-humanity/Pages/default.aspx) workshop in Göteborg in September. Thanks to the folks there for recording it.


Price performance Moore’s Law seems slow

*By Katja Grace, 26 November 2017*

When people make predictions about AI, they often assume that computing hardware will carry on getting cheaper for the foreseeable future, at about the same rate that it usually does. Since this is such a common premise, and whether reality has yet proved it false is checkable, it seems good to check sometimes. So we [did](http://aiimpacts.org/recent-trend-in-the-cost-of-computing/).

Looking up the price and performance of some hardware turned out to be a real mess, with conflicting numbers everywhere and the resolution of each error or confusion mostly just leading to several more errors and confusions.

I suppose the way people usually make meaningful figures depicting computer performance changing over time is that they are doing it over long enough periods of time that even if each point is only accurate to within three orders of magnitude, it is fine because the whole trend is traversing ten or fifteen orders of magnitude. But since I wanted to know what was happening in the last few years, this wouldn’t do—half an order of magnitude of progress could be entirely lost in that much noise.

In the end, the two best looking sources of data we could find are the theoretical performance of GPUs (via Wikipedia), and [Passmark](https://www.passmark.com/)‘s collection of performance records for their own benchmark. Neither is perfect, but both make it look like prices for computing are falling substantially slower than they were. Over the last couple of decades it [had been](http://aiimpacts.org/trends-in-the-cost-of-computing/) taking about four years for computing to get ten times cheaper, and now (on these measures) it’s taking more like twelve years. Which could in principle be to do with these measures being different from usual, but I think probably not.

There are quite a few confusions still to resolve here. For instance, in spite of showing slower progress, these numbers look a lot cheaper than what would have been predicted by extrapolating [past trends](http://aiimpacts.org/trends-in-the-cost-of-computing/) (or sometimes more expensive). Which might be because we are comparing performance using different metrics, and converting between them badly. Different records of past trends seem to disagree with one another too, which is perhaps a hint. Or it could be that there was faster growth somewhere in between that we didn’t see. Or we might not have caught all of the miscellaneous errors in this cursed investigation.

But before we get too bogged down trying to work these things out, I just wanted to say that price performance Moore’s Law tentatively looks slower than usual.

*See full investigation at: [Recent Trend in the Cost of Computing](http://aiimpacts.org/recent-trend-in-the-cost-of-computing/)*