Historic trends in slow light technology

*Published Feb 7 2020*

Group index of light appears to have seen discontinuities of 22 years in 1995 from Coherent Population Trapping (CPT) and 37 years in 1999 from EIT (condensate). Pulse delay of light over a short distance may have had a large discontinuity in 1994 but our data is not good enough to judge. After 1994, pulse delay does not appear to have seen discontinuities of more than ten years.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

That which is uncited on this page is our understanding, given familiarity with the topic.[1](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-1-1315 "Our primary researcher for this page, Rick Korzekwa, has a PhD in physics, with experience in experimental optical physics. In addition, Rick discussed the main ideas in this page with Professor Steve Harris, a researcher responsible for substantial progress on slow light. Prof. Harris has not looked over our conclusions, and any mistakes are Rick’s.")

“[Slow Light](https://en.wikipedia.org/wiki/Slow_light)” is a phenomenon where the speed at which a pulse of light propagates through a medium is greatly reduced. This has potential applications for lasers, communication, and cameras.[2](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-2-1315 "“In the future, slowing light could have a number of practical consequences, including the potential to send data, sound, and pictures in less space and with less power. Also, the results obtained by Hau’s experiment might be used to create new types of laser projection systems and night vision cameras with power requirements a million times less than what is presently possible.”
Cromie, William J., and William J. Cromie. “Physicists Slow Speed of Light.” Harvard Gazette. February 23, 2018. Accessed July 03, 2019. https://news.harvard.edu/gazette/story/1999/02/physicists-slow-speed-of-light/. ")
The speed of propagation of the light through the medium referred to as the ‘[group velocity](https://en.wikipedia.org/wiki/Group_velocity)‘ of the light, and it is a function of the medium’s [refractive index](https://en.wikipedia.org/wiki/Refractive_index) and [dispersion](https://en.wikipedia.org/wiki/Dispersion_(optics)) (the rate at which the refractive index changes with the frequency of the light).

In most materials—for instance glass, air, or water—the dispersion is low enough that the group velocity is simply the speed of light divided by the index of refraction. In order to slow down light by more than roughly a factor of 3, physicists needed to create optical media with a greater dispersion in the frequency range of interest. The challenge in this was doing so without the medium absorbing most of the light, since most materials exhibit maximum dispersion under conditions of high absorption. This was resolved using exotic phases of matter and sophisticated methods for inducing transparency in them.

#### Summary of historic developments

Diamonds have a very high index of refraction, and the ability to cut and polish them to achieve good optical quality has existed for hundreds of years[3](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-3-1315 "“The first “improvements” on nature’s design involved a simple polishing of the octahedral  crystal faces to create even and unblemished facets, or to fashion the  desired octahedral shape out of an otherwise unappealing piece of rough.  This was called the point cut and dates from the mid 14th century; by 1375 there was a guild of diamond polishers at Nürnberg.”  “Diamond Cut – Wikipedia.” Accessed October 25, 2019. https://en.wikipedia.org/wiki/Diamond_cut."). However there were no light sources available for studying low group velocities until the 1960s, so recorded progress begins then. The first pulsed sources of light that could reasonably be used for the investigation of slow light came about in 1962 with the invention of Q-switching, which is a method for generating series of short light pulses from a laser. We do not know whether early Q-switched lasers could be used for this work, but doubt that any earlier light sources were suitable.

Following Q-switching, progress for slowing light proceeded roughly in four stages:

1. **High index materials:** For instance, diamonds. There may have been marginally better materials, but we did not investigate because our understanding is that they should at most represent a few tens of a percent of difference, and later gains represent factors of millions to trillions.
2. **High absorption media:** Materials with very low group velocity at a particular wavelength range, at the cost of very high absorption (losing >99% of the light over <100 microns).
3. **Induced transparency:** Materials with a narrow window of transparency in spectral regions of low group velocity. This led to rapid increases in total delay of a pulse, both through longer propagation distance and lower speeds.
4. **Stopped light:** Eventually, group velocity had been lowered to the point that it was possible to destroy the pulse, but store enough information in the medium about it to reconstruct it after some delay. There is room for debate about whether this is really the same pulse of light, however there are applications in which treating is as such is reasonable. We view this as progress in pulse delay, but not group index. After the invention of stopped light, slow light was no longer a major target for progress.

### Trends

There are several metrics that one might plausibly be interested in in this area. Group velocity is a natural choice because it is simple, but it trades off against absorption. So it is relatively easy to make a medium that has a very low group velocity, but it will absorb too much light to be useful. Because of this, research was more plausibly aimed at some combination of low  group velocity and low absorption.

One simple way to combine absorption and group velocity into a single metric is group velocity with an absorption criterion (say, lowest group velocity in a medium that transmits at least 1% of the light). Another is total time delay of the pulse by the medium, since longer delays can be achieved either by slowing down the pulse more, or slowing it down over a longer distance (requiring lower absorption). Pulse delay seems to have been a goal for researchers, suggesting it tracks something important, making it more interesting from our perspective.

We chose to investigate pulse delay and group index (the speed of light divided by the group velocity).[4](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-4-1315 "For more on our methodology, see our methodology page.")
#### Pulse delay and group index

##### Data

We collected data from a variety of online sources into [this spreadsheet](https://docs.google.com/spreadsheets/d/1rtAbkBsR5Jo-f7zvloi4fepxErZKpMOVoVe495F1TfM/edit?usp=sharing). The sheet shows progress in pulse delay and group index over time as well as our source for each data point, and calculates unexpected progress at each step. Figures 1-3 illustrates these trends.

![](https://aiimpacts.org/wp-content/uploads/2019/11/PulseDelay-1024x791.png)**Figure 1:** Progress in delay of a pulse of light over a short distance
![](http://aiimpacts.org/wp-content/uploads/2019/11/GroupN-1024x791.png) **Figure 2:** Progress in group index of a material (speed of light divided by speed of light in that material)
[![](http://aiimpacts.org/wp-content/uploads/2019/03/DelayngData4.png)](http://aiimpacts.org/wp-content/uploads/2019/03/DelayngData4.png)**Figure 3:** Progress in pulse delay and group index. “Human speed” shows the rough scale of motion familiar to humans.
##### Discontinuity measurement

For comparing points to ‘past rates of progress’ we treat past progress for both pulse delay and group index as exponential, changing to a new exponential regime near 1995 in both cases.[5](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-5-1315 "See our methodology page for further explanation of how we measure discontinuities. See our spreadsheet for calculations.")

Compared to these rates of past progress, the 1994 point—EIT (hot gas)—could be a very large discontinuity in pulse delay, if there was a small amount of progress prior to it. There probably was, however our estimates of the points leading up to it are so uncertain that it isn’t clear that there was any well-defined progress, and if there was we have not measured it. So we do not attempt to judge whether there is a discontinuity there. Aside from that, pulse delay saw no discontinuities of more than ten years.

Group index has discontinuities of 22 years in 1995 from CPT and 37 years in 1999 from EIT (condensate).

In addition to the size of these discontinuities in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[6](https://aiimpacts.org/historic-trends-in-slow-light-technology/#easy-footnote-bottom-6-1315 "See our methodology page for more details.")
##### Discussion of causes

These trends are short and not characterized by a clearly established rate prior to any potential change of rate, making changes in apparent rate relatively unsurprising. This means they are both less in need of explanation, and less informative about what to expect in cases where a technology does have a better-established progress trend.

Increasing group index of light does not appear to have been a major research goal prior to the discovery of induced transparency in the mid 1990s. Most of the work up to that point (and, to a lesser, extent after) was directed toward controlling the properties of optical media in general, with group index as one particularly salient parameter that could be controlled, but perhaps at the expense of others. Thus the moderate discontinuities in group index might relate to the hypothesized pattern of metrics that receive ongoing concerted effort tending to be more continuous than those receiving weak or sporadic attention.

*Primary author: Rick Korzekwa*

*Thanks to Stephen Jordan for suggesting slow light as a potential area of discontinuity.*

Notes
-----


Penicillin and historic syphilis trends

Penicillin did not precipitate a discontinuity of more than ten years in deaths from syphilis in the US. Nor were there other discontinuities in that trend between 1916 and 2015.

The number of syphilis cases in the US also saw steep decline but no substantial discontinuity between 1941 and 2008.

On brief investigation, the effectiveness of syphilis treatment and inclusive costs of syphilis treatment do not appear to have seen large discontinuities with penicillin, but we have not investigated either thoroughly enough to be confident.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

Penicillin was first used to treat a patient in 1941[1](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-1-1603 "“In 1940, Florey carried out vital experiments, showing that penicillin could protect mice against infection from deadly Streptococci. Then, on February 12, 1941, a 43-year old policeman, Albert Alexander, became the first recipient of the Oxford penicillin.”

American Chemical Society. “Alexander Fleming Discovery and Development of Penicillin – Landmark.” Accessed January 15, 2020. https://www.acs.org/content/acs/en/education/whatischemistry/landmarks/flemingpenicillin.html.") and became mass-produced in the US between 1942 and 1944.[2](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-2-1603 "“On March 14, 1942, the first patient was treated for streptococcal sepsis with US-made penicillin produced by Merck & Co.[38] Half of the total supply produced at the time was used on that one patient, Anne Miller.[39] By June 1942, just enough US penicillin was available to treat ten patients.[40] In July 1943, the War Production Board drew up a plan for the mass distribution of penicillin stocks to Allied troops fighting in Europe.[41] The results of fermentation research on corn steep liquor at the Northern Regional Research Laboratory at Peoria, Illinois, allowed the United States to produce 2.3 million doses in time for the invasion of Normandy in the spring of 1944…As a direct result of the war and the War Production Board, by June 1945, over 646 billion units per year were being produced.” “Penicillin,” in Wikipedia, May 23, 2019, https://en.wikipedia.org/w/index.php?title=Penicillin&oldid=898359231.") It quickly became the preferred treatment for syphilis, and appears to be generally credited with producing a steep decline in the prevalence of syphilis which was seen at around that time.[3](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-3-1603 "e.g. “Within years, widespread use of penicillin for treatment of all stages of syphilis (primary, secondary, tertiary, latent) resulted in dramatic decreases in the incidence of syphilis and associated mortality.”

John M. Douglas, “Penicillin Treatment of Syphilis,” JAMA 301, no. 7 (February 18, 2009): 769–71, https://doi.org/10.1001/jama.2009.143.") [4](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-4-1603 "“Today, with the dawn of the space-age, there are few who would disagree with the remark of Professor Smelov (1956) at the First International Symposium on Venereal Diseases and Treponematoses in Washington, D.C., U.S.A., that “hardly any one doubts the curative power of penicillin against
syphilis”, or the opinion of Kinaqigil (1956) of Turkey expressed at the same meeting that penicillin is preferable to all other drugs in this condition. Since Mahoney and his colleagues first used this new antibiotic in the treatment of syphilis (Mahoney, Arnold, and Harris, 1943a, b, 1949), 18 years have passed and little has occurred to shake the faith of
many thousands of doctors and of millions of patients in the potency of penicillin in this serious disease (see Doliken, 1954; Danbolt, 1954; Perdrup, Heilesen, and Sylvest, 1954; Shafer, Usilton, and Price, 1954) (Table I). Indeed, no other testimonial is required than the striking fall in the incidence of early syphilis which has occurred throughout the world.”

R. R. Willcox, “Treatment of Early Venereal Syphilis with Antibiotics*,” British Journal of Venereal Diseases 38, no. 3 (September 1962): 109–25.
")
![](https://lh6.googleusercontent.com/KXWDvDgDAIV-s-b7lYP7uNBQVztRhaAskYRHLCYYPsbHIoNjw7Aivr8PVUPwtNC7Rtc7FBm6caUee_a7l1eGTJ_eAdmZhlQsUe4eY8j6MqMajjm5QuHStAP6z1aPA50q62T2IFzt)Figure 1: US World War II Poster[5](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-5-1603 "From Wikimedia Commons: Science History Institute [Public domain]")
### Trends

We consider four metrics of success in treating syphilis: the number of syphilis cases, the number of syphilis deaths, effectiveness of syphilis treatment, and the inclusive cost of treatment.

In addition to the size of any discontinuities in years, we tabulated a number of other potentially relevant statistics for each metric [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).

#### US Syphilis cases

##### Data

Figure 1 shows historic reported syphilis cases after 1941, according to the CDC.[6](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-6-1603 "Figure 1 is Figure 33 from Division of STD Prevention, “Sexually Transmitted Disease Surveillance 2009,” November 2010, https://web.archive.org/web/20170120091355/https://www.cdc.gov/std/stats09/surv2009-Complete.pdf.") We converted the data in the figure into [this spreadsheet](https://docs.google.com/spreadsheets/d/1Kw068YXCXAajeuoOpPq81YR_jLVYAf9jM5---ubl48Y/edit?usp=sharing).[7](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-7-1603 "We used an automatic figure data extraction tool to extract the data from the figure. Here is a link to a .tar file that can be loaded into this tool to reproduce our extraction.")
[![Syphilis infection in the US](http://aiimpacts.wpengine.com/wp-content/uploads/2015/01/SyphilisUS2009.gif)](http://aiimpacts.wpengine.com/wp-content/uploads/2015/01/SyphilisUS2009.gif)
Figure 1: Syphilis—Reported Cases by Stage of Infection, United States, 1941–2009, according to the CDC[8](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-8-1603 "From Figure 33 in Division of STD Prevention, “Sexually Transmitted Disease Surveillance 2009,” November 2010, https://web.archive.org/web/20170120091355/https://www.cdc.gov/std/stats09/surv2009-Complete.pdf.")
##### Discontinuity Measurement

According to this data, total cases of syphilis declined by around 80% over fifteen years (see Figure 1). We do not see any substantial discontinuities, with 1944 seeing the largest change, equal to only 4 years of progress at the previous rate. Unfortunately, we were unable to find quantitative data prior to 1941, so we were only able to track progress for the three years leading up to the mass production of penicillin.

From our perspective, progress by 1943 may already have been affected by availability of penicillin that we do not know about, in which case we have no earlier trend to go by. However we note that the scale of annual reductions following penicillin is not larger than the increase seen in 1943, and not vastly larger than later annual variations, so the largest abrupt decrease from penicillin seems unlikely to have been large compared to the usual scale of variation.

#### US Deaths from syphilis

##### Data

We collected data from two graphs of historical US syphilis deaths and put it in [this spreadsheet](https://docs.google.com/spreadsheets/d/10ASOiR65QzaOnaX44CB-i8bGfEn5EiRkD5NLP2IjvGQ/edit?usp=sharing). The first is shown in Figure 2, and comes from Armstrong et al.’s 1999 report on infectious disease mortality in the United States.[9](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-9-1603 "Table 4D in Gregory L. Armstrong, Laura A. Conn, and Robert W. Pinner, “Trends in Infectious Disease Mortality in the United States During the 20th Century,” JAMA 281, no. 1 (January 6, 1999): 61–66, https://doi.org/10.1001/jama.281.1.61.") The authors collected it from historical mortality and population data from the CDC and public use mortality data tapes.[10](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-10-1603 "“Data were obtained from yearly tabulations of causes of death on file at the Division of Vital Statistics of the Centers for Disease Control and Prevention’s National Center for Health Statistics and from public use mortality data tapes from 1962 through 1996. … Population data used in the calculation of mortality rates were also obtained from the National Center for Health Statistics. The data for years prior to 1933 included only the population of the death-registration states or death-registration area, corresponding to the scope of the mortality data being used.” – Armstrong, Gregory L. 1999. “Trends In Infectious Disease Mortality In The United States During The 20Th Century”. JAMA 281 (1): 61. American Medical Association (AMA). doi:10.1001/jama.281.1.61.") We used an automatic figure data extraction tool to extract data from the figure.[11](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-11-1603 "The tool was at https://apps.automeris.io/wpd/. We also extracted data between 1917 and 1967 manually using the same tool. Here is a link to a .tar file that can be loaded into the tool here to reproduce our extraction.") Mortality rates after the mid-60s are indistinguishable from zero in this figure, so we do not include them. Instead we include records of total US deaths from Peterman & Kidd, 2019[12](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-12-1603 "Peterman, Thomas A., and Sarah E. Kidd. “Trends in Deaths Due to Syphilis, United States, 1968-2015.” Sexually Transmitted Diseases 46, no. 1 (2019): 37–40. https://doi.org/10.1097/OLQ.0000000000000899."), which we combine with US population data to get mortality rates between 1957 and 2015.

[![](https://aiimpacts.org/wp-content/uploads/2020/02/Deaths-1024x791.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2015/01/syphilis.png)
Figure 2: Syphilis mortality rate in the US during the 20th century.[13](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-13-1603 "See Figure 4D in Gregory L. Armstrong, Laura A. Conn, and Robert W. Pinner, “Trends in Infectious Disease Mortality in the United States During the 20th Century,” JAMA 281, no. 1 (January 6, 1999): 61–66, https://doi.org/10.1001/jama.281.1.61.")
![](https://aiimpacts.org/wp-content/uploads/2020/02/LogDeaths-1024x768.png)Figure 3: Syphilis mortality rate in the US during the 20th century, plotted on a log scale
##### Discontinuity Measurement

We calculate discontinuities in [our spreadsheet](https://docs.google.com/spreadsheets/d/10ASOiR65QzaOnaX44CB-i8bGfEn5EiRkD5NLP2IjvGQ/edit?usp=sharing), according to [this methodology](https://aiimpacts.org/methodology-for-discontinuity-investigation/#changes-in-the-rate-of-progress). There were no substantial discontinuities in progress for reducing syphilis deaths in the US during the time for which we have data. The largest positive deviation from a previous trend was a drop representing five years of progress in around 1940, two years before even enough ‘US penicillin’ was available to treat ten people.[14](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-14-1603 "“On March 14, 1942, the first patient was treated for streptococcal sepsis with US-made penicillin produced by Merck & Co.[38] Half of the total supply produced at the time was used on that one patient, Anne Miller.[39] By June 1942, just enough US penicillin was available to treat ten patients.[40]” “Penicillin,” in Wikipedia, May 23, 2019, https://en.wikipedia.org/w/index.php?title=Penicillin&oldid=898359231.")
In sum, while deaths from syphilis rapidly declined around the 1940s, this progress was not discontinuous at the scale of years. And while penicillin seems likely to have helped in this decline, it did not yet exist to contribute to the most discontinuously fast progress in that trend (and that progress was still not rapid enough to count as a substantial discontinuity for this project).

##### Discussion of causes

The decline of syphilis mortality does not appear to be entirely from penicillin, since it is underway by 1940, just prior to the mass-production of penicillin. This is strange, so it is plausible that we misunderstand some aspect of the situation.

The only other factor we know about is US Surgeon General Thomas Parran’s launch of a national syphilis control campaign in 1938.[15](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-15-1603 "“The serious consequences of syphilis for the population led to its designation as the “shadow on the land” and prompted US Surgeon General Thomas Parran to launch a national syphilis control campaign in 1938 based on public education, serologic testing, treatment, and a national network of sexually transmitted disease (STD) clinics.”

Douglas, John M. “Penicillin Treatment of Syphilis.” JAMA 301, no. 7 (February 18, 2009): 769–71. https://doi.org/10.1001/jama.2009.143.") Wikipedia also attributes some of the syphilis decline over the 19th and 20th centuries to decreasing virulence of the spirochete, but we don’t know of any reason for that to especially coincide with the 1940s decline.[16](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-16-1603 "“The symptoms of syphilis have become less severe over the 19th and 20th century in part due to widespread availability of effective treatment and partly due to decreasing virulence of the spirochete.[7]“

“Epidemiology of Syphilis.” In Wikipedia, February 9, 2019. https://en.wikipedia.org/w/index.php?title=Epidemiology_of_syphilis&oldid=882541706. ")
#### Effectiveness at treating syphilis

Even if penicillin’s effect on the US death rate from syphilis was gradual, we might expect this to be due to frictions like institutional inertia, rather than from gradual progress in the underlying technology. It might still be that penicillin was a radically better drug than its predecessors, when applied.

We briefly investigated whether penicillin might have represented discontinuous progress in effectiveness at curing syphilis, and conclude that it probably did not, because it does not appear to have been clearly better than its predecessor in terms of cure rates. In a 1962 review of treatment of ‘early’ syphilis[17](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-17-1603 "‘Early syphilis’ is defined as follows in Willcox: “seri-negative primary syphilis, zero-positive primary syphilis, secondary syphilis, and early latent syphilis in the first year of infection (although in the U.S.A. the first four years are taken).”"), Willcox writes that ‘a seronegativity-rate of 85 per cent. at 11 months had been achieved’ in 1944 after penicillin became the primary treatment for syphilis, but also says that the previously common treatment—arsenic and bismuth—was successful in more than 90% of cases in which it was carried out.[18](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-18-1603 "“Before the discovery of penicillin, reliance had had to be placed on arsenic and bismuth therapy given over periods of approximately one year. The reported results for those patients who completed their treatment (see, for example, Burckhardt, 1949; Degos, Vissian, and Basset, 1950; Thompson and Smith, 1950; Arutyunov and Gurvich, 1958) in large series of cases were good and cure rates exceeding 90 per cent. were reported…

…As soon as it became available, penicillin was soon in use for the treatment of syphilis throughout the world…

…As early as 1946 it became apparent in the U.S.A. that the results were deteriorating. Before May, 1944, a seronegativity-rate of 85 per cent. at 11 months had been achieved, but after that time the figure had fallen to only 60 per cent.”

Willcox, R. R. “Treatment of Early Venereal Syphilis with Antibiotics*.” British Journal of Venereal Diseases 38, no. 3 (September 1962): 109–25. ")
Willcox explains that the major downsides of the earlier treatment were very high defection rates (with perhaps as few as a quarter of patients completing the treatment), and ‘serious toxic effects’.[19](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-19-1603 "One of the two great disadvantages of metal-therapy was that, because of the relatively weak treponemicidal powers of the drugs employed, prolonged treatment involving many injections was required, and default from treatment, and therefore absence of cure in those who defaulted, was very common. Indeed, a minimum curative dose might be received by only one quarter of the patients (Chope and Malcolm, 1948). The other disadvantage was the risk of serious toxic effects, which not only curtailed treatment in affected patients but, by reputation, encouraged other patients to default.”

Willcox, R. R. “Treatment of Early Venereal Syphilis with Antibiotics*.” British Journal of Venereal Diseases 38, no. 3 (September 1962): 109–25. ") We have not checked that exactly the same notion of success is being used in these figures, have not assessed the reliability of this source, and do not know how important treatment for ‘early’ syphilis is relative to treatment for all syphilis, so it could still be that penicillin was a more effective treatment overall. However we did not investigate this further.

#### Inclusive costs of treatment

Penicillin apparently allowed most patients to receive a curative dose of medicine, whereas ‘arsenic and bismuth therapy’ achieved this for perhaps as few as a quarter of patients.[20](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-20-1603 "“The out-patient therapy of early syphilis became feasible only with the introduction by Romansky and Rittman (1945) of penicillin in oil-beeswax…by such means, nearly all patients could now achieve a curative dose (Hayman, 1947, Aitken, 1947) instead of only about one-quarter as with arsenic and bismuth (Chope and Malcolm, 1948).”

Willcox, R. R. “Treatment of Early Venereal Syphilis with Antibiotics*.” British Journal of Venereal Diseases 38, no. 3 (September 1962): 109–25.") If penicillin made an abrupt difference to syphilis treatment then, it seems likely to have been in terms of inclusive costs (which were partly reflected in willingness to be treated).

Qualitatively, the costs of treatment do seem to have been much lower. The time for treatment dropped from a year to around eight days.[21](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-21-1603 "“Before the discovery of penicillin, reliance had had to be placed on arsenic and bismuth therapy given over periods of approximately one year…

…The use of sixty or more injections of crystalline penicillin G in aqueous solution within a period of 7 1/2 days, if not more than the patients could reasonably tolerate, required their admission to hospital…Good results were reported with eight daily injections of 600,000 units…and success rates of 80 to 85 per cent. were achieved…”

R. R. Willcox, “Treatment of Early Venereal Syphilis with Antibiotics*,” British Journal of Venereal Diseases 38, no. 3 (September 1962): 109–25.

“In 1943 penicillin was introduced as a treatment for syphilis by John Mahoney, Richard Arnold and AD Harris. [22] Mahoney and his colleagues at the US Marine Hospital, Staten Island, treated four patients with primary syphilis chancres with intramuscular injections of penicillin four-hourly for eight days for a total of 1,200,000 units by which time the syphilis had been cured. “

John Frith, “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins,” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58.") Our impression is that the side effects qualitatively reduced from horrible and sometimes deadly to apparently bearable.

However even if penicillin was a large improvement over its predecessors in absolute terms (which seems likely), it would be hard to make a clear case that it was large relative to previous progress in syphilis treatments, because recent progress was also incredible.

The ‘arsenic and bismuth therapy’ mentioned above, that preceded penicillin, seems to have been a combination of the arsenic-based drug salvarsan (arsphenamine) and similar drugs developed subsequently, with bismuth. [22](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-22-1603 "“Arsenicals, mainly arsphenamine, neoarsphenamine, acetarsone and mapharside, in combination with bismuth or mercury then became the mainstay of treatment for syphilis until the advent of penicillin in 1943.”

Frith, John. “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins.” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58.") Salvarsan (arsphenamine) was considered such radical improvement over its own predecessors that it was known as the ‘magic bullet’, and won its discoverer Paul Erhlich a Nobel prize.[23](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-23-1603 "This led in 1910 to the manufacture of arsphenamine, which subsequently became known as Salvarsan, or the “magic bullet”, and later in 1912, neoarsphenamine, Neo-salvarsan, or drug “914”. In 1908 Ehrlich was awarded the Nobel Prize for his discovery. [7, 11, 12]”

John Frith, “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins,” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58.") A physician at the time describes[24](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-24-1603 "John Frith, “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins,” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58."):

>
> “Arsenobenzol, designated “606,” whatever the future may bring to justify the present enthusiasm, is now actually a more or less incredible advance in the treatment of syphilis and in many ways is superior to the old mercury – as valuable as this will continue to be – because of its eminently powerful and eminently rapid spirochaeticidal property.”
>
>
>

It is easy to see how salvarsan could be hugely costly to take, yet still represent large progress over earlier options, when we note that the common treatment prior to salvarsan was mercury,[25](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-25-1603 "“Mercury stayed in favour as treatment for syphilis until 1910 when Ehrlich discovered the anti-syphilitic effects of arsenic and developed Salvarsan, popularly called the “magic bullet”.”

Frith, John. “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins.” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58.") which had ‘terrible side effects’ including the death of many patients, characteristically took years, and was not obviously helpful.[26](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/#easy-footnote-bottom-26-1603 "“Many physicians doubted the efficacy of mercury, especially as it had terrible side effects and many patients died of mercury poisoning. Beck (1997) describes a typical mercury treatment :

“A patient undergoing the treatment was secluded in a hot, stuffy room, and rubbed vigorously with the mercury ointment several times a day. The massaging was done near a hot fire, which the sufferer was then left next to in order to sweat. This process went on for a week to a month or more, and would later be repeated if the disease persisted. Other toxic substances, such as vitriol and arsenic, were also employed, but their curative effects were equally in doubt.” [9]

Mercury had terrible side effects causing neuropathies, kidney failure, and severe mouth ulcers and loss of teeth, and many patients died of mercurial poisoning rather than from the disease itself. Treatment would typically go on for years and gave rise to the saying,

“A night with Venus, and a lifetime with mercury” [8]”

Frith, John. “Syphilis – Its Early History and Treatment until Penicillin and the Debate on Its Origins.” Journal of Military and Veterans’ Health 20 (November 1, 2012): 49–58.")
So at a glance penicillin doesn’t look to have been clearly discontinuous relative to the impressive recent trend, and measuring inclusive costs is hard to do finely enough to see less clear discontinuities. Thus evaluating these costs quantitatively will remain beyond the scope of this investigation at present. We tentatively guess that penicillin did not represent a large discontinuity in inclusive costs of syphilis treatment, though it did represent huge progress.

### Conclusions

Penicillin probably made quick but not abrupt progress in reducing syphilis and syphilis mortality. Penicillin doesn’t appear to have been much more likely to cure a patient than earlier treatments, conditional on the treatment being carried out, but it penicillin treatment appears to have been around four times more likely to be carried out, due to lower costs. Qualitatively penicillin represented an important reduction in costs, but it is hard to evaluate this precisely or compare it with the longer term progress. It appears that as recently as 1910 another drug for syphilis also represented qualitatively huge progress in treatment, so it is unlikely that penicillin was a large discontinuity relative to past progress.

Notes
-----


Historic trends in the maximum superconducting temperature

The maximum superconducting temperature of any material up to 1993 contained four greater than 10-year discontinuities: A 14-year discontinuity with NbN in 1941, a 26-year discontinuity with LaBaCuO4 in 1986, a 140-year discontinuity with YBa2Cu3O7 in 1987, and a 10-year discontinuity with BiCaSrCu2O9 in 1987.

YBa2Cu3O7 superconductors seem to correspond to a marked change in the rate of progress of maximum superconducting temperature, from a rate of progress of .41 Kelvin per year to a rate of 5.7 Kelvin per year.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

Superconductors were discovered in 1911.[1](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-1-1618 "“Superconductivity is a phenomenon of exactly zero electrical resistance and expulsion of magnetic flux fields occurring in certain materials, called superconductors, when cooled below a characteristic critical temperature. It was discovered by Dutch physicist Heike Kamerlingh Onnes on April 8, 1911, in Leiden.” – “Superconductivity”. 2018. En.Wikipedia.Org. Accessed June 29 2019. https://en.wikipedia.org/w/index.php?title=Superconductivity&oldid=903681858.") Until 1986 the maximum temperature for superconducting behavior had gradually risen from around 4K to less than 30K (see figure 2 below). Theory at the time apparently predicted that 30K was an upper limit.[2](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-2-1618 "“Until 1986 it was thought that superconducting behaviour was confined to certain materials at temperatures below ~30 K. A theory called “BCS theory” after its creators John Bardeen, Leon Cooper and Robert Schrieffer had been formulated to describe superconductivity.  This theory, for which its creators received the Nobel Prize in Physics in 1972, appeared to back this up but put a limit on the critical temperature of around 30 K.” –I“Doitpoms – TLP Library Superconductivity – Discovery And Properties”. 2019. Doitpoms.Ac.Uk. Accessed June 29 2019. https://www.doitpoms.ac.uk/tlplib/superconductivity/discovery.php.") In 1986 a new class of ceramics known as YBCO superconductors was discovered to allow superconducting behavior at higher temperatures: above 80K,[3](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-3-1618 "“Yttrium barium copper oxide (YBCO) is a family of crystalline chemical compounds, famous for displaying high-temperature superconductivity. It includes the first material ever discovered to become superconducting above the boiling point of liquid nitrogen (77 K) at about 92 K.” – “Yttrium Barium Copper Oxide”. 2019. En.Wikipedia.Org. Accessed June 29 2019. https://en.wikipedia.org/w/index.php?title=Yttrium_barium_copper_oxide&oldid=903757351.") and within seven years, above 130K.[4](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-4-1618 "“Here we provide support for this conjecture, with the discovery of superconductivity above 130 K in a material containing HgBa2Ca2Cu3O1+x (with three CuO2 layers per unit cell), HgBa2CaCu2O6+x (with two CuO2 layers) and an ordered superstructure comprising a defined sequence of the unit cells of these phases”

Schilling, A., M. Cantoni, J. D. Guo, and H. R. Ott. 1993. “Superconductivity Above 130 K In The Hg–Ba–Ca–Cu–O System”. Nature 363 (6424): 56-58. Springer Nature. doi:10.1038/363056a0.")
![](https://lh6.googleusercontent.com/c-L-VX9JFoquympXdjnzKGACEui-kcIgZ7Z_nxcdpP4K0YhMHk2fNDkU4fQv5xLXLYCLfxLmVif03Wd8lLMF3KH5cX6zbwuBsQKtbB5cPw-dHwAq0wQ4ajmp31_cPeaxBijRJI3y)**Figure 1:** Levitation of a magnet above a superconductor[5](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-5-1618 "From Wikimedia Commons: Mai-Linh Doan [CC BY-SA 3.0 (http://creativecommons.org/licenses/by-sa/3.0/)]")
### Trends

#### Maximum temperature for superconducting behavior

We looked at data for the maximum temperature at which any material is known to have superconducting behavior.

##### Data

We found the following data in a figure from the University of Cambridge’s online learning materials course, DoITPoMS,[6](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-6-1618 "“Doitpoms – TLP Library Superconductivity – Discovery And Properties”. 2019. Doitpoms.Ac.Uk. Accessed June 29 2019. https://www.doitpoms.ac.uk/tlplib/superconductivity/discovery.php.") and have verified most of it against other data sources (see [our spreadsheet](https://docs.google.com/spreadsheets/d/1JZh0wfCW-DrJjYLNgGW_TML-gmq1xZ44PfsfSCP5nlo/edit?usp=sharing), where we also collected ‘Extended data’ to verify that these were indeed the record temperatures).

We display the original figure from DoITPoMS in Figure 2 below, followed by our figure, Figure 3, which includes the a more recent superconducting material, H2S.

![superconduction-timeline.jpg (600Ã—395)](http://aiimpacts.org/wp-content/uploads/2014/12/superconduction-timeline.jpg)**Figure 2:** Maximum superconducting temperature by material over time through 2000, from the University of Cambridge’s online learning materials course, DoITPoMS,[7](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-7-1618 "“Doitpoms – TLP Library Superconductivity – Discovery And Properties”. 2019. Doitpoms.Ac.Uk. Accessed June 29 2019. https://www.doitpoms.ac.uk/tlplib/superconductivity/discovery.php.")

![](https://aiimpacts.org/wp-content/uploads/2020/02/Temperature-1024x768.png)**Figure 3:** Maximum superconducting temperate by material over time through 2015

##### Discontinuity measurement

We modeled this data as linear within two different regimes, one up to LaBaCu04 in 1986, and another starting with 1986 until our last data point.[8](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-8-1618 "See our spreadsheet to see the trends, and our methodology page for details on how we divide the data into trends and how to interpret the spreadsheet.") Using previous rates from those trends, we calculated four greater than 10-year discontinuities (rounded), shown in the table below:[9](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-9-1618 "See our methodology page for more details, and our spreadsheet for our calculation.")

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Temperature** | **Discontinuity** | **Material** |
| 1941 | 16 K | 14 years | [NbN](https://en.wikipedia.org/wiki/Niobium_nitride) |
| 1986 | 35 K | 26 years | [LaBaCuO4](https://en.wikipedia.org/wiki/Yttrium_barium_copper_oxide) |
| 1987 | 93 K | 140 years | [YBa2Cu3O7](https://en.wikipedia.org/wiki/Yttrium_barium_copper_oxide) |
| 1987 | 105 K | 10 years | [BiCaSrCu2O9](https://iopscience.iop.org/article/10.1143/JJAP.27.L209/meta) |

In addition to the size of this discontinuity in years, we have tabulated a number of other potentially relevant metrics **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.[10](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-10-1618 "See our methodology page for more details.")
##### Changes in the rate of progress

We note that there was a marked change in the rate of progress of maximum superconducting temperature with YBa2Cu3O7. The maximum superconducting temperature changed from a rate of progress of .41 Kelvin per year to a rate of 5.7 Kelvin per year.[11](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/#easy-footnote-bottom-11-1618 "See our methodology page for more details, and our spreadsheet for our calculation.")
Notes
-----


Historic trends in chess AI

The Elo rating of the best chess program measured by the Swedish Chess Computer Association did not contain any greater than 10-year discontinuities between 1984 and 2018. A four year discontinuity in 2008 was notable in the context of otherwise regular progress.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

The history of chess-playing computers is long and rich, partly because chess-playing ability has long been thought (by some) to be a sign of general intelligence.[1](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-1-1638 "For a good history of chess-playing computers, see this article. It says: “It was in this context that Turing, Von Neumann, and Shannon posed an ancient question in a now modern guise, in what came to be called “Artificial Intelligence” in the coming decade: can a machine be made to think like a person? And the answer to the question—the question of machine intelligence—was from the start tied to the question of whether a machine could be made to play chess. Turing began the investigation of chess playing computers with a system written out with paper and pencil, where he played the role of the machine. Later Shannon extended Turing’s work in a 1949 paper, explaining about his interest in chess that: “Although of no practical importance, the question is of theoretical interest, and it is hoped that…this problem will act as a wedge in attacking other problems—of greater significance.” As became clear in later writing by the two computer pioneers, “greater significance” was no less than the quest to “build a brain,” as Turing had put it. The quest for Artificial Intelligence, then, began with the question of whether a computer could play chess. Could it?”
Best_Schools. “A Brief History of Computer Chess.” TheBestSchools.org. September 18, 2018. Accessed July 18, 2019. https://thebestschools.org/magazine/brief-history-of-computer-chess/.

Another example: The tenth Turing Lecture, available here, mentions chess 20 times and uses it as a central example of how the field of artificial intelligence has progressed over the years.  Newell, Allen, and Herbert A. Simon. “Computer Science as Empirical Inquiry: Symbols and Search.” ACM Turing Award Lectures: 1975. doi:10.1145/1283920.1283930. ") The first two ‘chess-playing machines’ were in fact fakes, with small human chess-players crouching inside.[2](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-2-1638 "1769 – Wolfgang von Kempelen builds the Automaton Chess-Player, containing a human chess player hidden inside, in what becomes one of the greatest hoaxes of its period.
1868 – Charles Hooper presented the Ajeeb automaton — which also had a human chess player hidden inside.

“Computer Chess.” Wikipedia. July 10, 2019. Accessed July 18, 2019. https://en.wikipedia.org/wiki/Computer_chess. ") It was not until 1951 that a program was published (by Alan Turing) that could actually play the full game.[3](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-3-1638 " “1951 – Alan Turing is first to publish a program, developed on paper, that was capable of playing a full game of chess (dubbed Turochamp).[1][2]”
“Computer Chess.” Wikipedia. July 10, 2019. Accessed July 18, 2019. https://en.wikipedia.org/wiki/Computer_chess. ") There has been fairly regular progress since then.[4](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-4-1638 "See Wikipedia’s page on the history of computer chess.
“Computer Chess.” Wikipedia. July 10, 2019. Accessed July 18, 2019. https://en.wikipedia.org/wiki/Computer_chess.  ")

In 1997 IBM’s chess machine Deep Blue beat Gary Kasparov, world chess champion at the time, under standard tournament time controls.[5](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-5-1638 "“Deep Blue was then heavily upgraded, and played Kasparov again in May 1997.[1] Deep Blue won game six, therefore winning the six-game rematch 3½–2½ and becoming the first computer system to defeat a reigning world champion in a match under standard chess tournament time controls.[2]”

“Deep Blue (Chess Computer).” In Wikipedia, June 26, 2019. https://en.wikipedia.org/w/index.php?title=Deep_Blue_(chess_computer)&oldid=903491291.") This was seen as particularly significant in light of the continued popular association between chess AI and general AI.[6](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-6-1638 " “Computer scientists believed that playing chess was a good measurement for the effectiveness of artificial intelligence, and by beating a world champion chess player, IBM showed that they had made significant progress. After the loss, Kasparov said that he sometimes saw deep intelligence and creativity in the machine’s moves, suggesting that during the second game, human chess players had intervened on behalf of the machine…” “Computer Chess.” Wikipedia. July 10, 2019. Accessed July 18, 2019. https://en.wikipedia.org/wiki/Computer_chess.  ") The event marked the point at which chess AI became superhuman, and received substantial press coverage.[7](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-7-1638 "“The studio seated about five hundred people, and was sold-out for each of the six games. It seemed that the entire world was watching ”
Best_Schools. “A Brief History of Computer Chess.” TheBestSchools.org. September 18, 2018. Accessed July 18, 2019. https://thebestschools.org/magazine/brief-history-of-computer-chess/.  ")

The Swedish Chess Computer Association (SSDF) measures computer chess software performance by playing chess programs against one another on standard hardware.[8](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-8-1638 "“The Swedish Chess Computer Association (Swedish: Svenska schackdatorföreningen, SSDF) is an organization that tests computer chess software by playing chess programs against one another and producing a rating list. […] The SSDF list is one of the only statistically significant measures of chess engine strength, especially compared to tournaments, because it incorporates the results of thousands of games played on standard hardware at tournament time controls. The list reports not only absolute rating, but also error bars, winning percentages, and recorded moves of played games.”

“Swedish Chess Computer Association”. 2009. En.Wikipedia.Org. Accessed June 19 2019. https://en.wikipedia.org/w/index.php?title=Swedish_Chess_Computer_Association&oldid=891692663.")
![](https://lh3.googleusercontent.com/m7bZT3RHdGEPDsZz_d5b-1ln3Q_kcR2lgI_xOAAUmFzZANCZaf8ub_-YbzvdUFbxK8Xetrc6Duseoxvyvwrdq7bdW6ds95V3C8RjB2BRZSyZp6-UfXU-3mmpH10wRWNo-vlwa-a7)Figure 1: Deep Blue[9](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-9-1638 "From Wikimedia Commons: James the photographer [CC BY 2.0 (https://creativecommons.org/licenses/by/2.0)]")
### Trends

#### SSDF Elo Ratings

According to Wikipedia[10](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-10-1638 "“Swedish Chess Computer Association.” In Wikipedia, April 9, 2019. https://en.wikipedia.org/w/index.php?title=Swedish_Chess_Computer_Association&oldid=891692663."):

> The **Swedish Chess Computer Association** (Swedish: *Svenska schackdatorföreningen*, SSDF) is an organization that tests computer chess software by playing chess programs against one another and producing a rating list…The SSDF list is one of the only statistically significant measures of chess engine strength, especially compared to tournaments, because it incorporates the results of thousands of games played on standard hardware at tournament time controls. The list reports not only absolute rating, but also error bars, winning percentages, and recorded moves of played games.
>
>

##### Data

We took data from Wikipedia’s list of SSDF Ratings[11](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-11-1638 "“Swedish Chess Computer Association”. 2009. En.Wikipedia.Org. Accessed June 19 2019. https://en.wikipedia.org/w/index.php?title=Swedish_Chess_Computer_Association&oldid=891692663.") (which we have not verified) and added it to [this spreadsheet](https://docs.google.com/spreadsheets/d/1gJU4lfAiQXLPp15xYHu1umZewc2HIHFsJzHTefS147o/edit?usp=sharing). See Figure 2 below.

![](https://aiimpacts.org/wp-content/uploads/2020/02/ChessRating-1024x768.png) Figure 2: Elo ratings of the best program on SSDF at the end of each year.
##### Discontinuity measurement

Looking at the data, we assume a linear trend in Elo.[12](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-12-1638 "See our methodology page for more details.") There are no discontinuities of 10 or more years.

###### Minor discontinuity

There is a four year discontinuity in 2008. While this is below the scale of interest for our [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/), it strikes us as notable in the context of otherwise very regular progress.[13](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-13-1638 "See our methodology page for more details, and our spreadsheet for our calculation.") We’ve tabulated a number of other potentially relevant metrics for this discontinuity in the ‘Notable discontinuities less than 10 years’ tab **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.[14](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-14-1638 "See our methodology page for more details.")

This jump appears to have been partially caused by the introduction of new hardware in the contest, as well as software progress.[15](https://aiimpacts.org/historic-trends-in-chess-ai/#easy-footnote-bottom-15-1638 "‘The jump perfectly corresponds to moving from all programs running on an Arena 256 MB Athlon 1200 MHz to some programs running on a 2 GB Q6600 2.4 GHz computer, suggesting the change in hardware accounts for the observed improvement. However, it also corresponds perfectly to Deep Rybka 3 overtaking Rybka 2.3.1. This latter event corresponds to huge jumps in the CCRL and CEGT records at around that time, and they did not change hardware then. The average program in the SSDF list gained 120 points at that time (Karlsson 2008), which is roughly the difference between the size of the jump in the SSDF records and the jump in records from other rating systems. So it appears that the SSDF introduced Rybka and new hardware at the same time, and both produced large jumps.’ – Grace, Katja. Algorithmic Progress in Six Domains. Report. December 9, 2013. Accessed June 19, 2019. https://intelligence.org/files/AlgorithmicProgress.pdf, p19")
Notes
-----


Effect of Eli Whitney’s cotton gin on historic trends in cotton ginning

We estimate that Eli Whitney’s cotton gin represented a 10 to 25 year discontinuity in pounds of cotton ginned per person per day, in 1793. Two innovations in 1747 and 1788 look like discontinuities of over a thousand years each on this metric, but these could easily stem from our ignorance of such early developments. We tentatively doubt that Whitney’s gin represented a large discontinuity in the cost per value of cotton ginned, though it may have represented a moderate one.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

Cotton fibers grow around cotton seeds, which they need to be separated from before use. This can be done by hand, but since 500 C.E.,[1](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-1-1359 "“A fifth-century Buddhist painting… constitutes the earliest evidence of a single-roller gin.”  Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 4.") and plausibly prehistory, a variety of tools have aided in speeding up the process. [2](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-2-1359 " “Archaeologists’ oversight may explain the absence of evidence that would locate the single-roller gin in prehistory. That the rollers of extant gins are made of iron does not preclude the possibility that the machine predates the Iron Age. The roller could have been made out of stone…”
Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003, 4.")
These tools are called ‘cotton gins’. Eli Whitney’s 1793 cotton gin was a particularly famous innovation, commonly credited with having vastly increased cotton’s profitability, fueling an otherwise diminishing demand for slave labor, and so substantially contributing to the American Civil War.[3](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-3-1359 "Wikipedia says: “A modern mechanical cotton gin was created by American inventor Eli Whitney in 1793 and patented in 1794…It revolutionized the cotton industry in the United States, but also led to the growth of slavery in the American South as the demand for cotton workers rapidly increased. The invention has thus been identified as an inadvertent contributing factor to the outbreak of the American Civil War.[4] ”

“Cotton Gin.” In Wikipedia, June 4, 2019. https://en.wikipedia.org/w/index.php?title=Cotton_gin&oldid=900249024.   ") Variants on Whitney’s gin are known as ‘saw gins’.[4](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-4-1359 "“In 1794 Eli Whitney patented a new ginning principle and built a new kind of gin. Instead of rollers that pinched off the fiber, he used course wire teeth that rotated through a tightly spaced metal grate to pull it from the seed…Industry ambivalence spurred others to adopt the gin but change it. Gin makers substituted an axle loaded with fine-toothed circular saws for Whitney’s wire-studded wooden cylinder. In 1796 Hogden Holmes of Augusta, Georgia, patented the adaptation, naming it the saw gin. The suit that followed capped a contentious and socially and legally mediated process from which Eli Whitney emerged as the inventor of the cotton gin.”

Angela Lakwete, Inventing the Cotton Gin: Machine and Myth in Antebellum America (JHU Press, 2003), 47.") (See Figure 1.) Cotton became more valuable than all other US exports combined during the antebellum era.[5](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-5-1359 "First table of p567,

Federal Reserve Bulletin (U.S. Government Printing Office, 1923), https://books.google.com/books?id=oNnL2qUnv1AC&printsec=frontcover#v=onepage&q&f=false") Thus Whitney’s gin is a good contender for representing a discontinuity in innovation.

Our investigation draws heavily from Lakwete’s *Inventing the Cotton Gin.* Lakwete summarizes the situation surrounding Whitney’s invention as follows[6](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-6-1359 "Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003, 48"):

> The introduction of a new gin in 1794 was as unexpected as it was unprecedented. It was unexpected because the British textile industry had expanded from the sixteenth through the eighteenth centuries without a change in the ginning principle. Cotton producers had increased the acres they planted in cotton and planted new varieties to suit textile makers. The market attracted new producers who, like established planters, used roller gins to process their crops. Roller gins, whether hand-cranked in the Levant and India, or foot-, animal-, and inanimately powered in the Americas, provided adequate amounts of fiber with the qualities that textile makers wanted, namely length and cleanliness. All roller gins removed the fiber by pinching it off in bundles, preserving its length and orientation as grown. Random fragments of fractured seeds were picked out of the fiber before it was bagged and shipped.
>
> In 1788 Joseph Eve gave planters and merchants a machine that bridged the medieval and modern. It preserved the ancient roller principle but completed the appropriation of the ginner’s skill, as Arkwright’s frame had that of the spinner. Appropriation had proceeded in stages beginning with the single-roller gin that mechanized the thumb and finger pinching motion. The roller gin in turn appropriated the agility and strength needed to manipulate the single roller, while the foot gin freed both hands to supply seed cotton. The barrel gin used animal and water power, removing humans as a power source but retaining them as seed cotton suppliers. The self-feeding animal-, wind-, or water-powered Eve gin replaced each of the skilled tasks of the ginner with mechanical components.
>
> Nevertheless, Eli Whitney’s unprecedented gin filled a vacuum. While large merchants invested in barrel gins and large planter in the Eve gin, the majority continued to use the skill- and labor-intensive foot gin to gin fuzzy-seed short-staple cotton as well as the smooth-seed, Sea Island cotton. Barrel gins had not decreased the number of ginners and only marginally improved ginner productivity, and Eve’s complicated gin was notoriously finicky. Whitney ignored these modernizing gins and offered a replacement for the ubiquitous foot gin.
>
>

![](https://lh5.googleusercontent.com/866WFGmUOmL7hCaEUwT6u6Yexe9JJyoQBkdk_XJPqsqD98ieKl2IUZm1YKDhW6hD7BbcY9hjCwB9lY40DHQCISBFN5jOq4WWovhblAVk2afOqpvfqe7nXLR1R6r3tIZRLQT0WCKd)Figure 1: What appears to be a saw gin of some kind on display at the Eli Whitney Museum[7](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-7-1359 "From Wikimedia Commons:
Tom Murphy VII (Public domain)")
### Trends

#### Pounds of cotton ginned per person-day

We are most interested in metrics that people were working to improve—in this case, perhaps ‘cost of producing a dollar’s worth of cotton’. Inclusive metrics are hard to measure however. Instead we have collected data on ‘pounds of cotton ginned per person per day’, which is simpler, often reported on, and probably a reasonable proxy. However, it departs from tracking the usefulness of a gin by ignoring several major factors:

1. Upfront costs: these presumably varied a lot, because a gin can for instance resemble a rolling pin and a board, or involve horses or steam power.[8](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-8-1359 "See a variety of gin descriptions at:
“Cotton Gin,” in Wikipedia, June 4, 2019, https://en.wikipedia.org/w/index.php?title=Cotton_gin&oldid=900249024.") Thus the gins with higher upfront costs are less useful than their cotton-per-person-day statistic would make them seem. In the mid-1860’s many farms still used foot gins, seemingly because Eve gins—while more efficiently producing high quality output—were expensive.[9](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-9-1359 "“Instead of being replaced, the foot gin outlasted the barrel gin and remained in use to the mid-1860s. … The [Eve gin] was labor-saving but capital-intensive. It was more expensive than the foot gin and required water or windmill as well as a reinforced building that could withstand the vibrations of the feeder. Yet it increased outturn without changing the quality of the fiber…” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 40.") Even if everything had the same upfront costs, the existence of upfront costs means that a gin which processes 200lb of cotton with two people per day would be better than one that processes 100lb of cotton with one person, so cotton/person-day still fails to match what we are interested in.
2. Variation in labor requirements: Some gins required especially skilled labor.[10](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-10-1359 "“Foot ginners could be productive… but they were the most valuable men a planter owned or hired. With Eve’s gin, planters used that labor to produce cotton and “only the most ordinary” workers to gin it.” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 45.")
3. Substitutes for people: some gins used people to power them and others used animals or water-power, along with a smaller number of people.[11](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-11-1359 "“The [Eve gin] was labor-saving but capital-intensive. It was more expensive than the foot gin and required water or windmill …” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 40.

“McCarthy’s gin was adopted for cleaning the Sea Island variety of extra-long staple cotton grown in Florida, Georgia and South Carolina. It cleaned cotton several times faster than the older gins, and, when powered by one horse, produced 150 to 200 pounds of lint a day.”

“Cotton Gin,” in Wikipedia, June 4, 2019, https://en.wikipedia.org/w/index.php?title=Cotton_gin&oldid=900249024.

") This again makes for higher output per person, but at the cost of additional animals, that we are not accounting for.

- Risks of injury: Some gins, particularly foot and barrel gins, were dangerous to operate.[12](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-12-1359 "“The oxen and water-driven barrel gins that Drayton described could not be stopped quickly, so ginners risked serious injury.”…”Foot gins were not without their own risks and limitations.” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 39.")

- Types and quality of cotton ginned: Whitney’s gin produced degraded cotton fiber, relative to other gins available at the time.[13](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-13-1359 "“In 1794 Eli Whitney patented a new ginning principle and built a new kind gin…It turned out large quantities of fiber but destroyed the very qualities that textile makers had valued.”

Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003, 47

“But textile makers continued to complain about the quality of the fiber the toothed gin turned out.” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 63") However, Whitney’s gin could process short staple cotton, an easier to grow strain which was previously hard to process.[14](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-14-1359 "“In the colonial era, small amounts high quality long-staple cotton were produced in the Sea Islands off the coast of South Carolina. Inland, only short-staple cotton could be grown but it was full of seeds and very hard to process into fiber. The invention of the cotton gin in the late 1790s for the first time made short-staple cotton usable.”

“History of Agriculture in the United States.” Wikipedia. April 11, 2019. Accessed April 25, 2019. https://en.wikipedia.org/wiki/History_of_agriculture_in_the_United_States#Cotton.

“The Indian roller cotton gin, known as the churka or charkha, was introduced to the United States in the mid-18th century, when it was adopted in the southern United States. The device was adopted for cleaning long-staple cotton, but was not suitable for the short-staple cotton that was more common in certain states such as Georgia. Several modifications were made to the Indian roller gin by Mr. Krebs in 1772 and Joseph Eve in 1788, but their uses remained limited to the long-staple variety, up until Eli Whitney’s development of a short-staple cotton gin in 1793″

“Cotton Gin.” Wikipedia. April 03, 2019. https://en.wikipedia.org/wiki/Cotton_gin.") The cotton industry might adjust to different cotton over time, so that long-run differences in quality of outputs of different gins are smaller than initial differences. If so, we expect value produced by a new gin producing lower quality cotton to grow continuously over an extended period.

We do also investigate overall cost per value of cotton ginned later, but do not have such clear data for it (see section, ‘cost per value of cotton ginned’).

##### Data

We collected claims about cotton gin productivity in the time leading up to Whitney’s gin, and some after. Many but not all are from Angela Lakwete’s book, *Inventing the Cotton Gin: Machine and Myth in Antebellum America*.[15](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-15-1359 "Angela Lakwete, Inventing the Cotton Gin: Machine and Myth in Antebellum America (JHU Press, 2003).") Our sources are mostly secondhand or thirdhand claims about nonspecific observations in the 1700s. We have the impression that claims in this space are not very reliable.[16](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-16-1359 "For instance, some claim Barclay was a slave, another elaborates his story as a plantation owner. Data from here suggests a large drop in cotton bale production between 1860 and 1880, yet this says, “By 1870, sharecroppers, small farmers, and plantation owners in the American south had produced more cotton than they had in 1860, and by 1880, they exported more cotton than they had in 1860.” Most of all, Whitney’s own claim about the productivity of his original gin is arguably too outlandish to be believed—while estimates of gin productivity slightly later don’t reach 400lbs/person-day (see our data below), Whitney apparently claimed his gin could produce 1250lb/person-day:

“Two ginners on the new gin could turn out as much fiber in a day as one hundred foot ginners each averaging twenty-five pounds, in all twenty-five hundred pounds, Whitney estimated.” (Whitney quote from Angela Lakwete, Inventing the Cotton Gin: Machine and Myth in Antebellum America (JHU Press, 2003), p49)
") We classified claims as ‘credible’ or not, but this is fairly ambiguous, and we would be unsurprised if some of the ‘credible’ claims turned out to be inaccurate, or the ‘non-credible’ ones were correct.

Our dataset of claims is [here](https://docs.google.com/spreadsheets/d/1oXOy2cyaSv_9svTXiInlDbOWn1vv6BG19dEQiZk40C8/edit#gid=91839643), and illustrated in Figures 2 – 5. Note that dates are those when a claim was made, not necessarily dates of the invention of the type of cotton gin in question. This is because invention dates are hard to find, and also because it seems likely that much improvement happened incrementally between distinct ‘inventions’ of new types. Nonetheless, this means that a report dated to a time could be from a gin that was built earlier.

![](http://aiimpacts.org/wp-content/uploads/2019/10/AllGinData-1024x768.png)Figure 2: Claimed cotton gin productivity, 1720 to modern day, coded by credibility and being records, and dated by when the claim was made (not necessarily when the gin was made). Claims that are both relatively credible and higher than previous relatively credible claims are few. The last credible best point before the modern day is an improved version of Whitney’s gin, two years after the original (the original features in the two high non-credible claims slightly earlier).
![](https://aiimpacts.org/wp-content/uploads/2019/10/Historic-cotton-gin-productivity-claims.png)Figure 3: Historic claimed cotton gin productivity, all time (zoomed out version of Figure 2)
![](https://aiimpacts.org/wp-content/uploads/2019/10/GinRecords-1-1024x768.png)Figure 4: Zoom-in on credible best cotton gins (excluding modern era)
##### **Discontinuity measurement**

For measuring discontinuities, we treat past progress as exponential at each point, but entering a new exponential regime at the fourth point. We confine our investigation to credible records. Given these things, we find the improved Whitney gin to be a 23-year discontinuity over the previous record in this dataset. However the foot gin and Eve’s mill gin appear to be at least one-thousand year discontinuities each.[17](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-17-1359 "See our methodology page page for more details, and our spreadsheet, tab ‘credible record gin calculations’ for our calculations.").

However our data has at least one key gap. Whitney’s original 1793 gin design was almost immediately copied and improved by many people, most notably Hodgen Holmes and Daniel Clark.[18](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-18-1359 "“Mississippi’s first saw gin was constructed in violation of Whitney’s patent rights during the summer of 1795 and put into operation in September. Daniel Clark, Sr., a wealthy planter of Wilkinson County, designed this famous machine after examining drawings made by a traveler who had seen one of Whitney’s gins while on a trip to Georgia.”

Moore, John Hebron. Agriculture in Ante-Bellum Mississippi. Columbia, SC: University of South Carolina Press, 2010, page 21.

In 1796 Hogden Holmes of Augusta, Georgia, patented the adaptation, naming it the saw gin. The suit that followed capped a contentious and socially and legally mediated process from which Eli Whitney emerged as the inventor of the cotton gin.

Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003, p47") The plausible productivity data we have appears to all be for these later variants, which we understand were non-negligibly better than Whitney’s original gin in some way.[19](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-19-1359 "We do have Whitney’s own estimate of his gin’s productivity, but since it is very much larger than estimates of later related gins (see our data), we assume it is an exaggeration:

“Two ginners on the new gin could turn out as much fiber in a day as one hundred foot ginners each averaging twenty-five pounds, in all twenty-five hundred pounds, Whitney estimated.”

Angela Lakwete, Inventing the Cotton Gin: Machine and Myth in Antebellum America (JHU Press, 2003), p49") So we know that Whitney’s gin should be somewhat lower and two years earlier than our first data for Whitney-style gins. This means at most, Whitney’s original gin would be a 25 year discontinuity. If it accounted for even half of the progress since Eve’s mill gin, and we are not missing further innovations between the two, Whitney’s gin would still represent a 13 year discontinuity, and the later improved version would no longer account for a discontinuity of more than ten years.[20](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-20-1359 "See spreadsheet, ‘Estimate of first Whitney gin discontinuity’.") It seems likely to us that Whitney’s gin was at least this revolutionary so we think the Whitney gin probably represented a moderate (10-25 year) discontinuity in pounds of cotton ginned per day.

We are fairly uncertain about whether the two larger discontinuities earlier are real, or due to gaps in our data. We did attempt to collect data for these earlier times (rather than just prior to the Whitney gin), but seem very likely to be missing a lot.

In addition to the size of these discontinuities in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[21](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-21-1359 "See our methodology page for more details.")
##### **Changes in the rate of progress**

Over the history of gin productivity, the average rate of progress became higher. It is unclear whether this happened at a particular point. In our data, it looks as though it happened with the foot gin, in 1747, and that progress went from around .05% per year to around 4% per year (see **[our spreadsheet, tab ‘credible record gin calculations’](https://docs.google.com/spreadsheets/d/1oXOy2cyaSv_9svTXiInlDbOWn1vv6BG19dEQiZk40C8/edit#gid=91839643)**). However our data is too sparse and uncertain to draw firm conclusions from.

#### Cost per value of cotton ginned

As discussed above, pounds of cotton ginned per person-day is not a perfect proxy of the value of a cotton gin, and therefore presumably not exactly what cotton-gin users were aiming for. ‘Cost per value of cotton ginned’ seems closer, if we measure costs inclusively and average across various cotton ginning situations. We did not collect data on this, but can make some inferences about the shape of this trend—and in particular whether Whitney’s gin represented a discontinuity—from what we know about the pounds/person-day figures and other aspects of the situation.

##### Evidence from the trend in pounds of cotton ginned per person-day

We expect that the pounds of cotton ginned per person-day roughly approximates cost per value of cotton ginned, with the following adjustments that we know of:

* Eve’s gin is worse on cost/value than on cotton/person-day because the latter metric doesn’t reflect its large upfront costs.
* Whitney’s gin may be worse on cost per value than it appears, because of its lower-quality cotton output.
* Whitney’s gin may be better than it appears, because it could handle short-staple cotton. However this value seems unlikely to have manifest immediately, since it presumably takes time for cotton users to adjust to a new material.
* Foot gins and barrel gins (e.g. Eve’s) were dangerous to operate, so are worse on cost/value than they appear.
* Foot gins apparently required especially skilled labor, so are worse on cost/value than they appear.
* Barrel gins and Eve gins often ran on non-human power-sources, so are worse on cost/value than they appear.[22](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-22-1359 "“The barrel gin used animal and water power, removing humans as a power source but retaining them as seed cotton suppliers. The self-feeding animal-, wind-, or water-powered Eve gin replaced each of the skilled tasks of the inner with mechanical components.” Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003, p48")

These are several considerations in favor of Whitney’s gin representing more progress on cost/value than on cotton/person-day, and one against. However it is unclear to us whether the downside of lower quality cotton was larger than the other considerations combined, so the overall effect on the expected size of discontinuity from Whitney’s gin seems ambiguous, but probably in favor of larger.

##### Evidence from takeup of gins

The foot gin persisted for at least sixty years after Whitney’s invention.[23](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-23-1359 "“Instead of being replaced, the foot gin outlasted the barrel gin and remained in use to the mid-1860s.”

Lakwete, Angela. Inventing the Cotton Gin: Machine and Myth in Antebellum America. Baltimore, MD: Johns Hopkins University Press, 2003., 40.") This suggests that Whitney’s gin wasn’t radically better on cost per value of cotton ginned than its predecessors, at least for some cotton producers.

On the other hand, apparently there was a rush to manufacture copies of Whitney’s gin, so much so that many mechanics became professional gin-makers, and most plantations had one of the new gins within five years. [24](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-24-1359 "“Because the demand for saw gins became very great during the next few years a number of local mechanics embarked upon careers as professional ginwrights. … By [1800] one [of the new gins] could be found on almost every plantation.” Moore, John Hebron. Agriculture in Ante-Bellum Mississippi. Columbia, SC: University of South Carolina Press, 2010  ")
This suggests that there were situations for which Whitney’s gin was substantially better than alternatives, and situations for which it was worse. This seems like weak evidence that on average across cotton ginning needs it was not radically better than precursors, though there might be narrower metrics we could define on which it was radically better.

##### Evidence from cotton production trends

If the Whitney gin made cotton much cheaper to process, we might expect cotton production at the time to sharply increase. Our impression is that this is a common story about what happened.[25](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-25-1359 "For instance, History.com:

“In 1794, U.S.-born inventor Eli Whitney (1765-1825) patented the cotton gin, a machine that revolutionized the production of cotton by greatly speeding up the process of removing seeds from cotton fiber. By the mid-19th century, cotton had become America’s leading export. Despite its success, the gin made little money for Whitney due to patent-infringement issues. Also, his invention offered Southern planters a justification to maintain and expand slavery even as a growing number of Americans supported its abolition. “

History com Editors, “Cotton Gin and Eli Whitney,” HISTORY, accessed June 18, 2019, https://www.history.com/topics/inventions/cotton-gin-and-eli-whitney.") However the data we could find on this, seemingly from a 1958 history of early US agriculture, suggests that cotton production was already growing rapidly, and continued on a similar trajectory after Whitney’s invention.[26](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-26-1359 "This spreadsheet of ours contains data from this image, which appears to be taken from History of Agriculture in the Southern United States to 1860, Volume 2, given that the author is the same and it is from the same page in the book. The full citation is:

Gray, Lewis Cecil, and Esther Katherine Thompson. History of Agriculture in the Southern United States to 1860. Peter Smith, 1958.") See Figure 5.

This dataset begins in 1790, only a few years before Whitney’s invention. This is enough to see that the trend just before 1793 is much like the trend just after, however we can further verify this by looking at earlier cotton export figures (Figure 7).[27](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-27-1359 "We made a rough collection of export and production data from numerous sources. See here. We have not vetted these, and the collection process was not thorough or reliable. However since the figures are fairly consistent, we expect that they are roughly correct.") Cotton exports appear to closely match overall productivity where the trends overlap, and the pre-1790 export trend appears to be roughly continuous with the rest of the curve, at least if we ignore the aberrantly low 1790 figure.

![](https://aiimpacts.org/wp-content/uploads/2019/04/image-9-1024x633.png)Figure 5: historic cotton production (bales), probably from Gray et al 1958[28](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-28-1359 "Gray, Lewis Cecil, and Esther Katherine Thompson. History of Agriculture in the Southern United States to 1860. Peter Smith, 1958."), see above for elaboration on source, [data here](https://docs.google.com/spreadsheets/d/1YN2r40TkukftMjzlZWJCz34cOC2KdkHxZVa8CuMQa3M/edit?usp=sharing).
![](https://aiimpacts.org/wp-content/uploads/2019/06/Historic-US-cotton-productivity.png)Figure 6: Rough collected figures for US cotton exports and production over period from 1780 – 1830, [data and sources here](https://docs.google.com/spreadsheets/d/193D_4nRLvcqX0Z_hso5g3u3z0Ah-BGEWFCjjkjMbwAo/edit?usp=sharing).
![](https://aiimpacts.org/wp-content/uploads/2019/04/Historic-US-cotton-productivity-close-up-on-1793.png)Figure 7: close up on relevant years from Figure 6
This does not preclude a large change in gin efficacy—perhaps there were other bottlenecks to cotton productivity, or it took time for the gains from Whitney’s gin to manifest in national productivity data. However it does cause us to doubt the story of Whitney’s gin being evidently responsible for massive growth in the cotton industry, which was a reason for suspecting the gin may have represented discontinuous progress. So this is some evidence against Whitney’s gin representing a large discontinuity in cost per value of cotton ginned.

##### Evidence from the 1879 evaluation of ginning technology

This [1879 evaluation of ginning technology](https://books.google.com/books?id=lSxAAQAAMAAJ&pg=PA410&lpg=PA410&dq=forbes+watson+cotton+gin&source=bl&ots=UwUDOiYSbu&sig=ACfU3U3qQXoA_RpNsoj9QASWVJ42Zb0ZPg&hl=en&sa=X&ved=2ahUKEwiArO2PyqjhAhWFAXwKHdTjDHYQ6AEwDnoECAgQAQ#v=onepage&q=forbes%20watson%20cotton%20gin&f=false) reports on extensive measurement trials of different cotton gins. It was seemingly conducted to understand why Indian cotton production lagged behind American. The author says all methods for ginning cotton in India were primitive until recently; he hears that in some places it was done by hand as late as 1859.[29](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-29-1359 "“How primitive, till comparatively a few years since, were the means of cleaning the ‘kuppas’ or seed cotton is best shown by the fact that in the accounts of even 20 years ago repeated mention is made in some districts of picking the wool off the seed by hand…” Watson, Forbes. Report on cotton gins and on the cleaning and quality of Indian cotton. London, Allen, 1879.") This is confusing because if Whitney’s gin really was much better in terms of cost-per-value than the alternatives, it would be surprising if sixty years later the alternatives were still in use. However many alternatives seem clearly more cost-effective than ginning by hand, so this seems like little evidence about Whitney’s gin in particular.

The outputs of the gins in the experiment seem different from (and usually higher than) the outputs for similarly named gins in our dataset. Which might be confusing, but we expect is because there are modest improvements to gin technology over time within particular classes of gin.

In sum, this evidence looks as though it might be informative, but we do not see it as such on consideration.

##### Evidence from historians

We have not thoroughly reviewed popular or academic opinions on the discontinuousness of the cotton gin, but our impression is that a common popular view is that Eli Whitney’s cotton gin was a discontinuous improvement over the state-of-the-art. On the other hand Dr Lakwete, author of *Inventing the Cotton Gin*—a book we found most helpful in this project, and that also won an award for being the best scholarly book published about the history of technology[30](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/#easy-footnote-bottom-30-1359 "“Inventing the Cotton Gin won the 2004 Edelstein Prize given by the Society for the History of Technology for the best scholarly book published about the history of technology in the past three years.”

“Angela Lakwete – Department of History – College of Liberal Arts – Auburn University.” Accessed October 27, 2019. https://cla.auburn.edu/history/people/emeritus/angela-lakwete/. ")—disagrees, actually explicitly saying it was continuous (though she may mean something different by this than we do):

> Collapsing two hundred years of cotton production and roller gin use in North America in to the moment when Eli Whitney invented the toothed gin, Phineas Miller and Judge Johnson marked 1794 as a turning point in southern development. Before, southeners languished without an effective gin for short-staple cotton; afterwards, the cotton economy blossomed. Arguing for discontinuity, the idea allowed the visualization of a moment and a machine that separated the colonial past from the new republic. Continuity, however, marked the history of cotton and the gin in America. Continuity would characterize the first two decades of the nineteenth century, as saw-gin and roller gin makers competed for dominance in the expanding short-staple cotton market.
>
> Lakwete, Angela. *Inventing the Cotton Gin: Machine and Myth in Antebellum America*. Baltimore, MD: Johns Hopkins University Press, 2003. , 71.

##### Conclusions on cost per value of cotton ginned

On the earlier ‘pounds of cotton ginned per person per day’ metric, we estimated that Whitney’s gin was worth around 10-25 years of past progress. Various considerations suggested Whitney’s gin might have been a bigger deal for overall cost-effectiveness of ginning cotton than that calculation suggested, but the quality of cotton was lower. We took this in total as neutral to weakly favoring Whitney’s gin being better than it seemed. We then saw that the Whitney gin was taken up with enthusiasm by a subset of people needing to gin cotton, that it didn’t seem to recognizably affect the growth of US cotton production, and that at least one historian with particular expertise in this topic thinks that progress was relatively continuous. This does not particularly suggest to us that Whitney’s gin represented a large discontinuity in cost per value of cotton ginned, and seems like some evidence against.

Notes
-----


Historic trends in flight airspeed records

Flight airspeed records between 1903 and 1976 contained one greater than 10-year discontinuity: a 19-year discontinuity corresponding to the Fairey Delta 2 flight in 1956.

The average annual growth in flight airspeed markedly increased with the Fairey Delta 2, from 16mph/year to 129mph/year.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

Flight airspeed records are measured relative to particular classes of aircraft, with official rules defined by the Fédération Aéronautique Internationale (FAI). is “the highest airspeed attained by any aircraft of a particular class”.[1](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-1-1624 "“An air speed record is the highest airspeed attained by an aircraft of a particular class. The rules for all official aviation records are defined by Fédération Aéronautique Internationale (FAI),[1] which also ratifies any claims. Speed records are divided into multiple classes with sub-divisions. There are three classes of aircraft: landplanes, seaplanes, and amphibians; then within these classes, there are records for aircraft in a number of weight categories. There are still further subdivisions for piston-engined, turbojet, turboprop, and rocket-engined aircraft. Within each of these groups, records are defined for speed over a straight course and for closed circuits of various sizes carrying various payloads.” “Flight Airspeed Record”. 2019. En.Wikipedia.Org. Accessed May 25 2019. https://en.wikipedia.org/wiki/Flight_airspeed_record.")

### Trends

#### Flight airspeed records

##### Data

We took data from Wikipedia’s list of flight airspeed records[2](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-2-1624 "“Flight Airspeed Record”. 2019. En.Wikipedia.Org. Accessed May 25 2019. https://en.wikipedia.org/wiki/Flight_airspeed_record.") (which we have not verified) and added it to [this spreadsheet](https://docs.google.com/spreadsheets/d/1m7xLC684oPjwnjLGOskV0LyO_JOeQh7G23f55ihPva4/edit?usp=sharing). We understand it to be fastest records across all classes of manned aircraft that are able to take off under their own power, but it is not well explained on the page. We included only official airspeed records. See Figure 1 below.

![](https://lh6.googleusercontent.com/8t3ONdcpzCrC_6h34Pb5XSS4h1MkCt8HAZ-FbzJYJpHykEOCPV4KDAk-3Bt0LGhTvY_iCXAzJotvOABhAq4QflopZdvbvvED4Y4-K4qiWAH1WjLO03YR143gayqc-L_RJpRy1KXS)**Figure 1:** Flight airspeed records over time
##### Discontinuity measurement

We treat the data as linear, and once deem it to have begun a new trend, for the purpose of determining the past rate of progress. [3](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-3-1624 "See our methodology page for more details.") We calculate the size of discontinuities in [**this spreadsheet**](https://docs.google.com/spreadsheets/d/1m7xLC684oPjwnjLGOskV0LyO_JOeQh7G23f55ihPva4/edit?usp=sharing).[4](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-4-1624 "See our methodology page for details.") In 1956, there was a 19-year discontinuity in flight airspeed records with the Fairey Delta 2 flight.

We tabulated a number of other related metrics **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.[5](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-5-1624 "See our methodology page for more details.")
![](https://lh4.googleusercontent.com/of_EyYHBuISjX0q4Yz5NzGxSlm93AsaQBzcCmrgJvIzl1zctzuhaXA_mB6-UaINmQoa1-L0T-2fx717elAoiq5VvpiOX4VPJSW070ebpOy0cZO3vF2mzwkWXtsUhhPsTRcn60OND)Figure 2: Fairey Delta 2[6](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-6-1624 "From Wikimedia Commons: Roland Turner from Birmingham, Great Britain [CC BY-SA 2.0 (https://creativecommons.org/licenses/by-sa/2.0)]"), whose 1956 record represented a 19 year discontinuity.
##### Change in the growth rate

The average annual growth in flight airspeed markedly increased at around the time of the Fairey Delta 2. Airspeed records grew by an average of 16mph/year up until the one before Fairey Delta 2, whereas from that point until 1965 they grew by an average of 129mph/year.[7](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/#easy-footnote-bottom-7-1624 "See spreadsheet for calculations.")

Notes
-----


Comparison of naturally evolved and engineered solutions

*This page describes a project that is in progress, and does not yet have results*

We are comparing naturally evolved and engineered solutions to problems, to learn about regularities that might let us make inferences about artificial intelligence from what we know about naturally evolved intelligence.

Details
-------

### Motivation

Engineers and evolution have faced many similar design problems. For instance, the problem of designing an efficient flying machine. Another instance of a design problem that engineers and evolution have both worked on is designing intelligent machines. We hope that by looking at other instances of engineers and evolution working on similar problems, we will be able to learn more about how future AI systems will compare to evolved intelligences.

### Methods

We will collect examples of optimization problems that engineers and evolution would perform better on if they could. Here are some candidate examples of such problems:

* Flying
* Hovering
* Swimming
* Running
* Traveling long distances
* Traveling quickly
* Jumping
* Balancing
* Height of structure
* Piercing
* Applying compressive force
* Striking
* Tensile strength
* Pumping blood
* Breathing
* Liver function
* Detecting light
* Recording light
* Producing light
* Detecting sound
* Recording sound
* Producing sound
* Heat insulation
* Determining chemical composition of a substance
* Detecting chemical composition in the air
* Adhesiveness
* Picking heavy things up
* Joint activation
* Elasticity
* Toxicity
* Extracting energy from sunlight
* Storing energy

We will then collect the best solutions we can readily find to these design problems, made by human engineers and by evolution respectively, and quantitative data on their performances. We will try to collect this over time, for engineered solutions.

#### Analysis

We will use the data to answer the following questions for different design problems:

1. How long does it take engineers to half, match, double, triple, etc. the performance of evolution’s current best designs?
2. What does the shape of engineers’ performance curve look like around the point where engineers’ solutions first match evolution’s?
3. How efficient (in terms of performance per energy or mass used) are the first solutions that match evolution’s performance compared to evolution’s best solutions?
4. How long does it take engineers to find a more efficient solution after finding an equally good solution in terms of absolute performance?
5. From a design perspective, how similar are engineers’ first equally good solutions to evolution’s best solutions?

We will use patterns in the answers to these questions across technologies to make inferences about the answers for natural and artificial intelligence.

In general, the more similar the answers to these questions turn out to be across design problems, the more strongly we will expect the answers for problems addressed by future AI developments to fit the same patterns.

We expect to make the data publicly available, so that others can check our conclusions, investigate related questions, or use it in other investigations of technology and evolution.