Derivative

The derivative of $y$ with respect to $x$ describes the [https://arbital.com/p/-rate](https://arbital.com/p/-rate) at which $y$ changes, given a change in $x$. In particular, we consider how tiny changes in one variable affect another variable. To take the derivative of a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy), we can draw a line that is [https://arbital.com/p/-tangent](https://arbital.com/p/-tangent) to a graph of the function. The slope of the tangent line is the value of the derivative at that point. The derivative of a function $f(x)$ is itself a function: it returns, for any $x$, the slope of the line that is tangent to $f(x)$ at the point $(x, f(x))$.

[https://arbital.com/p/toc:](https://arbital.com/p/toc:)

## Examples ##

 - The time-derivative of your car's mileage is your car's speed (because your car's speed is how quickly your car's mileage changes *over time*)

 - The time-derivative of your car's speed is your acceleration (because acceleration means how quickly your *speed* is changing over time)

 - The time-derivative of human population size is the birth rate minus the death rate (because if we take the birth rate minus the death rate, that tells how quickly the population is changing over time)

 - The time-derivative of wealth is income minus spending (because if we take income minus spending, that tells us how your wealth is changing over time)

 - The time-derivative of blood pumped through your heart is the flow rate through the aortic valve (rates in general describe how something changes over time)

 - The time-derivative of charge on a capacitor is the current flowing to it (currents in general are time-derivatives of how much total stuff has flowed)

 - The time-derivative of how good of a life you'll have lived is how happy you are right now (according to [hedonic utilitarians](https://foundational-research.org/hedonistic-vs-preference-utilitarianism/))

Okay, let's take another stab at this. Time-derivatives, or derivatives "with respect to time," describe how things change over time. We can take derivatives with respect to other things too.

 - The derivative of your car's mileage with respect to how much fuel you've burned is your miles per gallon (because your mpg describes how your mileage changes per unit of fuel you burn)

 - The derivative of the temperature of a pot of water with respect to how much heat you blast it with is called the heat capacity of water

 - The derivative of [potential energy](https://en.wikipedia.org/wiki/Potential_energy) with respect to altitude is gravitational force (a higher up object has some stored energy that it would release if it fell; a lower down object has less "potential energy." The change in potential energy as you change the altitude is why there's a force in the first place.)

There were two goals with all those examples, one explicit, and one covert. The explicit one was to give you a sense for what derivatives are. The covert one was to quietly suggest that you will never understand the the way the world works unless you understand derivatives. But hey, look at you! You kind of understand derivatives already! Let's get to the math now, shall we?

## Setting Up The Math ##

You just got your new car.

![](http://o.aolcdn.com/commerce/autodata/images/USC30TSC021B021001.jpg)
It's a [Tesla](http://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html) because you care about the environment almost as much as you care about looking awesome. Your mileage is sitting at 0. The world is your oyster. At time $t = 0$, you put your foot on the accelerator. For the next few seconds, your mileage will be $4.7 t^2$, where your mileage is in meters, and $t$ is in seconds since you pressed your foot on the accelerator. Now the first question is this: if that equation tells us how many meters we've traveled after how many seconds, how fast are we going at any given point in time?

The astute reader will have noticed that this was the first example of a time-derivative that we gave: the time-derivative of your car's mileage is your car's speed. Let's think about this *sans* math for a second. If we know where we are at any time, we should be able to figure out how fast we're going. There isn't any extra information we need. The only question is how. Well, we take the derivative of the mileage with respect to time to get our speed. In other words:

$$\frac{\mathrm{d}}{\mathrm{d} t} mileage = speed$$

That means the derivative with respect to $t$, where $t$ is the time in seconds. But we know what the mileage is, *in terms of $t$*. Our mileage is just $4.7 t^2$. So we can write:

$$\frac{\mathrm{d}}{\mathrm{d} t} 4.7 t^2 = speed$$

## Solving The Math ##

Sorry to leave you hanging for a sec, but we're going to start with something a little simpler.

![](http://i66.tinypic.com/1p87pl.png)
$$distance\ traveled = 2t$$

If this is the graph of how far someone has traveled after how many seconds, we can see that every second they go 2 more meters. In other words, they are traveling 2 meters per second, which you might notice is the slope of this line. In general, the derivative of a function is like the slope of the function when you graph it out.

This works fine if our function is something like $distance\ traveled = 2t$. What if our function isn't a line though. What if it's $distance\ traveled = t^2$?

![](http://i64.tinypic.com/htu0pv.png)

Things that aren't lines don't have slopes. So if this is a graph of our distance traveled over time, it's not as easy to see how fast we were going. But let's say we want to see how fast we were going at $t=1$. If we zoom in enough on that curve, it will start to flatten out into a straight line until we can't tell the difference. The slope of *that* line is what gives us our speed. The process of taking a curve like this one, and getting the "slope" at any given point is called "taking the derivative."

Let's take the derivative of $d = t^2$, where $d$ is the distance and $t$ is the time. (We'll take the derivative with respect to $t$). Prepare yourself, now take a look at the graph down there.

![](http://i67.tinypic.com/mcxloj.png)

We know how to find the slope of a line if we're given two points, so we're going to do that, and then slowly move the points together until they're on top of each other. The coordinates of the points are shown above, and we can calculate the slope pretty easily by doing $\frac{\Delta d}{\Delta t}$. This gives us a slope of 2.

Now let's say that our first point is at $(t,t^2)$, that our second point is $h$ units to the right, so it's coordinates are $((t+h),(t+h)^2)$. Now we have:
$$∆d=(t+h)^2-t^2$$
$$∆t=(t+h) - t$$
Algebradabra:
$$∆d=2ht + h^2$$
$$∆t=h$$
$$\frac{\Delta d}{\Delta t}=\frac{2ht + h^2}{h}=2t+h$$

Now as we make $h$ really small, the points get closer and closer together, and the slope of the line becomes $2t$. So when $t$ is $1$, the slope is $2$. And when $t$ is $5$, the slope is $10$.

We say the derivative of $t^2$ is $2t$. With similar logic, you can show that the derivative of $4.7t^2$ is $9.4t$. And *that* means that if you put your foot on the accelerator of your new Tesla at time $t=0$, your speed after $t$ seconds will be $9.4t$. After 1 second, you'll be traveling 9.4 meters per second. After 3 seconds, you'll be going 28.2 meters per second (or 64 mph).

## Concluding ##

That's what derivatives are. The Tesla case was just one example of actually finding the derivative of something. Obviously, if our distance traveled had been some totally different function of time, the derivative would have been different. We found that the derivative of $t^2$ is $2t$. Below is a list of other derivatives. You can imagine that if the function on the left was our distance traveled after a time $t$, the function on the right would be our speed at a time $t$. (All of the $c$'s and $n$'s are constants.)
$$\frac{\mathrm{d} }{\mathrm{d} t}c=0$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct=c$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=2ct$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=3ct^2$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^n=nct^{n-1}$$
$$\frac{\mathrm{d} }{\mathrm{d} t}e^t=e^t$$
$$\frac{\mathrm{d} }{\mathrm{d} t}sin(t)=cos(t)$$
$$\frac{\mathrm{d} }{\mathrm{d} t}cos(t)=-sin(t)$$

If you're up for it, try to use the method we showed for solving derivatives to verify some of these. Good luck!

## See also

If you enjoyed this explanation, consider exploring some of [Arbital's](https://arbital.com/p/3d) other [featured content](https://arbital.com/p/6gg)!

Arbital is made by people like you, if you think you can explain a mathematical concept then consider [https://arbital.com/p/-4d6](https://arbital.com/p/-4d6)!

Eric Bruylant
Michael Cohen
Alexei Andreev
Patrick Stevens


The characteristic of the logarithm

Consider the interpretation of logarithms as [the cost of communicating a message](https://arbital.com/p/45q). Every time the number of possible messages to send doubles, your communication costs increase by the price of a coin, or whatever cheaper [https://arbital.com/p/-storage_medium](https://arbital.com/p/-storage_medium) you have that can communicate one of two messages. It doesn't matter whether the number of possible messages goes from 4 to 8 or whether it goes from 4096 to 8192; in both cases, your costs go up by the price of a coin. It is the factor by which the set grew (or shrank) that affects the cost; not the absolute number of messages added (or removed) from the space of possibilities. If the space of possible messages halves, your costs go down by one coin, regardless of how many possibilities there were before the halving.

Algebraically, writing $f$ for the function that measures your costs, $c(x \cdot 2) =$ $c(x) + c(2),$ and, in general, $c(x \cdot y) =$ $c(x) + c(y),$ where we can interpret $x$ as the number of possible messages before the increase, $y$ as the factor by which the possibilities increased, and $x \cdot y$ as the number of possibilities after the increase.

This is the key characteristic of the logarithm: It says that, when the input goes up by a factor of $y$, the quantity measured goes up by a fixed amount (that depends on $y$). When you see this pattern, you can bet that $c$ is a logarithm function. Thus, whenever something you care about goes up by a fixed amount every time something else doubles, you can measure the thing you care about by taking the logarithm of the growing thing. For example:

- Consider the problem of checking whether a date is contained in a gigantic [sorted list](https://arbital.com/p/sorted_list) of dates. You can do this by jumping to the middle of the list, seeing whether your date is earlier or later than the date in the middle, and thereby cutting the search space in half. Each time you do this, you cut the list of dates you're searching for in half, and so the total number of elements you need to look at goes up by one every time the size of the list doubles. Thus, the cost of searching an ordered list grows logarithmically in the size of the list. See also [https://arbital.com/p/binary_search](https://arbital.com/p/binary_search).
- Consider a colony of bacteria where each bacterium in the colony reproduces once per day. Thus, the size of the colony roughly doubles each day. If you care about how long this colony of bacteria has been growing, you can measure the days by taking the logarithm of the number of bacteria in the colony. The logarithm (base 2) counts how many times the colony has doubled (and the log base 3 counts how many times it has tripled, and so on).
- The length of a number in [https://arbital.com/p/4sl](https://arbital.com/p/4sl) grows more-or-less logarithmically in the magnitude of the number: When the magnitude of the number goes up by a factor of 10, the number of digits it takes to write the number down grows by 1. However, this analogy is not perfect: Sometimes, multiplying a number by two does not increase its length (consider the number 300), and sometimes, dividing a number by 10 does not decrease its length by one digit (consider the number 1). See also [Length isn't quite logarithmic](https://arbital.com/p/log_v_length).

Conversely, whenever you see a $\log_2$ in an equation, you can deduce that someone wants to measure some sort of thing by counting the number of doublings that another sort of thing has undergone. For example, let's say you see an equation where someone takes the $\log_2$ of a [https://arbital.com/p/-1rq](https://arbital.com/p/-1rq). What should you make of this? Well, you should conclude that there is some quantity that someone wants to measure which can be measured in terms of the number of doublings in that likelihood ratio. And indeed there is! It is known as [](https://arbital.com/p/bayesian_evidence), and the key idea is that the strength of evidence for a hypothesis $A$ over its negation $\lnot A$ can be measured in terms of $2 : 1$ updates in favor of $A$ over $\lnot A$. (For more on this idea, see [What is evidence?](https://arbital.com/p/)).

In fact, a given function $f$ such that $f(x \cdot y) = f(x) + f(y)$ is almost guaranteed to be a logarithm function — modulo a few technicalities.

[https://arbital.com/p/checkbox](https://arbital.com/p/checkbox)

[Conditional text depending what's next on the path.](https://arbital.com/p/fixme:)

Nate Soares
Eric Rogstad
Alexei Andreev
Adom Hartell

Summary: Any time you find an output that adds whenever the input multiplies, you're probably looking at a (roughly) logarithmic relationship. For example, imagine storing a number using [digit wheels](https://arbital.com/p/42d). Every time the number goes up by a factor of 10, you need one additional digit wheel: It takes 3 wheels to store the number 500; 4 to store the number 5000; 5 to store the number 50000; and so on. Thus, the relationship between the magnitude of a number and the number of digits it takes to write down is logarithmic. This pattern is the key characteristic of the logarithm, and whenever you see an output adding when the input multiplies, you can measure the output using logarithms.


Group homomorphism

summary(Technical): Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.

A group homomorphism is a function between [groups](https://arbital.com/p/3gd) which "respects the group structure".

#Definition

Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.

#Examples

 - For any group $G$, there is a group homomorphism $1_G: G \to G$, given by $1_G(g) = g$ for all $g \in G$. This homomorphism is always [bijective](https://arbital.com/p/499).
 - For any group $G$, there is a (unique) group homomorphism into the group $\{ e \}$ with one element and the only possible group operation $e * e = e$. This homomorphism is given by $g \mapsto e$ for all $g \in G$. This homomorphism is usually not [injective](https://arbital.com/p/4b7): it is injective if and only if $G$ is the group with one element. (Uniqueness is guaranteed because there is only one *function*, let alone group homomorphism, from any set $X$ to a set with one element.)
 - For any group $G$, there is a (unique) group homomorphism from the group with one element into $G$, given by $e \mapsto e_G$, the identity of $G$. This homomorphism is usually not [surjective](https://arbital.com/p/4bg): it is surjective if and only if $G$ is the group with one element. (Uniqueness is guaranteed this time by the property proved below that the identity gets mapped to the identity.)
 - For any group $(G, +)$, there is a bijective group homomorphism to another group $G^{\mathrm{op}}$ given by taking inverses: $g \mapsto g^{-1}$. The group $G^{\mathrm{op}}$ is defined to have underlying set equal to that of $G$, and group operation $g +_{\mathrm{op}} h := h + g$.
 - For any pair of groups $G, H$, there is a homomorphism between $G$ and $H$ given by $g \mapsto e_H$.
 - There is only one homomorphism between the group $C_2 = \{ e_{C_2}, g \}$ with two elements and the group $C_3 = \{e_{C_3}, h, h^2 \}$ with three elements; it is given by $e_{C_2} \mapsto e_{C_3}, g \mapsto e_{C_3}$. For example, the function $f: C_2 \to C_3$ given by $e_{C_2} \mapsto e_{C_3}, g \mapsto h$ is *not* a group homomorphism, because if it were, then $e_{C_3} = f(e_{C_2}) = f(gg) = f(g) f(g) = h h = h^2$, which is not true. (We have used that the identity gets mapped to the identity.)

# Properties

- The identity gets mapped to the identity. ([Proof.](https://arbital.com/p/49z))
- The inverse of the image is the image of the inverse. ([Proof.](https://arbital.com/p/4b1))
- The [image](https://arbital.com/p/3lh) of a group under a homomorphism is another group. ([Proof.](https://arbital.com/p/4b4))
- The composition of two homomorphisms is a homomorphism. ([Proof.](https://arbital.com/p/4b6))


Cyclic group

summary(technical):  A [group](https://arbital.com/p/3gd) $G$ is **cyclic** if it has a single [generator](https://arbital.com/p/generator_mathematics): there is one [element](https://arbital.com/p/element_mathematics) $g$ such that every element of the group is a [power](https://arbital.com/p/) of $g$.

# Definition

A cyclic group is a group $(G, +)$ (hereafter abbreviated as simply $G$) with a single generator, in the sense that there is some $g \in G$ such that for every $h \in G$, there is $n \in \mathbb{Z}$ such that $h = g^n$, where we have written $g^n$ for $g + g + \dots + g$ (with $n$ terms in the summand).
That is, "there is some element such that the group has nothing in it except powers of that element".

We may write $G = \langle g \rangle$ if $g$ is a generator of $G$.

# Examples

 - $(\mathbb{Z}, +) = \langle 1 \rangle = \langle -1 \rangle$
 - The group with two elements (say $\{ e, g \}$ with identity $e$ with the only possible group operation $g^2 = e$) is cyclic: it is generated by the non-identity element. Note that there is no requirement that the powers of $g$ be distinct: in this case, $g^2 = g^0 = e$.
 - The integers [modulo](https://arbital.com/p/modular_arithmetic) $n$ form a cyclic group under addition, for any $n$: it is generated by $1$ (or, indeed, by $n-1$).
 - The [symmetric groups](https://arbital.com/p/497) $S_n$ for $n > 2$ are *not* cyclic. This can be deduced from the fact that they are not [abelian](https://arbital.com/p/3h2) (see below).

# Properties

## Cyclic groups are [abelian](https://arbital.com/p/3h2)
Suppose $a, b \in G$, and let $g$ be a generator of $G$. Suppose $a = g^i, b = g^j$. Then $ab = g^i g^j = g^{i+j} = g^{j+i} = g^j g^i = ba$.

## Cyclic groups are [countable](https://arbital.com/p/2w0)
The elements of a cyclic group are nothing more nor less than $\{ g^0, g^1, g^{-1}, g^2, g^{-2}, \dots \}$, which is an enumeration of the group (possibly with repeats).

Eric Bruylant
Mark Chimes
Patrick Stevens

Summary: The cyclic [groups](https://arbital.com/p/3gd) are the simplest kind of group; they are the groups which can be made by simply "repeating a single element many times". For example, the rotations of a polygon.


Integer

An **integer** is a [https://arbital.com/p/-54y](https://arbital.com/p/-54y) that can be represented as either a [https://arbital.com/p/-45h](https://arbital.com/p/-45h) or its [https://arbital.com/p/-additive_inverse](https://arbital.com/p/-additive_inverse). -4, 0, and 1,003 are examples integers. 499.99 is not an integer. Integers are real numbers; they are [rational numbers](https://arbital.com/p/-4zq); and they are not [fractions](https://arbital.com/p/-fraction) or [decimals](https://arbital.com/p/-decimal).

## A Mathier Definition ##

Instead of describing the properties of an integer we'll describe the membership rules for the [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz) $\mathbb{Z}$. After we're done, anything that's been allowed into $\mathbb{Z}$ counts as an integer.

Start by putting $0$ and $1$ into $\mathbb{Z}$. Now, pick an element of $\mathbb{Z}$, pick another element of $\mathbb{Z}$, and add them together (you can pick the same element twice). Is that number in $\mathbb{Z}$ yet? No?! Well let's put it in there fast. We can do the same thing as before except instead of adding, we subtract, and if the difference isn't in $\mathbb{Z}$ yet, we put it in there. Anything that could be let into $\mathbb{Z}$ with these procedures is an integer.

This is not an efficient algorithm for building out $\mathbb{Z}$, but it does show the primary motivation for having integers in the first place. Natural numbers (positive integers) are closed under addition, meaning that if you add any two elements in the set, the sum will be in the set, but natural numbers are not closed under subtraction. Integers are what you get when you expand natural numbers to make a set that is closed under subtraction as well.

## Formal construction

Given access to the set $\mathbb{N}$ of [natural numbers](https://arbital.com/p/45h), we may construct $\mathbb{Z}$ as follows.
Take the collection of all pairs $(a, b)$ of natural numbers, and take the [https://arbital.com/p/-quotient](https://arbital.com/p/-quotient) by the [https://arbital.com/p/-53y](https://arbital.com/p/-53y) $\sim$ such that $(a,b) \sim (c,d)$ if and only if $a+d = b+c$.
(The intuition is that the pair $(a,b)$ stands for the integer $a-b$, and we take the quotient so that any given integer has just one representative.)

Writing $[https://arbital.com/p/a,b](https://arbital.com/p/a,b)$ for the equivalence class of the pair $(a,b)$, we define the [https://arbital.com/p/-55j](https://arbital.com/p/-55j) structure as:

- $[https://arbital.com/p/a,b](https://arbital.com/p/a,b) + [https://arbital.com/p/c,d](https://arbital.com/p/c,d) = [https://arbital.com/p/a+c,b+d](https://arbital.com/p/a+c,b+d)$
- $[b](https://arbital.com/p/a,) \times [d](https://arbital.com/p/c,) = [bc+ad](https://arbital.com/p/ac+bd,)$
- $[https://arbital.com/p/a,b](https://arbital.com/p/a,b) \leq [https://arbital.com/p/c,d](https://arbital.com/p/c,d)$ if and only if $a+d \leq b+c$.

This does define the structure of a totally ordered ring ([proof](https://arbital.com/p/)).

Alexei Andreev
Eric Bruylant
Nate Soares
Michael Cohen
Dylan Hendrickson
Patrick Stevens
Joe Zeng


Square visualization of probabilities on two events

$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

summary:
$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

We can represent a [probability distribution](https://arbital.com/p/joint_probability_distribution_on_event) $\bP(A,B)$ over  two [events](https://arbital.com/p/event_probability) $A$ and $B$ as a square:

We could also represent $\bP$ by [factoring](https://arbital.com/p/factoring_probability), so using $\bP(A,B) = \bP(A)\; \bP(B \mid A)$ we'd make this picture:

Say we have two [events](https://arbital.com/p/event_probability), $A$ and $B$, and a [probability distribution](https://arbital.com/p/joint_probability_distribution_on_event) $\bP$ over whether or not they happen. We can represent $\bP$ as a square:

So for example, the [probability](https://arbital.com/p/1rh) $\bP(A,B)$ of both $A$ and $B$ occurring is the ratio of \[area of the dark red region\](https://arbital.com/p/the) to \[area of the entire square\](https://arbital.com/p/the):

Visualizing probabilities in a square is neat because we can draw simple pictures that highlight interesting facts about our probability distribution.

Below are some pictures illustrating:

* [independent events](https://arbital.com/p/4cf) (What happens if the columns and the rows in our square *both* line up?)

* [marginal probabilities](https://arbital.com/p/marginal_probability) (If we're looking at a square of probabilities, where's the probability $\bP(A)$ of $A$ or the probability $\bP(\neg B)$?)

* [conditional probabilities](https://arbital.com/p/1rj) (Can we find in the square the probability $\bP(B \mid A)$ of $B$ if we condition on seeing $A$? What about the conditional probability $\bP(A \mid B)$?)

* [factoring a distribution](https://arbital.com/p/factoring_probability) (Can we always write $\bP$ as a square? Why do the columns line up but not the rows?)

* the process of computing [joint probabilities](https://arbital.com/p/1rh) from [factored probabilities](https://arbital.com/p/factoring_probability)

Independent events
===

Here's a picture of the joint distribution of [two independent events](https://arbital.com/p/4cf) $A$ and $B$:

Now the rows for $\bP(B)$ and  $\bP(\neg B)$ line up across the two columns. This is because $\bP(B \mid A) = \bP(B) = \bP(B \mid \neg A)$. When $A$ and $B$ are independent, updating on $A$ or $\neg A$ doesn't change the probability of $B$.

For more on this visualization of independent events, see the aptly named [https://arbital.com/p/4cl](https://arbital.com/p/4cl).

Marginal probabilities
===

We can see the [marginal probabilities](https://arbital.com/p/marginal_probability) of $A$ and $B$ by looking at some of the blocks in our square. For example, to find the probability $\bP(\neg A)$ that $A$ doesn't occur, we just need to add up all the blocks where $\neg A$ happens: $\bP(\neg A) = \bP(\neg A, B) + \bP(\neg A, \neg B)$.

Here's the probability $\bP(A)$ of $A$, and the probability $\bP(\neg A)$ of $\neg A$:

Here's the probability $\bP(\neg B)$ of $\neg B$:

In these pictures we're dividing by the area of the whole square. Since the probability of anything at all happening is 1, we could just leave it out, but it'll be helpful for comparison while we think about conditionals next.

Conditional probabilities
===

We can start with some probability $\bP(B)$, and then *assume* that $A$ is true to get a [conditional probability](https://arbital.com/p/1rj) $\bP(B \mid A)$ of $B$. Conditioning on $A$ being true is like restricting our whole attention to just the possible worlds where $A$ happens:

Then the conditional probability of $B$ given $A$ is the proportion of these $A$ worlds where $B$ also happens:

If instead we condition on $\neg A$, we get:

So our square visualization gives a nice way to see, at a glance, the conditional probabilities of $B$ given $A$ or given $\neg A$:

 We don't get such nice pictures for $\bP(A \mid B)$:

More on this next.

Factoring a distribution
===

Recall the square showing our joint distribution $\bP$:

Notice that in the above square, the reddish blocks for $\bP(A,B)$ and $\bP(A,\neg B)$ are the same width and form a column; and likewise the blueish blocks for $\bP(\neg A,B)$ and $\bP(\neg A,\neg B)$. This is because we chose to [factor](https://arbital.com/p/factoring_probability) our probability distribution starting with $A$:

$$\bP(A,B) = \bP(A) \bP( B \mid A)\ .$$

Let's use the [equivalence](https://arbital.com/p/event_variable_equivalence) between [events](https://arbital.com/p/event_probability) and [binary random variables](https://arbital.com/p/binary_variable), so if we say $\bP( B= \true \mid A= \false)$ we mean $\bP(B \mid \neg A)$. For any choice of truth values $t_A \in \{\true, \false\}$ and $t_B \in \{\true, \false\}$, we have

$$\bP(A = t_A,B= t_B) = \bP(A= t_A)\; \bP( B= t_B \mid A= t_A)\ .$$

The first factor $\bP(A = t_A)$ tells us how wide to make the red column $(\bP(A = \true))$ relative to the blue column $(\bP(A = \false))$. Then the second factor $\bP( B= t_B \mid A= t_A)$ tells us the proportions of dark $(B = \true)$ and light $(B = \false)$ within the column for $A = t_A$.

We could just as well have factored by $B$ first:

$$\bP(A = t_A,B= t_B) = \bP(B= t_B)\; \bP( A= t_A \mid B= t_b)\ .$$

Then we'd draw a picture like this:

By the way, earlier when we factored by $A$ first, we got simple pictures of the probabilities $\bP(B \mid A)$ for $B$ conditioned on $A$.  Now that we're factoring by $B$ first, we have simple pictures for the conditional probability $\bP(A \mid B)$:

and for the conditional probability $\bP(A \mid \neg B)$:

Computing joint probabilities from factored probabilities
===

Let's say we know the factored probabilities for $A$ and $B$, factoring by $A$. That is, we know $\bP(A = \true)$, and we also know $\bP(B = \true \mid A = \true)$ and $\bP(B = \true \mid A = \false)$. How can we recover the joint probability $\bP(A = t_A, B = t_B)$ that $A = t_A$ is the case and also $B = t_B$ is the case?

Since

$$\bP(B = \false \mid A = \true) = \frac{\bP(A = \true, B = \false)}{\bP(A = \true)}\ ,$$

we can multiply the prior $\bP(A)$ by the conditional $\bP(\neg B \mid A)$ to get the joint $\bP(A, \neg B)$:

$$\bP(A = \true)\; \bP(B = \false \mid A = \true) = \bP(A = \true, B = \false)\ .$$

If we do this at the same time for all the possible truth values $t_A$ and $t_B$, we get back the full joint distribution:


Symmetric group

The notion that group theory captures the idea of "symmetry" derives from the notion of the symmetric group, and the very important theorem due to Cayley that every group is a subgroup of a symmetric group.

# Definition

Let $X$ be a [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz). A [bijection](https://arbital.com/p/499) $f: X \to X$ is a *permutation* of $X$.
Write $\mathrm{Sym}(X)$ for the set of permutations of the set $X$ (so its elements are functions).

Then $\mathrm{Sym}(X)$ is a group under the operation of composition of functions; it is the *symmetric group on $X$*.
(It is also written $\mathrm{Aut}(X)$, for the *automorphism group*.)

We write $S_n$ for $\mathrm{Sym}(\{ 1,2, \dots, n\})$, the *symmetric group on $n$ elements*.

# Elements of $S_n$

We can represent a permutation of $\{1,2,\dots, n\}$ in two different ways, each of which is useful in different situations.

## Double-row notation

Let $\sigma \in S_n$, so $\sigma$ is a function $\{1,2,\dots,n\} \to \{1,2,\dots,n\}$.
Then we write $$\begin{pmatrix}1 & 2 & \dots & n \\ \sigma(1) & \sigma(2) & \dots & \sigma(n) \\ \end{pmatrix}$$
for $\sigma$.
This has the advantage that it is immediately clear where every element goes, but the disadvantage that it is quite hard to see the properties of an element when it is written in double-row notation (for example, "$\sigma$ cycles round five elements" is hard to spot at a glance), and it is not very compact.

## Cycle notation

[Cycle notation](https://arbital.com/p/49f) is a different notation, which has the advantage that it is easy to determine an element's order and to get a general sense of what the element does.
Every element of $S_n$ [can be expressed in ](https://arbital.com/p/49k).

## Product of transpositions

It is a useful fact that every permutation in a (finite) symmetric group [may be expressed](https://arbital.com/p/4cp) as a product of [transpositions](https://arbital.com/p/4cn).

# Examples

- The group $S_1$ is the group of permutations of a one-point set. It contains the identity only, so $S_1$ is the trivial group.
- The group $S_2$ is isomorphic to the [https://arbital.com/p/-47y](https://arbital.com/p/-47y) of order $2$. It contains the identity map and the map which interchanges $1$ and $2$.

Those are the only two [abelian](https://arbital.com/p/3h2) symmetric groups.
Indeed, in cycle notation, $(123)$ and $(12)$ do not commute in $S_n$ for $n \geq 3$, because $(123)(12) = (13)$ while $(12)(123) = (23)$.

- The group $S_3$ contains the following six elements: the identity, $(12), (23), (13), (123), (132)$. It is isomorphic to the [https://arbital.com/p/-4cy](https://arbital.com/p/-4cy) $D_6$ on three vertices. ([Proof.](https://arbital.com/p/group_s3_isomorphic_to_d6))

# Why we care about the symmetric groups

A very important (and rather basic) result is [Cayley's Theorem](https://arbital.com/p/49b), which states the link between group theory and symmetry.

%%%knows-requisite([https://arbital.com/p/4bj](https://arbital.com/p/4bj)):
# Conjugacy classes of $S_n$

It is a useful fact that the conjugacy class of an element in $S_n$ is precisely the set of elements which share its [cycle type](https://arbital.com/p/4cg). ([Proof.](https://arbital.com/p/4bh))
We can therefore [list the conjugacy classes](https://arbital.com/p/4bk) of $S_5$ and their sizes.
%%%

# Relationship to the [https://arbital.com/p/-4hf](https://arbital.com/p/-4hf)

The [https://arbital.com/p/-4hf](https://arbital.com/p/-4hf) $A_n$ is defined as the collection of elements of $S_n$ which can be made by an even number of [transpositions](https://arbital.com/p/4cn). This does form a group ([proof](https://arbital.com/p/4hg)).

%%%knows-requisite([https://arbital.com/p/4h6](https://arbital.com/p/4h6)):
In fact $A_n$ is a [https://arbital.com/p/-4h6](https://arbital.com/p/-4h6) of $S_n$, obtained by taking the quotient by the [sign homomorphism](https://arbital.com/p/4hk).
%%%


Bijective function

A bijective function is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) which has an inverse. Equivalently, it is both [injective](https://arbital.com/p/injective_function) and [surjective](https://arbital.com/p/surjective_function).

Eric Rogstad
Mark Chimes
Patrick Stevens


Cayley's Theorem on symmetric groups

Cayley's Theorem states that every group $G$ appears as a certain subgroup of the [https://arbital.com/p/-497](https://arbital.com/p/-497) $\mathrm{Sym}(G)$ on the [https://arbital.com/p/-3gz](https://arbital.com/p/-3gz) of $G$.

# Formal statement

Let $G$ be a group.
Then $G$ is [isomorphic](https://arbital.com/p/49x) to a subgroup of $\mathrm{Sym}(G)$.

# Proof

Consider the left regular [action](https://arbital.com/p/3t9) of $G$ on $G$: that is, the function $G \times G \to G$ given by $(g, h) \mapsto gh$.
This [induces a homomorphism](https://arbital.com/p/49c) $\Phi: G \to \mathrm{Sym}(G)$ given by [https://arbital.com/p/-currying](https://arbital.com/p/-currying): $g \mapsto (h \mapsto gh)$.

Now the following are equivalent:

- $g \in \mathrm{ker}(\Phi)$ the [kernel](https://arbital.com/p/49y) of $\Phi$
- $(h \mapsto gh)$ is the identity map
- $gh = h$ for all $h$
- $g$ is the identity of $G$

Therefore the kernel of the homomorphism is trivial, so it is injective.
It is therefore bijective onto its [image](https://arbital.com/p/3lh), and hence an isomorphism onto its image.

Since [the image of a group under a homomorphism is a subgroup of the codomain of the homomorphism](https://arbital.com/p/4b4), we have shown that $G$ is isomorphic to a subgroup of $\mathrm{Sym}(G)$.


Group action induces homomorphism to the symmetric group

Just as we can [curry](https://arbital.com/p/currying) functions, so we can "curry" [homomorphisms](https://arbital.com/p/47t) and [actions](https://arbital.com/p/3t9).

Given an action $\rho: G \times X \to X$ of group $G$ on set $X$, we can consider what happens if we fix the first argument to $\rho$. Writing $\rho(g)$ for the induced map $X \to X$ given by $x \mapsto \rho(g, x)$, we can see that $\rho(g)$ is a [bijection](https://arbital.com/p/499).

Indeed, we claim that $\rho(g^{-1})$ is an inverse map to $\rho(g)$.
Consider $\rho(g^{-1})(\rho(g)(x))$.
This is precisely $\rho(g^{-1})(\rho(g, x))$, which is precisely $\rho(g^{-1}, \rho(g, x))$.
By the definition of an action, this is just $\rho(g^{-1} g, x) = \rho(e, x) = x$, where $e$ is the group's identity.

We omit the proof that $\rho(g)(\rho(g^{-1})(x)) = x$, because it is nearly identical.

That is, we have proved that $\rho(g)$ is in $\mathrm{Sym}(X)$, where $\mathrm{Sym}$ is the [https://arbital.com/p/-497](https://arbital.com/p/-497); equivalently, we can view $\rho$ as mapping elements of $G$ into $\mathrm{Sym}(X)$, as well as our original definition of mapping elements of $G \times X$ into $X$.

# $\rho$ is a homomorphism in this new sense

It turns out that $\rho: G \to \mathrm{Sym}(X)$ is a homomorphism.
It suffices to show that $\rho(gh) = \rho(g) \rho(h)$, where recall that the operation in $\mathrm{Sym}(X)$ is composition of permutations.

But this is true: $\rho(gh)(x) = \rho(gh, x)$ by definition of $\rho(gh)$; that is $\rho(g, \rho(h, x))$ because $\rho$ is a group action; that is $\rho(g)(\rho(h, x))$ by definition of $\rho(g)$; and that is $\rho(g)(\rho(h)(x))$ by definition of $\rho(h)$ as required.


Cycle notation in symmetric groups

There is a convenient way to represent the elements of a [https://arbital.com/p/-497](https://arbital.com/p/-497) on a finite set.

# $k$-cycle

A $k$-cycle is a member of $S_n$ which moves $k$ elements to each other cyclically.
That is, letting $a_1, \dots, a_k$ be distinct in $\{1,2,\dots,n\}$, a $k$-cycle $\sigma$ is such that $\sigma(a_i) = a_{i+1}$ for $1 \leq i < k$, and $\sigma(a_k) = a_1$, and $\sigma(x) = x$ for any $x \not \in \{a_1, \dots, a_k \}$.

We have a much more compact notation for $\sigma$ in this case: we write $\sigma = (a_1 a_2 \dots a_k)$.
(If spacing is ambiguous, we put in commas: $\sigma = (a_1, a_2, \dots, a_k)$.)
Note that there are several ways to write this: $(a_1 a_2 \dots a_k) = (a_2 a_3 \dots a_k a_1)$, for example.
It is conventional to put the smallest $a_i$ at the start.

Note also that a cycle's inverse is extremely easy to find: the inverse of $(a_1 a_2 \dots a_k)$ is $(a_k a_{k-1} \dots a_1)$.

For example, the double-row notation $$\begin{pmatrix}1 & 2 & 3 \\ 2 & 3 & 1 \\ \end{pmatrix}$$
is written as $(123)$ or $(231)$ or $(312)$ in cycle notation.

However, it is unclear without context which symmetric group $(123)$ lies in: it could be $S_n$ for any $n \geq 3$.
Similarly, $(145)$ could be in $S_n$ for any $n \geq 5$.

# General elements, not just cycles

Not every element of $S_n$ is a cycle. For example, the following element of $S_4$ has [order](https://arbital.com/p/4cq) $2$ so could only be a $2$-cycle, but it moves all four elements:
$$\begin{pmatrix}1 & 2 & 3 & 4 \\ 2 & 1 & 4 & 3 \\ \end{pmatrix}$$

However, it may be written as the composition of the two cycles $(12)$ and $(34)$: it is the result of applying one and then the other.
Note that since the cycles are disjoint (having no elements in common), [it doesn't matter in which order we perform them](https://arbital.com/p/49g).
It is a very important fact that [every permutation may be written as the product of disjoint cycles](https://arbital.com/p/49k).
If $\sigma$ is a permutation obtained by first doing cycle $c_1 = (a_1 a_2 \dots a_k)$, then by doing cycle $c_2$, then cycle $c_3$, we write $\sigma = c_3 c_2 c_1$; this is by analogy with function composition, indicating that the first permutation to apply is on the rightmost end of the expression.
(Be aware that some authors differ on this.)

## Order of an element

Firstly, a cycle has [order](https://arbital.com/p/4cq) equal to its length.
Indeed, the cycle $(a_1 a_2 \dots a_k)$ has the effect of rotating $a_1 \mapsto a_2 \mapsto a_3 \dots \mapsto a_k \mapsto a_1$, and if we do this $k$ times we get back to where we started.
(And if we do it fewer times - say $i$ times - we can't get back to where we started: $a_1 \mapsto a_{i+1}$.)

Now, suppose we have an element in disjoint cycle notation: $(a_1 a_2 a_3)(a_4 a_5)$, say, where all the $a_i$ are different.
Then the order of this element is $3 \times 2 = 6$, because:

- $(a_1 a_2 a_3)$ and $(a_4 a_5)$ are disjoint and hence commute, so $[https://arbital.com/p/(](https://arbital.com/p/()^n = (a_1 a_2 a_3)^n (a_4 a_5)^n$
- $(a_1 a_2 a_3)^n (a_4 a_5)^n$ is the identity if and only if $(a_1 a_2 a_3)^n = (a_4 a_5)^n = e$ the identity, because otherwise (for instance, if $(a_1 a_2 a_3)^n$ is not the identity) it would move $a_1$.
- $(a_1 a_2 a_3)^n$ is the identity if and only if $n$ is divisible by $3$, since $(a_1 a_2 a_3)$'s order is $3$.
- $(a_4 a_5)^n$ is the identity if and only if $n$ is divisible by $2$.

This reasoning generalises: the order of an element in disjoint cycle notation is equal to the [https://arbital.com/p/-least_common_multiple](https://arbital.com/p/-least_common_multiple) of the lengths of the cycles.

# Examples

- The element $\sigma$ of $S_5$ given by first performing $(123)$ and then $(345)$ is $(345)(123) = (12453)$. Indeed, the first application takes $1$ to $2$ and the second application does not affect the resulting $2$, so $\sigma$ takes $1$ to $2$; the first application takes $2$ to $3$ and the second application takes the resulting $3$ to $4$, so $\sigma$ takes $2$ to $4$; the first application does not affect $4$ and the second application takes $4$ to $5$, so $\sigma$ takes $4$ to $5$; and so on.

This example suggests a general procedure for expressing a permutation which is already in cycle form, in *disjoint* cycle form. It turns out that [this can be done in an essentially unique way](https://arbital.com/p/49k).

## Cycle type

The [https://arbital.com/p/-4cg](https://arbital.com/p/-4cg) is given by taking the list of lengths of the cycles in the disjoint cycle form.


Disjoint cycles commute in symmetric groups

Consider two [cycles](https://arbital.com/p/49f) $(a_1 a_2 \dots a_k)$ and $(b_1 b_2 \dots b_m)$ in the [https://arbital.com/p/-497](https://arbital.com/p/-497) $S_n$, where all the $a_i, b_j$ are distinct.

Then it is the case that the following two elements of $S_n$ are equal:

- $\sigma$, which is obtained by first performing the permutation notated by $(a_1 a_2 \dots a_k)$ and then by performing the permutation notated by $(b_1 b_2 \dots b_m)$
- $\tau$, which is obtained by first performing the permutation notated by $(b_1 b_2 \dots b_m)$ and then by performing the permutation notated by $(a_1 a_2 \dots a_k)$

Indeed, $\sigma(a_i) = (b_1 b_2 \dots b_m)[https://arbital.com/p/(](https://arbital.com/p/() = (b_1 b_2 \dots b_m)(a_{i+1}) = a_{i+1}$ (taking $a_{k+1}$ to be $a_1$), while $\tau(a_i) = (a_1 a_2 \dots a_k)[https://arbital.com/p/(](https://arbital.com/p/() = (a_1 a_2 \dots a_k)(a_i) = a_{i+1}$, so they agree on elements of $(a_1 a_2 \dots a_k)$.
Similarly they agree on elements of $(b_1 b_2 \dots b_m)$; and they both do not move anything which is not an $a_i$ or a $b_j$.
Hence they are the same permutation: they act in the same way on all elements of $\{1,2,\dots, n\}$.

This reasoning generalises to more than two disjoint cycles, to show that disjoint cycles commute.

Summary: In a symmetric group, if we are applying a collection of permutations which are each disjoint cycles, we get the same result no matter the order in which we perform the cycles.


Disjoint cycle notation is unique


Pi

Pi, usually written $π$, is a number equal to the ratio of a circle's [https://arbital.com/p/-circumference](https://arbital.com/p/-circumference) to its [https://arbital.com/p/-diameter](https://arbital.com/p/-diameter). The value of $π$ is approximately $3.141593$.

![](http://rlv.zcache.ca/mathematical_definition_of_pi_ceramic_tiles-r78c5d7b1adfa46d7b24e2c8bc3bb3f65_agtk1_8byvr_324.jpg)

If the length of a curve seems like an ill-defined concept to you (maybe you only understand how lines could have lengths), consider bigger and bigger [regular polygons](https://arbital.com/p/-regular_polygon) that make better and better approximations of the circle. As the number of sides $N$ of the polygon goes to $∞$, the perimeter will approach a length of $π$ times the diameter.

One could also define $π$ to be the area of a circle divided the area of a square, whose edge is the radius of the circle.

## What Kind of Number It Is ##

It's not an [https://arbital.com/p/48l](https://arbital.com/p/48l).

![](https://sensemadehere.files.wordpress.com/2015/03/pi.png?w=584)

If the diameter here is 1, then the perimeter of the hexagon is 3, the perimeter of the square is 4, and the circumference of the circle is in between. There are no integers between 3 and 4.

It's [not rational](https://en.wikipedia.org/wiki/Proof_that_%CF%80_is_irrational), [nor is it algebraic](https://en.wikipedia.org/wiki/Lindemann%E2%80%93Weierstrass_theorem#Transcendence_of_e_and_.CF.80). It's transcendental.

Eric Bruylant
Michael Cohen
Eric Rogstad


Complexity theory

**Complexity theory** is the study of the efficiency of algorithms with respect to several metrics, usually time and memory usage. Complexity theorists aim to classify different problems into [classes of difficulty](https://arbital.com/p/complexity_class) and study the relations that hold between the classes.

When studying [computability](https://arbital.com/p/), we are concerned with the identification of that which is or not computable in an ideal sense, without worrying about time or memory limitations.

However, often in practice we have to be more pragmatic. A program which takes a [googol](https://arbital.com/p/42m) years to run is not going to see much use. If you need more [GbBytes](https://arbital.com/p/) to solve a computational problem that atoms exist in the universe, you may as well go ahead and declare the problem unsolvable for all practical purposes.

**Complexity theory** raises the standards of computability in drawing the boundary between that which you can do with a computer and that which you cannot. It concerns the study of the [asymptotic behavior](https://arbital.com/p/oh_notation) of programs when fed inputs of growing size, in terms of the resources they consume. The kind of resources with which complexity theorists work more often are the *time* a program takes to finish and the highest *memory usage* %%note:the memory usage is frequently called **space complexity**%% in any given point of the execution.

Complexity theory allows us to have a deeper understanding of what makes an algorithm efficient, which in turn allows us to develop better and faster algorithms. Surprisingly enough, it turns out that proving that some computational problems are hard to solve has incredibly important practical applications in [https://arbital.com/p/-cryptography](https://arbital.com/p/-cryptography).

----

The abstract framework in which we develop this theory are [Turing machines](https://arbital.com/p/) and [decision problems](https://arbital.com/p/). In this context, the [time complexity](https://arbital.com/p/) is associated with the number of steps a TM takes to halt and return an output, while the [space complexity](https://arbital.com/p/) corresponds to the length of the tape we would need for the TM to never fall off when moving left or right.

One may worry that the complexity measures are highly dependent on the choice of computational framework used. After all, if we allow our TM to skip two spaces per step each time it moves it is going to take potentially half the time to compute something. However, the asymptotic behavior of complexity is [surprisingly robust](https://arbital.com/p/robustness_of_tm), though there are [some caveats](https://arbital.com/p/failure_of_strong_CT).

The most interesting characterization of complexity comes in the form of [complexity classes](https://arbital.com/p/), which break down the family of [decision problems](https://arbital.com/p/) into sets of problems which can be solved with particular constraints.

Perhaps the most important complexity class is [$P$](https://arbital.com/p/), the class of decision problems which can be efficiently computed %%note:For example, checking whether a graph is connected or not%%. The second best known class is [$NP$](https://arbital.com/p/), the class of problems whose solutions can be easily checked %%note:An example is factoring a number: it is hard to factor $221$, but easy to multiply $13$ and $17$ and check that $13 \cdot 17 = 221$%% . It is an open problem whether those two classes are [one and the same](https://arbital.com/p/4bd); that is, that every problem whose solutions are easy to check is also easy to solve.

There are many more important complexity classes, and it can be daunting to contemplate the sheer variety with which complexity theory deals. Feel free to take a guided tour though the [complexity zoo](https://arbital.com/p/4b9) if you want an overview of some of the most relevant.

An important concept is that of a [reduction](https://arbital.com/p/). Some complexity classes have problems such that if you were able to solve them efficiently you could translate other problems in the class to this one and solve them efficiently. Those are called [complete problems of a complexity class](https://arbital.com/p/).

-----

This page is meant to be a starting point to learn complexity theory from an entry level. If there is any concept which feels mysterious to you, try exploring the greenlinks in their order of appearance. If you feel like the concepts presented are too basic, try a different lens.

Eric Bruylant
Eric Rogstad
Adom Hartell
Jaime Sevilla Molina

Summary: Study of the computational resources needed to solve a problem, usually time and memory


Group isomorphism

A group isomorphism is a [https://arbital.com/p/-47t](https://arbital.com/p/-47t) which is [bijective](https://arbital.com/p/499).
We say that two groups are *isomorphic* if there is an isomorphism between them.

It turns out that isomorphism is a much more useful concept than true equality of [groups](https://arbital.com/p/-3gd), and it captures the idea that "these two objects are the same group": the isomorphism shows us how to relabel the elements to see that they are indeed the same group.

For example, the trivial group is in some sense "the only group with one element", but it can be instantiated in many different ways: as $(\{ a \}, +_a)$, or $(\{ b \}, +_b)$, and so on (where $+_x$ is the [https://arbital.com/p/-3kb](https://arbital.com/p/-3kb) taking $(x, x)$ to $x$).
They all behave in exactly the same ways for the purpose of group theory, but they are not literally identical.
They are all isomorphic, though: the map $\{a \} \to \{ b \}$ given by $a \mapsto b$ is an isomorphism of the respective groups.

Two groups are isomorphic if and only if they have the same [Cayley table](https://arbital.com/p/cayley_table), possibly with rearrangement of rows/columns and with relabelling of elements.


Kernel of group homomorphism

The kernel of a [https://arbital.com/p/-47t](https://arbital.com/p/-47t) $f: G \to H$ is the collection of all elements $g$ in $G$ such that $f(g) = e_H$ the identity of $H$.

It is important to note that the kernel of any group homomorphism $G \to H$ is always a subgroup of $G$.
Indeed:

- if $f(g_1) = e_H$ and $f(g_2) = e_H$ then $e_H = f(g_1) f(g_2) = f(g_1 g_2)$, so the kernel is closed under $G$'s operation;
- if $f(x) = e_H$ then $e_H = f(e_G) = f(x^{-1} x) = f(x^{-1}) f(x) = f(x^{-1})$ (where we have used that [the image of the identity is the identity](https://arbital.com/p/49z)), so inverses are contained in the putative subgroup;
- $f(e_G) = e_H$ because the image of the identity is the identity, so the identity is contained in the putative subgroup.

It turns out that the notion of "[https://arbital.com/p/-4h6](https://arbital.com/p/-4h6)" coincides exactly with the notion of "kernel of homomorphism". ([Proof.](https://arbital.com/p/4h7))
The "kernel of homomorphism" viewpoint of normal subgroups is much more strongly motivated from the point of view of [https://arbital.com/p/-4c7](https://arbital.com/p/-4c7); Timothy Gowers [considers this to be the correct way](https://gowers.wordpress.com/2011/11/20/normal-subgroups-and-quotient-groups/) to introduce the teaching of normal subgroups in the first place.


Image of the identity under a group homomorphism is the identity

For any [https://arbital.com/p/-47t](https://arbital.com/p/-47t) $f: G \to H$, we have $f(e_G) = e_H$ where $e_G$ is the identity of $G$ and $e_H$ the identity of $H$.

Indeed, $f(e_G) f(e_G) = f(e_G e_G) = f(e_G)$, so premultiplying by $f(e_G)^{-1}$ we obtain $f(e_G) = e_H$.


Under a group homomorphism, the image of the inverse is the inverse of the image

For any [https://arbital.com/p/-47t](https://arbital.com/p/-47t) $f: G \to H$, we have $f(g^{-1}) = f(g)^{-1}$.

Indeed, $f(g^{-1}) f(g) = f(g^{-1} g) = f(e_G) = e_H$, and similarly for multiplication on the left.


The image of a group under a homomorphism is a subgroup of the codomain

Let $f: G \to H$ be a [https://arbital.com/p/-47t](https://arbital.com/p/-47t), and write $f(G)$ for the set $\{ f(g) : g \in G \}$.
Then $f(G)$ is a group under the operation inherited from $H$.

# Proof

To prove this, we must verify the group axioms.
Let $f: G \to H$ be a group homomorphism, and let $e_G, e_H$ be the identities of $G$ and of $H$ respectively.
Write $f(G)$ for the image of $G$.

Then $f(G)$ is closed under the operation of $H$: since $f(g) f(h) = f(gh)$, so the result of $H$-multiplying two elements of $f(G)$ is also in $f(G)$.

$e_H$ is the identity for $f(G)$: it is $f(e_G)$, so it does lie in the image, while it acts as the identity because $f(e_G) f(g) = f(e_G g) = f(g)$, and likewise for multiplication on the right.

Inverses exist, by "the inverse of the image is the image of the inverse".

The operation remains associative: this is inherited from $H$.

Therefore, $f(G)$ is a group, and indeed is a subgroup of $H$.


Expected value

The expected value of an action is the [https://arbital.com/p/-mean](https://arbital.com/p/-mean) numerical outcome of the possible results weighted by their [https://arbital.com/p/-1rf](https://arbital.com/p/-1rf). It may actually be impossible to get the expected value, for example, if a coin toss decides between you getting \$0 and \$10, then we say you get "\$5 in expectation" even though there is no way for you to get \$5.

The expectation of V (often shortened to "the expected V") is how much V you expect to get on average. For example, the expectation of a payoff, or an expected payoff, is how much money you will get on average; the expectation of the duration of a speech, or an expected duration, is how long the speech will last "on average."

Suppose V has discrete possible values, say $V = x_{1},$ or $V = x_{2}, ..., $ or $V = x_{k}$. Let $P(x_{i})$ refer to the probability that $V = x_{i}$. Then the expectation of V is given by:

$$\sum_{i=1}^{k}x_{i}P(x_{i})$$

Suppose V has continuous possible values, x. For instance, let $x \in \mathbb{R}$. Let $P(x)$ be the continuous probability distribution, or $\lim_{dx \to 0}$ of the probability that $x<V<(x+dx)$ divided by $dx$. Then the expectation of V is given by:

$$\int_{-∞}^{∞}xP(x)dx$$

## Importance ##

A common principle of reasoning under uncertainty is that if you are trying to achieve a good G, you should choose the act that maximizes the expectation of G.

Eric Bruylant
Michael Cohen
Alexei Andreev


The composition of two group homomorphisms is a homomorphism

Given two [group homomorphisms](https://arbital.com/p/47t) $f: G \to H$ and $g: H \to K$, the composition $gf: G \to K$ is also a homomorphism.

To prove this, note that $g(f(x)) g(f(y)) = g(f(x) f(y))$ since $g$ is a homomorphism; that is $g(f(xy))$ because $f$ is a homomorphism.


Injective function

A [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) $f: X \to Y$ is *injective* if it has the property that whenever $f(x) = f(y)$, it is the case that $x=y$. Given an element in the [image](https://arbital.com/p/3lm), it came from applying $f$ to exactly one element of the [domain](https://arbital.com/p/3js).

This concept is also commonly called being "one-to-one".
That can be a little misleading to someone who does not already know the term, however, because many people's natural interpretation of "one-to-one" (without otherwise having learnt the term) is that every element of the domain is matched up in a one-to-one way with *every* element of the domain, rather than simply with *some* element of the domain.
That is, a rather natural way of interpreting "one-to-one" is as "[bijective](https://arbital.com/p/499)" rather than "injective".

# Examples

- The function $\mathbb{N} \to \mathbb{N}$ (where $\mathbb{N}$ is the set of [natural numbers](https://arbital.com/p/-45h)) given by $n \mapsto n+5$ is injective: since $n+5 = m+5$ implies $n = m$. Note that this function is not [surjective](https://arbital.com/p/4bg): there is no natural number $k$ such that $k+5 = 2$, for instance, so $2$ is not in the range of the function.
- The function $f: \mathbb{N} \to \mathbb{N}$ given by $f(n) = 6$ for all $n$ is not injective: since $f(1) = f(2)$ but $1 \not = 2$, for instance.


Complexity theory: Complexity zoo

Welcome, visitor! Get ready to exercise your sense of wonder and expand your mind on the **zoo of complexity**.

That where are the cages of the animals, you ask? No cage could contain our specimens, I must say. But you can get a glimpse of their nature through the chalky definitions in each of those blackboards. Where shall we go first?

----

Oh yes! [$P$](https://arbital.com/p/), the class of problems solvable in polynomial time. A popular creature in our installations. It is commonly accepted that this  class encompasses the problems which can be solved efficiently. Now, bear in mind that the classes in this zoo only classify problems attending to how fast the computational power you need to solve them grows compared to the size of the problem you are solving! That means, even if all grow at the same polynomial pace, some of the problems who lurk inside this class need a huge overload of resources from the very beginning, which makes them intractable! Not to talk about the multiplicative factor, which also gets ignored in making this distinction between classes, or the degree of the polynomial which upper bounds the function! Beliefs like that $x$ grows as fast as $1000 x^{42}+10^{100}$ can be misleading if not properly understood.

However, in practice such instances are rare, and $P$ is still used thorough all computer science as a synonym of efficiency - though they pay special attention to the degree and distinguish between functions in $\mathcal{O}(n)$ and functions in $\mathcal{O}(n*log(n))$, even though all programs associated fall within $P$.

----

The next creature is without a doubt the most famous of the zoo! Meet [$NP$](https://arbital.com/p/), the class of problems whose solution are easily checkable. This little creature is the responsible of modern cryptography, which relies on [functions easy to compute but hard to invert](https://arbital.com/p/one_way_functions).

$NP$ also is the main star, together with $P$, of one great show in the zoo. Oh, look! It is about to start!...

*Announcer*: Ladies, gentlemen and children who are today with us! We present to you the main attraction of our bestiary, the infamous [$P$ vs $NP$](https://arbital.com/p/)!

*Announcer*: You may already have seen both of those creatures, but are they two different beasts? Or only two aspects of the same thing?

*Announcer*: Look again at them, and realize. If finding a solution to a problem is easy, then checking whether a given solution is true is also easy! You can always just find the solution again from scratch and check that both coincide. So we conclude that $P\subset NP$. Can the reverse inclusion be true as well?

*Announcer*: Imagine for a moment! The hard problems of creativity, solved at once through the clever application of an efficient algorithm! [Automated short proof writing](https://arbital.com/p/Entscheidungsproble)! Instant reversal of one way functions!

*Announcer*: However, we suspect that they are different, though a proof has remained elusive through the years.

(the public cries: If we have been searching for decades for such a proof and none has been found then you should revise your beliefs and update against that! Why suspect otherwise?)

*Announcer*: We seem to have an exceptionally well educated audience today! Then again, we have been also on the search for a proof of the opposite without fruition. Why give this fact more weight in the Bayesian scheme of the things? There happens to be a somewhat convoluted reason for this. For if $P=NP$, then we would expect proofs to be easier to find, while if $P!=NP$ the opposite is true! So it only makes sense that is more probable that we live in the world where $P!=NP$%%note:For a formalization of this notion check [natural proofs](https://arbital.com/p/). Also, see [Arguments against $P=NP$](https://arbital.com/p/4bf)%%!

*Announcer*: And that's all for today, folks. Keep enjoying your visit though the zoo!

---

(To be continued)

Duncan Wilson
Eric Rogstad
Jaime Sevilla Molina


Real number

A **real number** is any number that can be used to represent a physical quantity.

Intuitively, real numbers are any number that can be found between two [integers](https://arbital.com/p/48l), such as $0,$ $1,$ $-1,$ $\frac{3}{2},$ $\frac{-7}{2},$ [$\pi,$](https://arbital.com/p/49r) [$e$](https://arbital.com/p/e), $100 \cdot \sqrt{2},$ and so on. The set of real numbers is written $\mathbb R.$ You can think of $\mathbb R$ as [$\mathbb Q$](https://arbital.com/p/4zq) extended to include the [irrational numbers](https://arbital.com/p/54z) like $\pi$ and $e$ which can be found between rational numbers but which cannot be completely written out in [https://arbital.com/p/-4sl](https://arbital.com/p/-4sl).

## Definitions of the real numbers

The most commonly used definitions of the real numbers are constructions as extensions of the [rational numbers](https://arbital.com/p/4zq), which involve either [Cauchy sequences](https://arbital.com/p/53b) or [Dedekind cuts](https://arbital.com/p/dedekind_cut).

### Cauchy sequences

Broadly speaking, a Cauchy sequence is a sequence where as the sequence goes on, all the elements past that point get closer and closer together. In the real numbers, every Cauchy sequence [converges](https://arbital.com/p/convergence_analysis) to a real number. However, in the set of rational numbers, not all Cauchy sequences converge to a rational number.
In the set of rationals, a Cauchy sequence which does not converge to a rational number cannot really be said to "converge" at all: the set of rationals is "missing some of the points" that would be required to make every Cauchy sequence converge.

For example, the sequence of fractions of consecutive Fibonacci numbers $1/1, 2/1, 3/2, 5/3, 8/5, \ldots$ gets closer and closer to $\frac{1 + \sqrt{5}}{2}$, but cannot be said to converge to that number because it is not in the set of rational numbers.

For each of these non-convergent Cauchy sequences, we define a new irrational number to "fill in the gap", and for the Cauchy sequences that do converge, we define a real number equal to that rational number.

### Dedekind cuts

A Dedekind cut of a [https://arbital.com/p/-540](https://arbital.com/p/-540) is a [https://arbital.com/p/-partition](https://arbital.com/p/-partition) of that set into two sets so that every element in the first set is [less than](https://arbital.com/p/) every element in the second set, and the second set has no smallest element. The latter restriction requires that the set also be a [perfect set](https://arbital.com/p/) (have no [isolated points](https://arbital.com/p/isolated_point)), in the sense used in topology.

In the real numbers, such a partition will always have the first set having a greatest element, which is known as the least-upper-bound property. However, in the rational numbers, we might come across a partition where the first set does not have such an element.

For example, define a Dedekind cut $(A, B)$ of the rational numbers such that $B = \{x \in \mathbb{Q} \ | \ x > 0 \wedge x^2 > 2\}$ and $A$ is the complement of $B$. In plainer language, $B$ consists of all the numbers greater than $\sqrt{2}$, but because $\sqrt{2}$ doesn't exist in the space of rational numbers, we can't use that to formulate our definition. Obviously every element of $A$ is less than every element of $B$, but $A$ has no greatest element either, because we can create a sequence of numbers in $A$ that gets bigger and bigger (as it approaches $\sqrt{2}$) but never stops at a maximum value.

For each of these "strict cuts" where neither set has a "boundary element", we define a new irrational number to "fill in the gap", just like with the Cauchy sequences. For the Dedekind cuts where one of the sets does have a least or greatest element, we define a real number equal to that rational number.

This definition has the advantage that each real number is represented by a unique Dedekind cut, unlike the Cauchy sequences where multiple sequences can converge to the same number.

Alexei Andreev
Eric Rogstad
Eric Bruylant
Nate Soares
Kevin Clancy
Michael Cohen
M Yass
Patrick Stevens
Joe Zeng


P vs NP

The greatest currently open problem in complexity theory and computer science in general, which talks about the relationship between the class of efficiently solvable problems, [$P$](https://arbital.com/p/), and the class of problems with easily checkable solutions, [$NP$](https://arbital.com/p/).

It is a basic result that [$P\subseteq NP$](https://arbital.com/p/). However, we still do not know whether the reverse implication, $P \supseteq NP$, holds.

A positive, constructive result would have profound implications through all the fields of science and math, not to mention the whole of society.

Alexei Andreev
Mark Chimes
Jaime Sevilla Molina

Summary: Are problems with solutions easy to check also easy to solve?


P vs NP: Arguments against P=NP

#Natural proofs
If $P \neq NP,$ then there are results showing that lower bounds in complexity are inherently harder to prove in a [technical yet natural sense](https://arbital.com/p/natural_proof). In other words, if $P \neq NP$ then proving $P \neq NP$ is hard.

The opposite proposition for $P=NP$ is also expected to be true. That is, it would make sense that if $P=NP,$ then it should be easier to prove, since we could build far more efficient [theorem provers](https://arbital.com/p/).

#Empirical argument
We have been trying to get a fast algorithm for [$NP$-complete](https://arbital.com/p/) problems since the 70s, without success. And take into account that "we" does not only comprise a minor group of theoretical computer scientists, but a whole industry trying to get faster algorithms for commercial purposes.

One could also argue more weakly that if $P=NP$, then evolution could have made use of this advantage to sped up its search process, or create more efficient minds to solve $NP$-complete problems at thoughtspeed.

#The arithmetical hierarchy argument
Some authors have drawn an analogy between the [polynomial hierarchy](https://arbital.com/p/) and the [arithmetical hierarchy](https://arbital.com/p/1mg).

There are results showing that the [arithmetical hierarchy is strict](https://arbital.com/p/), and if the analogy holds then we would have that the polynomial hierarchy is strict as well, [which automatically implies $P \neq NP$](https://arbital.com/p/collapse_of_the_polynomial_hierarchy).

Eric Bruylant
Nate Soares
Eric Rogstad
Jaime Sevilla Molina

Summary:
- Proving $P \neq NP$ is harder if true than proving $P=NP$ if true.
 - After 50 years we haven't found any efficient algorithm for $NP$-complete problems.
 - The polynomial hierarchy is compared to the arithmetical hierarchy, and the analogy implies $P \neq NP$


Surjective function

A function $f:A \to B$ is *surjective* if every $b \in B$ has some $a \in A$ such that $f(a) = b$.
That is, its [codomain](https://arbital.com/p/3lg) is equal to its [image](https://arbital.com/p/3lh).

This concept is commonly referred to as being "onto", as in "The function $f$ is onto."

# Examples

- The function $\mathbb{N} \to \{ 6 \}$ (where $\mathbb{N}$ is the set of [natural numbers](https://arbital.com/p/45h)) given by $n \mapsto 6$ is surjective. However, the same function viewed as a function $\mathbb{N} \to \mathbb{N}$ is not surjective, because it does not hit the number $4$, for instance.
- The function $\mathbb{N} \to \mathbb{N}$ given by $n \mapsto n+5$ is *not* surjective, because it does not hit the number $2$, for instance: there is no $a \in \mathbb{N}$ such that $a+5 = 2$.


Conjugacy class is cycle type in symmetric group

In the [https://arbital.com/p/-497](https://arbital.com/p/-497) on a finite set, the [https://arbital.com/p/-4bj](https://arbital.com/p/-4bj) of an element is determined exactly by its [cycle type](https://arbital.com/p/4cg).

More precisely, two elements of $S_n$ are [conjugate](https://arbital.com/p/group_conjugate) in $S_n$ if and only if they have the same cycle type.

# Proof

## Same conjugacy class implies same cycle type
Suppose $\sigma$ has the cycle type $n_1, \dots, n_k$; write $$\sigma = (a_{11} a_{12} \dots a_{1 n_1})(a_{21} \dots a_{2 n_2}) \dots (a_{k 1} a_{k 2} \dots a_{k n_k})$$
Let $\tau \in S_n$.

Then $\tau \sigma \tau^{-1}(\tau(a_{ij})) = \tau \sigma(a_{ij}) = a_{i (j+1)}$, where $a_{i (n_i+1)}$ is taken to be $a_{i 1}$.

Therefore $$\tau \sigma \tau^{-1} = (\tau(a_{11}) \tau(a_{12}) \dots \tau(a_{1 n_1}))(\tau(a_{21}) \dots \tau(a_{2 n_2})) \dots (\tau(a_{k 1}) \tau(a_{k 2}) \dots \tau(a_{k n_k}))$$
which has the same cycle type as $\sigma$ did.

## Same cycle type implies same conjugacy class

Suppose $$\pi = (b_{11} b_{12} \dots b_{1 n_1})(b_{21} \dots b_{2 n_2}) \dots (b_{k 1} b_{k 2} \dots b_{k n_k})$$
so that $\pi$ has the same cycle type as the $\sigma$ from the previous direction of the proof.

Then define $\tau(a_{ij}) = b_{ij}$, and insist that $\tau$ does not move any other elements.

Now $\tau \sigma \tau^{-1} = \pi$ by the final displayed equation of the previous direction of the proof, so $\sigma$ and $\pi$ are conjugate.

# Example
This result makes it rather easy to [list the conjugacy classes](https://arbital.com/p/4bk) of $S_5$.


Conjugacy class

Given an element $g$ of a [https://arbital.com/p/-3gd](https://arbital.com/p/-3gd) $G$, the conjugacy class of $g$ in $G$ is $\{ x g x^{-1} : x \in G \}$.
It is the collection of elements to which $g$ is [conjugate](https://arbital.com/p/4gk).


Conjugacy classes of the symmetric group on five elements

The [https://arbital.com/p/-497](https://arbital.com/p/-497) $S_5$ on five generators has size $5! = 120$ (where the exclamation mark denotes the [https://arbital.com/p/-factorial](https://arbital.com/p/-factorial) function).
By the result that [in a symmetric group, conjugacy classes coincide with cycle types](https://arbital.com/p/4bh), we can list the conjugacy classes of $S_5$ easily.

# The table

$$\begin{array}{|c|c|c|c|}
\hline
\text{Representative}& \text{Size of class} & \text{Cycle type} & \text{Order of element} \\ \hline
(12345) & 24 & 5 & 5 \\ \hline
(1234) & 30 & 4,1 & 4 \\ \hline
(123) & 20 & 3,1,1 & 3 \\ \hline
(123)(45) & 20 & 3,2 & 6 \\ \hline
(12)(34) & 15 & 2,2,1 & 2 \\ \hline
(12) & 10 & 2,1,1,1 & 2 \\ \hline
e & 1 & 1,1,1,1,1 & 1 \\ \hline
\end{array}$$

# Determining the list of cycle types and sizes

There are five elements to permute; we need to find all the possible ways of grouping them up into disjoint cycles.
We will go through this systematically.
Note first that since there are only five elements to permute, there cannot be any element with a $6$-cycle or higher.

## Those with largest cycle of length $5$

If there is a $5$-cycle, then it permutes everything, so its [cycle type](https://arbital.com/p/4cg) is $5$.
That is, we can take a representative $(12345)$, and this is the only conjugacy class with a $5$-cycle.

Recall that every cycle of length $5$ may be written in five different ways: $(12345)$ or $(23451)$ or $(34512)$, and so on.
Without loss of generality, we may assume $1$ comes at the start (by cycling round the permutation if necessary).

Then there are $4!$ ways to fill in the remaining slots of the cycle (where the exclamation mark denotes the [https://arbital.com/p/-factorial](https://arbital.com/p/-factorial) function).

Hence there are $24$ elements of this conjugacy class.

## Those with largest cycle of length $4$

If there is a $4$-cycle, then it permutes everything except one element, so its cycle type must be $4,1$ (abbreviated as $4$).
That is, we can take a representative $(1234)$, and this is the only conjugacy class with a $4$-cycle.

Either the element $1$ is permuted by the $4$-cycle, or it is not.

- In the first case, we have $4$ possible ways to pick the image $a$ of $1$; then $3$ possible ways to pick the image $b$ of $a$; then $2$ possible ways to pick the image $c$ of $b$; then $c$ must be sent back to $1$.
That is, there are $4 \times 3 \times 2 = 24$ possible $4$-cycles containing $1$.
- In the second case, the cycle does not contain $1$, so there are $3$ possible ways to pick the image $a$ of $2$; then $2$ possible ways to pick the image $b$ of $a$; then $1$ possible way to pick the image $c$ of $b$; then $c$ must be sent back to $2$.
So there are $3 \times 2 \times 1 = 6$ possible $4$-cycles not containing $1$.

This comes to a total of $30$ possible $4$-cycles in this conjugacy class.

## Those with largest cycle of length $3$

Now we have two possible conjugacy classes: $3,1,1$ and $3,2$.

### The $3,1,1$ class

An example representative for this class is $(123)$.

We proceed with a slightly different approach to the $4,1$ case.
Using the notation for the [https://arbital.com/p/-binomial_coefficient](https://arbital.com/p/-binomial_coefficient), we have $\binom{5}{3} = 10$ possible ways to select the numbers which appear in the $3$-cycle.
Each selection has two distinct ways it could appear as a $3$-cycle: the selection $\{1,2,3\}$ can appear as $(123)$ (or the duplicate cycles $(231)$ and $(312)$), or as $(132)$ (or the duplicate cycles $(321)$ or $(213)$).

That is, we have $2 \times 10 = 20$ elements of this conjugacy class.

### The $3,2$ class

An example representative for this class is $(123)(45)$.

Again, there are $\binom{5}{3} = 10$ possible ways to select the numbers which appear in the $3$-cycle; having made this selection, we have no further choice about the $2$-cycle.

Given a selection of the elements of the $3$-cycle, as before we have two possible ways to turn it into a $3$-cycle.

We are also given the selection of the elements of the $2$-cycle, but there is no choice about how to turn this into a $2$-cycle because, for instance, $(12)$ is equal to $(21)$ as cycles.
So this time the selection of the elements of the $3$-cycle has automatically determined the corresponding $2$-cycle.

Hence there are again $2 \times 10 = 20$ elements of this conjugacy class.

## Those with largest cycle of length $2$

There are two possible cycle types: $2,2,1$ and $2,1,1,1$.

### The $2,2,1$ class

An example representative for this class is $(12)(34)$.

There are $\binom{5}{2}$ ways to select the first two elements; then once we have done so, there are $\binom{3}{2}$ ways to select the second two.
Having selected the elements moved by the first $2$-cycle, there is only one distinct way to make them into a $2$-cycle, since (for example) $(12)$ is equal to $(21)$ as permutations; similarly the selection of the elements determines the second $2$-cycle.

However, this time we have double-counted each element, since (for example) the permutation $(12)(34)$ is equal to $(34)(12)$ by the result that [disjoint cycles commute](https://arbital.com/p/49g).

Therefore there are $\binom{5}{2} \times \binom{3}{2} / 2 = 15$ elements of this conjugacy class.

### The $2,1,1,1$ class

An example representative for this class is $(12)$.

There are $\binom{5}{2}$ ways to select the two elements which this cycle permutes.
Once we have selected the elements, there is only one distinct way to put them into a $2$-cycle, since (for example) $(12)$ is equal to $(21)$ as permutations.

Therefore there are $\binom{5}{2} = 10$ elements of this conjugacy class.

## Those with largest cycle of length $1$

There is only the identity in this class, so it is of size $1$.


Needs work

Meta tag for pages which need content improvement.

Eric Bruylant
Eliezer Yudkowsky
Eric Rogstad


Church-Turing thesis

The **Church-Turing thesis** (often abbreviated **CT thesis**) states:
> Every effectively computable function is Turing-computable

That is, for every function which can be computed by physical means %%note:So no [hypercomputers](https://arbital.com/p/)%% there exists a [Turing machine](https://arbital.com/p/) which computes exactly that.

The Church-Turing thesis is not a definite mathematical statement, but an inductive statement which affirms that every sensible model of computation we can come up with is equivalent or at least [reducible](https://arbital.com/p/reduction) to the model proposed by Turing. Thus, we cannot prove it in a mathematical sense, but we can [gather evidence for it](https://arbital.com/p/).

For example, this model was proven to coincide with Church's lambda calculus, another universal model of computation, and the equivalence between Church's lambda calculus and Turing's automatic machines is often taken as [evidence](https://arbital.com/p/4j7) that they correctly capture our intuition of "effectively computable".

There are many [consequences](https://arbital.com/p/) of the CT thesis for computer science in general, artificial intelligence, [epistemology](https://arbital.com/p/11w), and other fields of knowledge.

Jaime Sevilla Molina
Alexei Andreev
Eric Rogstad
Connor Flexman
Eric Bruylant
Patrick Stevens

Summary: Every [https://arbital.com/p/-effectively_computable](https://arbital.com/p/-effectively_computable) [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) is [Turing-computable](https://arbital.com/p/Turing_machine).


Properties of the logarithm

With a [solid interpretation of logarithms](https://arbital.com/p/45q) under our belt, we are now in a position to look at the basic properties of the logarithm and understand what they are saying. The defining [characteristic of logarithm functions](https://arbital.com/p/47k) is that they are [https://arbital.com/p/-real_valued](https://arbital.com/p/-real_valued) functions $f$ such that

__Property 1: Multiplying inputs adds outputs.__
$$
f(x \cdot y) = f(x) + f(y) \tag{1}
$$
for all $x, y \in$ [$\mathbb R^+$](https://arbital.com/p/positive_reals). This says that whenever the input grows (or shrinks) by a factor of $y$, the output goes up (or down) by only a fixed amount, which depends on $y$. In fact, equation (1) alone tells us quite a bit about the behavior of $f,$ and from it, we can _almost_ guarantee that $f$ is a logarithm function. First, let's see how far we can get using equation (1) all by itself:

---

__Property 2: 1 is mapped to 0.__
$$
f(1) = 0. \tag{2}
$$

This says that the amount the output changes if the input grows by a factor of 1 is zero — i.e., the output does not change if the input changes by a factor of 1. This is obvious, as "the input changed by a factor of 1" means "the input did not change."

__Exercise:__ Prove (2) from (1).
%%hidden(Proof):
$$f(x) = f(x \cdot 1) = f(x) + f(1),\text{ so }f(1) = 0.$$
%%

---

__Property 3: Reciprocating the input negates the output.__
$$
f(x) = -f\left(\frac{1}{x}\right). \tag{3}
$$

This says that the way that growing the input by a factor of $x$ changes the output is exactly the opposite from the way that shrinking the input by a factor of $x$ changes the output. In terms of the "communication cost" interpretation, if doubling (or tripling, or $n$-times-ing) the possibility space increases costs by $c$, then halving (or thirding, or $n$-parts-ing) the space decreases costs by $c.$

__Exercise:__ Prove (3) from (2) and (1).
%%hidden(Proof):
$$x \cdot \frac{1}{x} = 1,\text{ so }f(1) = f\left(x \cdot \frac{1}{x}\right) = f(x) + f\left(\frac{1}{x}\right).$$

$$f(1)=0,\text{ so }f(x)\text{ and }f\left(\frac{1}{x}\right)\text{ must be opposites.}$$
%%

---

__Property 4: Dividing inputs subtracts outputs.__

$$
f\left(\frac{x}{y}\right) = f(x) - f(y). \tag{4}
$$

This follows immediately from (1) and (3).

__Exercise:__ Give an interpretation of (4).
%%hidden(Interpretation):
There are at least two good interpretations:

1. $f\left(x \cdot \frac{1}{y}\right) = f(x) - f(y),$ i.e., shrinking the input by a factor of $y$ is the opposite of growing the input by a factor of $y.$
2. $f\left(z \cdot \frac{x}{y}\right) = f(z) + f(x) - f(y),$ i.e., growing the input by a factor of $\frac{x}{y}$ affects the output just like growing the input by $x$ and then shrinking it by $y.$

Try translating these into the communication cost interpretation if it is not clear why they're true.
%%

---

__Property 5: Exponentiating the input multiplies the output.__

$$
f\left(x^n\right) = n \cdot f(x). \tag{5}
$$

This says that multiplying the input by $x$, $n$ times incurs $n$ identical changes to the output. In terms of the communication cost metaphor, this is saying that you can emulate an $x^n$ digit using $n$ different $x$-digits.

__Exercise:__ Prove (5).
%%hidden(Proof):
This is easy to prove when $n \in \mathbb N:$
$$f\left(x^n\right) = f(\underbrace{x \cdot x \cdot \ldots x}_{n\text{ times}}) = \underbrace{f(x) + f(x) + \ldots f(x)}_{n\text{ times}} = n \cdot f(x).$$

For $n \in \mathbb Q,$ this is a bit more difficult; we leave it as an exercise to the reader. Hint: Use the proof of (6) below, for $n \in \mathbb N,$ to bootstrap up to the case where $n \in \mathbb Q.$

For $n \in \mathbb R,$ this is actually _not_ provable from (1) alone; we need an additional assumption (such as [https://arbital.com/p/-continuity](https://arbital.com/p/-continuity)) on $f$.
%%

Property 5 is actually _false,_ in full generality — it's possible to create a function $f$ that obeys (1), and obeys (5) for $n \in \mathbb Q,$ but which exhibits pathological behavior on irrational numbers. For more on this, see [pathological near-logarithms](https://arbital.com/p/).

This is the first place that property (1) fails us: 5 is true for $n \in \mathbb Q,$ but if we want to guarantee that it's true for $n \in \mathbb R,$ we need $f$ to be [continuous](https://arbital.com/p/continuity), i.e. we need to ensure that if $f$ follows 5 on the rationals it's not allowed to do anything insane on irrational numbers only.

---

__Property 6: Rooting the input divides the output.__

$$
f(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}) = \frac{f(x)}{n}. \tag{6}
$$

This says that, to change the output one $n$th as much as you would if you multiplied the input by $x$, multiply the input by the $n$th root of $x$. See [https://arbital.com/p/44l](https://arbital.com/p/44l) for a physical interpretation of this fact.

__Exercise:__ Prove (6).
%%hidden(Proof):
$$(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x})^n = x,\text{ so }f\left((\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x})^n\right)\text{ has to equal }f(x).$$

$$f\left((\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x})^n\right) = n \cdot f(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}),\text{ so }f(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}) = \frac{f(x)}{n}.$$
%%

As with (5), (6) is always true if $n \in \mathbb Q,$ but not necessarily always true if $n \in \mathbb R.$ To prove (6) in full generality, we additionally require that $f$ be continuous.

---

__Property 7: The function is either trivial, or sends some input to 1.__

$$
\text{Either $f$ sends all inputs to $0$, or there exists a $b \neq 1$ such that $f(b)=1.$}\tag{7}
$$

This says that either $f$ is very boring (and does nothing regardless of its inputs), or there is some particular factor $b$ such that when the input changes by a factor of $b$, the output changes by exactly $1$. In the communication cost interpretation, this says that if you're measuring communication costs, you've got to pick some unit (such as $b$-digits) with which to measure.

__Exercise:__ Prove (7).
%%%hidden(Proof):
Suppose $f$ does not send all inputs to $0$, and let $x$ be an input that $f$ sends to some $y \neq 0.$ Then $f(\sqrt[https://arbital.com/p/y](https://arbital.com/p/y){x}) = \frac{f(x)}{y} = 1.$%%note: You may be wondering, "what if $y$ is negative, or a fraction?" If so, see [Strange roots](https://arbital.com/p/). Short version: $\sqrt[https://arbital.com/p/-3/4](https://arbital.com/p/-3/4){x}$ is perfectly well-defined.%%

$b$ is $\sqrt[https://arbital.com/p/y](https://arbital.com/p/y){x}.$ We know that $b \neq 1$ because $f(b) = 1$ whereas, by (2), $f(1) = 0$.
%%%

---

__Property 8: If the function is continuous, it is either trivial or a logarithm.__

$$
\text{If $f(b)=1$ then } f(b^x) = x. \tag{8}
$$

This property follows immediately from (5). Thus, (8) is always true if $x$ is a [rational](https://arbital.com/p/4zq), and if $f$ is continuous then it's also true when $x$ is irrational.

Property (8) states that if $f$ is non-trivial, then it inverts [exponentials](https://arbital.com/p/4ts) with base $b.$ In other words, $f$ counts the number of $b$-factors in $x$. In other words, $f$ counts how many times you need to multiply $1$ by $b$ to get $x$. In other words, $f = \log_b$!

---

Many texts take (8) to be the defining characteristic of the logarithm. As we just demonstrated, one can also define logarithms by (1) as [continuous](https://arbital.com/p/continuity) [non-trivial](https://arbital.com/p/trivial_mathematics) functions whose outputs grow by a constant (that depends on $y$) whenever their inputs grow by a factor of $y$. All other properties of the logarithm follow from that.

If you want to remove the "continuous" qualifier, you're still fine as long as you stick to [rational](https://arbital.com/p/4zq) inputs. If you want to remove the "non-trivial" qualifier, you can interpret the function $z$ that sends everything to zero as [$\log_\infty$](https://arbital.com/p/4c8). Allowing $\log_\infty$ and restricting ourselves to rational inputs, _every_ function $f$ that satisfies equation (1) is [isomorphic](https://arbital.com/p/4f4) to a logarithm function.

In other words, if you find a function whose output changes by a constant (that depends on $y$)  whenever its input grows by a factor of $y$, there is basically only one way it can behave. Furthermore, that function only has one degree of freedom — the choice of $b$ such that $f(b)=1.$ As we will see next, even that degree of freedom is rather paltry: All logarithm functions behave in essentially the same way. As such, if we find any $f$ such that $f(x \cdot y) = f(x) + f(y)$ (or any physical process well-modeled by such an $f$), then we immediately know quite a bit about how $f$ behaves.

Summary:
- $\log_b(x \cdot y) = \log_b(x) + \log_b(y)$ for any $b$, this is the defining characteristic of logarithms.
- $\log_b(1) = 0,$ because $\log_b(1) = \log_b(1 \cdot 1) = \log_b(1) + \log_b(1).$
- $\log_b\left(\frac{1}{x}\right) = -\log_b(x),$ because $\log_b(1) = \log_b\left(x \cdot \frac{1}{x}\right) = \log_b(x) + \log_b\left(\frac{1}{x}\right) = 0.$
- $\log_b\left(\frac{x}{y}\right) = \log_b(x) - \log_b(y),$ which follows immediately from the above.
- $\log_b\left(x^n\right) = n \cdot \log_b(x),$ because $x^n$ = $\underbrace{x \cdot x \cdot \ldots x}_{n\text{ times}}$.
- $\log_b\left(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}\right) = \frac{\log_b(x)}{n},$ because $\log_b(x) = \log_b\left((\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x})^n\right) = n \cdot \log_b(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}).$
- For every $f$ that satisfies $f(x \cdot y) = f(x) + f(y)$ for all $x, y \in \mathbb R^+,$ either $f$ sends every input to 0 or there exists some $b$ such that $f(b) = 1,$ in which case we call $f$ $\log_b.$ Thus, $\log_b(b) = 1.$
- $\log_b(b^n) = n,$ because $\log_b(x^n) = n \log_b(x)$ and $\log_b(b) = 1.$