Primer on Infinite Series

What does it mean to add things together forever? While many get introduced to infinite series in Calculus class, it seems that everyone skips ahead to memorizing terminology without first motivating the subject, or considering that it's sorta weird. In reality, the long history of infinite series would suggest that the subject is subtle and strange. But by steaming ahead, memorizing the vocabulary, examples, and rules for convergence, learners miss a chance to really think about what these infinite sums really are or could be. They end up being able to more or less accurately parrot those rules, but if you speak with them a bit or overhear a conversation with others about the topic, you realize their understanding is a hollow shell. This article is a look at infinite series, adding things together forever, from more or less first principles.

Example One: Zeno Crosses the Room
----------------------------------

One of the origin stories for infinite series is attributed to the Greek Philosopher Zeno. It's a good place to start motivating why you end up dealing with infinite series and also a place from which you can begin to appreciate the need for a way of thinking about infinity that doesn't lead to pure nonsense. Although he had many paradoxes, they all boil down to certain weirdnesses that come up when you imagine space or time being infinitely divisible. Things that are obvious and simple from the point of view of everyday experience are contradicted by their analysis from the point of view of infinite divisibility. Consider the following scenario.

You want to cross the room.

That is you wish to cover one room's worth of distance. Easy enough right? You don't even need to stretch. But...(Zeno's voice) to cross the room, first you need to get halfway there. Before you cross the 2nd half of the room, you need to cross half of the 2nd half of the room. And so forth. At each stage you subdivide the remaining distance in half and cover that distance.

Zeno's problem was that this sequence of instants is infinite. If each turn took say 1 second, it would take infinity seconds to cross the room. From this perspective any motion—not just room crossing—appears impossible. Zeno's conclusion was that bringing infinity into the discussion, in this case dividing a given distance into infinitely many pieces, leads to nonsense and should be avoided.

%note:For this and other similar reasons, the official use of infinity in mathematical arguments was postponed until the innovation of Calculus. Essentially, due to the simultaneous creation of mechanics and all the practical uses to which it could be put, the problems with reasoning with infinite sums were swept under the rug for a while. Calculus uses infinity all the time and doesn't apologize for it. Newton, Leibniz, etc. didn't figure out how to solve Zeno's paradox, they just ignored it. It wasn't until a couple centuries later that mathematicians really confronted the problem.%

Zeno's room crossing can be written using numbers and symbols instead of sentences, where it might look a bit more familiar as an "infinite sum".

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32}+\ldots$$

And we can think of its sum either from the statement of the problem (where it is clear that the sum should be 1, a whole room) or by a direct calculation of the distance remaining to cover and noting where its limit is.

<colgroup>
<col style="text-align:left;"/>
<col style="text-align:left;"/>
<col style="text-align:left;"/>
</colgroup>

	Step
	Distance Covered
	Distance Remaining

	0
	0
	1

	1
	\(\frac{1}{2}\)
	\(\frac{1}{2}\)

	2
	\(\frac{1}{2}+\frac{1}{4}\)
	\(\frac{3}{4}\)

	3
	\(\frac{1}{2}+\frac{1}{4}+\frac{1}{8}\)
	\(\frac{7}{8}\)

	4
	\(\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}\)
	\(\frac{15}{16}\)

	5
	\(\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32}\)
	\(\frac{31}{32}\)

So, at this point, we have a good reason to want to be able to write

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32}+\ldots = 1$$

Metaphysical or existential doubts notwithstanding, it just looks like we should end up with one if this process of making up half the remaining distance were to continue. In this and many other situations, its clear that adding together infinitely many things should work out to give something finite. The way we would read these symbols in words is

>If you keep adding halves of halves ($\frac{1}{2}$ + \frac{1{4}$ and so on), you get closer and closer to 1. If somehow you could add forever, you'd have exactly one.

Exact and Approximate
---------------------

At this point we should clear up one thing about *exact* versus *approximate*.  You might be thinking that, regardless of the "infinite" part of the sum, this is a clear case of "good enough for all practical purposes". At some point you get close enough to touch the wall at the end of the room. So yes, we can think of the sum as approximating one. And these infinite sums were originally very useful because they gave ways of approximating difficult to calculate things with very easy to calculate things. But there is some subtlety here in the words we use. When we say,

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32}+\ldots = 1$$

We do not mean

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32}+\ldots \approx \text{ (is approximately equal to) }1$$

It *would* be correct to say

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8} \approx 1$$

or also

$$\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}+\frac{1}{32} \approx 1$$

but once we have infinitely many terms, once we are asking what would happen if the process were allowed to continue indefinitely, we are no longer taking about approximately, we are talking about exactly equal.

And this is where people, not just young students but the many mathematical thinkers through history, have had trouble. We are used to the things related by equal signs both being simple numbers. Finite things. Not infinite processes. So what does it mean for an infinite process to be *exactly equal* to a finite number? So far in this discussion we're not yet sure. But we have a clue. We have exactly one case where we have an infinite process that has an obvious limit. We'd like to equate them.

Next, let's look at a case where the infinite process, instead of telling us something we already know, helps us see something new.

Example 2: Archie and the Area of a Circle
------------------------------------------

You may have been asked to remember formulas for the circumference and area of a circle in school.
$$ \text{Circumference }(C) = \pi \cdot \text{Diameter }(D) = 2\pi \cdot \text{Radius }(r)$$

$$ \text{Area of a Circle}(A) = \pi \cdot r^2$$

But did you ever wonder why these formulas are true or why they both happen to have $\pi$ in them? Did you ever wonder how we figured out the area of a round shape like the circle in the first place?

Well, Archimedes did. And the way he figured it out was by creating an infinite sum.

The formula for circumference is easy. It's just a definition. Ancient Greeks and many others noticed that $\frac{C}{D}$ was the same ratio for any circle (*WHY?*). They didn't call this $\pi$ or even consider it a number, but we do. By definition, $\frac{C}{D} = \pi$, so of course $C=\pi D$.

The real mystery is with the area formula. And this is what Archimedes did. He imagined a regular polygon inscribed within a circle. Here's a picture.

![](some pic)

Now, in my picture, there are 8 sides. But there's nothing special about 8. It could be 6, or 12, or a million. But...the more sides there are, the more the polygon looks like a circle, the better approximation its area is for the area of the circle. And, as it turns out, the area of the polygon is easy to compute. Moreover, computing the area of the pylon, we can see how it changes with the number of sides and how it stays the same, *and* we can see what happens to this area if we somehow had infinitely many sides. Let's look:

First, we notice that a polygon can be split into a bunch of triangles, each of which is the same. How many? The same number as we have sides of the polygon. Just like slices of pie. Add up the area of all the slices to get the area of the original polygon.

$$\text{Area of polygon with }n \text{ sides} = \text{sum of } n \text{ triangles}$$

Of course, each triangle has the same area, and any triangle has area $= \frac{1}{2} \cdot \text{base}\cdot \text{height}$. Here, the base is the side of a polygon (let's call it $s$) and the height we can call $h$. Our formula for the area of a polygon becomes:

$$\text{Area of polygon with }n \text{ sides} = \underbrace{\frac{1}{2}sh+\frac{1}{2}sh+\ldots+\frac{1}{2}sh}_{n \text{ times}}$$

Now, factor the $\frac{1}{2}h$ out of the above expression to get something even nicer.

$$\text{Area of polygon with }n \text{ sides} = \frac{1}{2}h(\underbrace{s+s+\ldots+s}_{n \text{ times}})$$

And look, $n$ sides added together is nothing other than the perimeter ($P$) of the polygon.

$$\text{Area of polygon} = \frac{1}{2}hP$$

This formula is true regardless of how many sides we have. All we have to do now to recover the formula for the area of the circle is to imagine what happens to all of the parts when the number of sides, the extent of the approximation, the number of terms in the underlying sum, gets larger.

$$ \text{Area of polygon} \rightarrow \text{Area of a Circle}(A)$$

$$h \rightarrow r$$

$$P \rightarrow C$$

So our formula becomes

$$ A = \frac{1}{2}rC = \frac{1}{2}r(\pi D) = \frac{1}{2}r(\pi(2r))=\pi r^2$$

Amazing, no?

There are *tons* of other situations where you can answer a question by putting together infinitely many pieces, situations where we would like to be able to formally write down something of the sort:

$$\text{infinite sum} = \text{finite number}$$

but there are also obvious and not so obvious situations where doing so is nonsense. We would like to go so far as to extend our rules for algebra to be able to operate on these infinite sums (things like multiplying both sides of an equation by the same number or grouping terms), but the truth is we don't have enough information to know when that does and doesn't work out. Next we will take a look at this more general question.


Lambda calculus

What's the least we can put in a programming language and still have it be able to do anything other [languages can](https://arbital.com/p/computability) %%note:i.e. [https://arbital.com/p/turing_complete](https://arbital.com/p/turing_complete)%%? It turns out we can get away with including only a single built in feature: [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) definition, written using $\lambda$. The language that has only this one feature is called "lambda calculus" or "$\lambda$ calculus." We also need a way to apply our functions to inputs; we write $f\ x$ to indicate the value of the function $f$ on $x$, perhaps more familiarly written $f(x)$.

A $\lambda$ expression is something of the form $\lambda x.f(x)$, which means "take in an input $x$, and return $f(x)$. For example, $\lambda x.x+1$ takes in a number and returns one more than it. We only directly have functions with one input, but we can build functions with multiple inputs out of this. For example, applying $\lambda x.\lambda y.x+y$ to $3$, and applying the result to $4$ gives $3+4=7$ . We can think of this as a two-input function, and abbreviate it as $\lambda xy.x+y$; "$\lambda xy$" is simply a shorthand for "$\lambda x.\lambda y$." We don't lose any power by restricting ourselves to unary functions, because we can think of functions with multiple inputs as taking only one input at a time, and returning another function at each intermediate step.

We said the only feature we have is function definition, so what's this about "$+$" and "$1$"? These symbols don't exist in $\lambda$ calculus, so we'll have to define [numbers](https://arbital.com/p/-54y) and operations on them using $\lambda$ %%note:The standard way of doing this is called the [https://arbital.com/p/6qb](https://arbital.com/p/6qb).%%. But $\lambda$ expressions are useful even when we have other features as well; programming languages such as Python allow function definition using $\lambda$. It's also useful when trying to understand what $\lambda$ means to see examples with functions you're more familiar with.

We're not going to put constraints on what inputs $\lambda$ expressions can take. If we allowed expressions like $\lambda x.x+1$, it would get confused when we try to give it something other than a number; this is why we can't have $+$ and $1$. So what sorts of objects will we give our functions? The only objects have we: other functions, of course!

[https://arbital.com/p/toc:](https://arbital.com/p/toc:)

#Formal definition

Let's formally define $\lambda$ calculus. First, the language of $\lambda$ calculus contains

- Parentheses ( and )
- Symbols $\lambda$ and .
- Variables $v_1,v_2,\dots$ (in practice we'll use other letters too).

What counts as a $\lambda$ expression? We define them recursively:

- For any variable $x$, $x$ is a $\lambda$ expression.
- For any variable $x$ and $\lambda$ expression $M$, $(\lambda x.M)$ is a $\lambda$ expression.
- If $M$ and $N$ are $\lambda$ expressions, so is $(M\ N)$.

We also want to define free and bound variables, and also do this recursively:

- The variable $x$ is free in the expression $x$.
- Every instance of $x$ is bound in the expression $(\lambda x.M)$. If an instance of $x$ is free in $M$, we say the initial $\lambda x$ in $(\lambda x.M)$ *binds* it. We say that $M$ is the *scope* of that $\lambda x$.
- An instance of $x$ in $(M\ N)$ is free if it is free in $M$ or $N$, and bound otherwise.

We can use the rules to build expressions like $((\lambda x.(\lambda y.(x\ y))))\ x)$, in which the first instance of $x$ and the only instance of $y$ are bound, and the second instance of $x$ is free. We have an oppressive amount of parentheses; we'll establish informal conventions to reduce them in the next section.

#Parenthesis conventions

If we want to be extremely careful, we should write the add-two-numbers function as $(\lambda x.(\lambda y.(x+y)))$, using parentheses as in the formal definition. But we often don't need them, and use the following rules:

- We don't need outermost parentheses. We can write $\lambda x.(\lambda y.(x+y))$ instead of $(\lambda x.(\lambda y.(x+y)))$.
- Application is, by default, left associative. $f\ x\ y$ means "the function $f$ with inputs $x$ and $y$," i.e. $(f\ x)\ y$, not $f\ (x\ y)$.
- The scope of any $\lambda$ is as large is possible. $\lambda x.\lambda y.x+y$ means $\lambda x.(\lambda y.(x+y))$, not $(\lambda x.\lambda y.x)+y$ or $\lambda x.(\lambda y.x)+y$ or anything like that.
- We can abbreviate sequences of $\lambda$s. As mentioned before, we can write $\lambda xy.x+y$ for $\lambda x.\lambda y.x+y$.

Of course, we don't have to omit parentheses whenever we can. These rules govern what it means when we don't have all the parentheses strictly necessary, but we can include them if we want to be extra clear.

#Reducing lambda expressions

So far, $\lambda$ expressions don't have any meaning; we've said what we're allowed to write down but not how the relate to each other. What we usually do with a $\lambda$ expression is reduce it, and there are three rules that we can use: [$\beta$-reduction](https://arbital.com/p/beta_reduction), [$\alpha$-conversion](https://arbital.com/p/alpha_conversion), and [$\eta$-conversion](https://arbital.com/p/eta_conversion).

[$\beta$-reduction](https://arbital.com/p/beta_reduction) is the fancy name for "stick in the inputs to a function." For example, to evaluate $(\lambda x.\lambda y.x+y)\ 6\ 3$, we first notice that $6$ is the input to a $\lambda$ expression that starts $\lambda x$, and substitute $6$ for $x$ in the expression, giving us $(\lambda y.6+y)\ 3$. Now $3$ is substituted for $y$, giving us $6+3=9$.

Formally, for a variable $x$ and $\lambda$ expressions $M$ and $N$, $\beta$-reduction converts $((\lambda x.M)\ N)$ to $M[https://arbital.com/p/N/x](https://arbital.com/p/N/x)$, i.e. $M$ with every free instance of $x$ replaced with $N$. The expressions $((\lambda x.M)\ N)$ and $M[https://arbital.com/p/N/x](https://arbital.com/p/N/x)$ are equivalent. %%todo: is there a more standard notation for this?%%

[$\alpha$-conversion](https://arbital.com/p/alpha_conversion) says we can rename variables; $\lambda x.f\ x$ is the same as $\lambda y.f\ y$. Formally, if $M$ is a $\lambda$ expression containing $\lambda x$, and no instance of $x$ bound by that $\lambda x$ is within the scope of a $\lambda y$, then we can replace every $x$ in the scope of the $\lambda x$ with a $y$ to get an equivalent expression. We need the second criterion because otherwise we could replace the $x$s in $\lambda x.\lambda y.x$ with $y$s to get $\lambda y.\lambda y.y$, and then replace the outer $y$ with $x$ to get $\lambda x.\lambda y.x$, which is not equivalent (the second substitution here is valid; the first one is the problem).

Finally, [$\eta$-conversion](https://arbital.com/p/eta_conversion) says that if two expressions give the same result on any input, then they're equivalent. We should have $\lambda x.f\ x$ and $f$ be the same, since $\beta$-reduction converts $(\lambda x.f\ x)\ x$ to $f\ x$, so they're the same on any input. Formally, if $x$ isn't free in $M$, then $(\lambda x.(M\ x))$ is equivalent to $M$.

#[https://arbital.com/p/50p](https://arbital.com/p/50p)

Let's look again at the function that adds two numbers, $\lambda x.\lambda y.x+y$. There are two different ways we can think of this: first, as a function that takes in two numbers, and returns their sum. If we're talking about [natural numbers](https://arbital.com/p/45h), this is a function $\mathbb N^2\to\mathbb N$.

We could also think of the function as taking one input at a time. It says "take in input $x$, return the expression $\lambda y.x+y$." For example, on input $6$ we have $(\lambda x.\lambda y.x+y)\ 6=\lambda y.6+y$. Now we have a function that takes in a number, and gives us a function that takes in a number and returns a number. We can write this as $\mathbb N\to(\mathbb N\to\mathbb N)$.

This equivalence between functions on two (or more) inputs, and functions on one input that return a new function, is known as [https://arbital.com/p/50p](https://arbital.com/p/50p) and is important in $\lambda$ calculus, where the only objects are functions. We have to get used to the idea that a function can take other functions as arguments and give functions as return values.

#Defining variables

$\lambda$ calculus doesn't have a built in way to define a [https://arbital.com/p/-variable](https://arbital.com/p/-variable). Suppose we have the function $d=\lambda f.\lambda x.f\ (f\ x)$ which applies its first input to its second input twice. Currying, $d$ takes in a function and returns that function composed with itself. Say we want to know what $d\ d$ is. We could just write $d$ twice, to get $(\lambda f.\lambda x.f\ (f\ x))\ (\lambda f.\lambda x.f\ (f\ x))$. $\beta$-reduction eventually reduces this to $\lambda f.\lambda x.f\ (f\ (f\ (f\ x)))$, the function that applies its first input to its second input four times (as you might have expected, applying a function twice, twice, is the same as applying it four times).

We can also get $d\ d$ while only writing out $d$ once. Take the $\lambda$ expression $(\lambda x.x\ x)\ d$ or, writing out $d$, $(\lambda x.x\ x)\ (\lambda f.\lambda x.f\ (f\ x))$. A single step of $\beta$-reduction turns this into the previous version, where we wrote $d$ out twice. The expression we started with was shorter, and expressed better the idea that we wanted to use the same value of $d$ twice. In general, if we want to say something like $x = a; f\ x$ (i.e. define the variable $x$, and then use it in some expression), we can write $(\lambda x.f\ x)\ a$, first building the expression we want to use the value of $x$ in, and then substituting in the value using function application. This will be very useful in the next section when we make [recursive](https://arbital.com/p/recursion) functions.

#Loops

It might seem like $\lambda$ expressions are very finite, and $\beta$-reduction will always finish after a reasonable amount of time. But for $\lambda$ calculus to be as powerful as normal programming languages, there must be $\lambda$ expressions that don't reduce in a [finite amount of time](https://arbital.com/p/46h). The way to resolve this is to figure out how to build while loops in $\lambda$ calculus; if you can make a while loop, you can write a program that runs forever pretty easily.

Since all we have is functions, we'll use [https://arbital.com/p/-recursion](https://arbital.com/p/-recursion) to make loopy programs. For example, consider this factorial function in Python:

    factorial = lambda n: 1 if n==0 else n*factorial(n-1)

How can we write this in lambda calculus? Let's assume we have [numbers](https://arbital.com/p/54y), [https://arbital.com/p/-arithmetic](https://arbital.com/p/-arithmetic), [booleans](https://arbital.com/p/57f), %%todo: possibly change these links to church encoding pages once they exist%%
and a function $Z$ that returns True on $0$ and False on anything else. Let's also a assume we have an "if" function $I$ that takes in three inputs: $I = (\lambda p.\lambda x.\lambda y.$if $p$, then $x$, else $y)$. So $I\ True\ x\ y=x$ and $I\ False\ x\ y=y$.

Starting to translate from Python to $\lambda$ calculus, we have $F = \lambda n.I\ (Z\ n)\ 1\ (n\times(F\ (n-1)))$, which says "if $n$ is $0$, then $1$, else $n\times F(n-1)$. But we can't define $F$ like this; we have to use the method described in the previous section. In order for each recursive call of the factorial function to know how to call further recursive calls, we need to pass in $F$ to itself. Consider the expression

$$F=(\lambda x.x\ x)\ (\lambda r.\lambda n.I\ (Z\ n)\ 1\ (n\times(r\ r\ (n-1))))$$

Let $g=\lambda r.\lambda n.I\ (Z\ n)\ 1\ (n\times(r\ r\ (n-1)))$. Then $F=(\lambda x.x\ x)\ g=g\ g$ is $g$ applied to itself. Plugging in $g$ for $r$, $g\ g=\lambda n.I\ (Z\ n)\ 1\ (n\times(g\ g\ (n-1)))$. But since $g\ g=F$, this is just $\lambda n.I\ (Z\ n)\ 1\ (n\times(F\ (n-1)))$, exactly what we wanted $F$ to be.

Let's generalize this. Say we want to define a recursive formula $f=\lambda x.h\ f\ x$, where $h$ is any expression (in the factorial example $h=\lambda f.\lambda n.I\ (Z\ n)\ 1\ (n\times(f\ (n-1)))$). We define

\begin{align*}
    g&=\lambda r.h\ (r\ r)\newline
    &=\lambda r.\lambda x.h\ (r\ r)\ x
\end{align*}

and use this to build

\begin{align*}
    f&=(\lambda x.x\ x)\ g\newline
    &=g\ g\newline
    &=(\lambda r.\lambda x.h\ (r\ r)\ x)\ g\newline
    &=\lambda x.h\ (g\ g)\ x\newline
    &=\lambda x.h\ f\ x.
\end{align*}

We can express this process taking any $h$ to the recursive function $f$ as a $\lambda$ expression:

\begin{align*}
    Y&=\lambda h.(\lambda x.x\ x)\ (\lambda r.h\ (r\ r))\newline
    &=\lambda h.(\lambda r.h\ (r\ r))\ (\lambda r.h\ (r\ r)).
\end{align*}

For any $h$, which takes as arguments $f$ and $x$ and returns some expression using them, $Y\ h$ recursively calls $h$. Specifically, letting $f=Y\ h$, we have $f=\lambda x.h\ f\ x$.

For fun, let's make a $\lambda$ expression that enters an infinite loop by reducing to itself. If we make $h$ the function that applies $f$ to $x$, recursively calling it should accomplish this, since it's analogous to the Python function `f = lambda x: f(x)`, which just loops forever. So let $h=\lambda f.\lambda x.f x=\lambda f.f$. Now \begin{align*}
    Y\ h&=(\lambda x.x\ x)\ (\lambda r.(\lambda f.f)\ (r\ r))\newline
    &=(\lambda x.x\ x)\ (\lambda r.r\ r)\newline
    &=(\lambda r.r\ r)\ (\lambda r.r\ r).
\end{align*}

Continued $\beta$-reduction doesn't get us anywhere. This expression is also interesting because it's a simple [https://arbital.com/p/-322](https://arbital.com/p/-322), a program that outputs its own code, and this is a sort of template you can use to make quines in other languages.

This strategy of building recursive functions results in some very long intermediate computations. If $F$ is the factorial function, we compute $F(100)$ as $100\times F(99)=100\times 99\times F(98)$, etc. By the time the recursive calls finally stop, we have the very long expression $100\times99\times\dots\times2\times1$, which takes a lot of space to write down or store in a computer. Many programming languages will give you an error if you try to do this, complaining that you exceeded the max recursion depth.

The fix is something called [https://arbital.com/p/-tail_recursion](https://arbital.com/p/-tail_recursion), which roughly says "return only a function evaluation, so you don't need to remember to do something with it when it finishes." The general way do accomplish this is to pass along the intermediate computation to the recursive calls, instead of applying it to the result of the recursive calls. We can make a tail recursive version of our Python factorial like so %%note:Python will still give you a "maximum recursion depth exceeded" error with this, but the strategy works in lambda calculus and lazy languages like Lisp and Haskell, and Python is easier to demonstrate with.%%:

    f = lambda n, k: k if n==0 else f(n-1, k*n)

We pass along $k$ as an intermediate value. It has to start with $k=1$; we can do this by defining a new function `g = lambda n: f(n,1)` or using Python's default value feature to have $k$ be $1$ be default. Now if we want $100$ factorial, we get $f(100,1)=f(99,1*100)=f(99,100)=f(98,9900)=\dots$. We compute the products as we go instead of all at the end, and only ever have one call of $f$ active at a time.

Writing a tail recursive version of factorial in $\lambda$ calculus is left as an exercise for the reader.

#[https://arbital.com/p/Y_combinator](https://arbital.com/p/Y_combinator)

You might be wondering why we called the recursive-function-maker $Y$, so let's look at that again. We said earlier that $Y\ h=\lambda x.h\ (Y\ h)\ x$. By $\eta$-conversion, $Y\ h=h\ (Y\ h)$. That means $Y\ h$ is a fixed point of $h$! For any function $h$, $Y$ finds a value $f$ such that $h\ f=f$. This kind of object is called a [https://arbital.com/p/-fixed_point_combinator](https://arbital.com/p/-fixed_point_combinator); this particular one is called the [https://arbital.com/p/Y_combinator](https://arbital.com/p/Y_combinator), conventionally denoted $Y$.

Why is a fixed point finder the right way to make recursive functions? Consider again the example of factorial. If a function $f$ says "do factorial for $k$ recursive steps," then $h\ f$ says "do factorial for $k+1$ recursive steps;" $h$ was designed to be the function which, calling recursively, gives factorial. Naming the [identity function](https://arbital.com/p/identity_map) $i=\lambda n.n$, $i$ does factorial for $0$ steps, so $h\ i$ does it for $1$ step, $h\ (h\ i)$ for $2$, and so on. To completely compute factorial, we need a function $F$ that says "do factorial for infinity recursive steps." But doing one more than infinity is the same as just doing infinity, so it must be true that $h\ F=F$; i.e. that $F$ is a fixed point of $h$. There may be other fixed points of $h$, but the Y combinator finds the fixed point that corresponds to recursively calling it.

## With $Y$ and without $Y$

What do we get, and what do we lose, if we allow the use of $Y$?

$Y$ takes a lambda-term which may have corresponded to a [https://arbital.com/p/-primitive_recursive](https://arbital.com/p/-primitive_recursive) function, and outputs a lambda term which need not correspond to a primitive recursive function.
That is because it allows us to perform an *unbounded* search: it expresses fully-general looping strategies.
(As discussed earlier, without a fixed-point finder of some kind, we can only create lambda-terms corresponding to programs which provably halt.)

On the other side of the coin, $Y$ may produce non-[well-typed](https://arbital.com/p/well_typed) terms, while terms without a fixed-point finder are guaranteed to be well-typed.
That means it's hard or even impossible to perform type-checking (one of the most basic forms of static analysis) on terms which invoke $Y$.

Strongly relatedly, in a rather specific sense through the [Curry-Howard correspondence](https://arbital.com/p/curry_howard), the use of $Y$ corresponds to accepting non-[constructiveness](https://arbital.com/p/constructive_proof) in one's proofs.
Without a fixed-point finder, we can always build a constructive proof corresponding to a given lambda-term; but with $Y$ involved, we can no longer keep things constructive.


Arbital Labs

This is a temporary group created as an experiment in both software and community design. It's meant to act as a seed for multiple future Arbital communities. Currently membership is by invite only.


Monotone function: exercises

Try these exercises and become a *deity* of monotonicity.

Monotone composition
-----

Let $P, Q$, and $R$ be posets and let $f : P \to Q$ and $g : Q \to R$ be monotone
functions. Prove that their composition $g \circ f$ is a monotone function from $P$ to $R$.

Evil twin
-------

Let $P$ and $Q$ be posets. A function $f : P \to Q$ is called **antitone** if it *reverses* order: that is, $f$ is antitone whenever $p_1 \leq_P p_2$ implies  $f(p_1) \geq_Q f(p_2)$. Prove that the composition of two antitone functions is monotone.

Partial monotonicity
----------

A two argument function $f : P \times A \to Q$ is called  *partially monotone in the 1st argument* whenever $P$ and $Q$ are posets and for all $a \in A$, $p_1 \leq_P p_2$ implies $f(a, p_1) \leq_Q f(a, p_2)$. Likewise a 2-argument function $f : A \times P \to Q$ is called *partially monotone in the second argument* whenever $P$ and $Q$ are posets and for all $a \in A$, $p_1 \leq_P p_2$ implies $f(p_1, a) \leq_Q f(p_2, a)$.

Let $P, Q, R$, and $S$ be posets, and let $f : P \times Q \to R$ be a function that is partially monotone in both of its arguments. Furthermore, let $g_1 : S \to P$ and $g_2 : S \to Q$ be monotone functions.

Prove that the function $h : S \to R$ defined as $h(s) \doteq f(g_1(s), g_2(s))$ is monotone.

Brain storm
------

List all of the commonly used two argument functions you can think of that are partially monotone in both arguments. Also, list all of the commonly used two argument functions you can think of that are partially monotone in one argument and partially antitone in the other.


Stub

Stub-Class pages are tiny, often a single paragraph, or a couple of sentences.  Stubs may have been created to provide a [https://arbital.com/p/-5xs](https://arbital.com/p/-5xs) for parenting, tagging, or reference purposes in a section under construction.  An editor expanding a Stub page might start over from scratch.

Contrast:

- [https://arbital.com/p/3rk](https://arbital.com/p/3rk), which is a larger page, but still visibly incomplete or disorganized.
- [https://arbital.com/p/4gs](https://arbital.com/p/4gs), for brief pages which only contain a formal definition and don't try to explain the topic.

**Also** use the tag [https://arbital.com/p/4v](https://arbital.com/p/4v) if the page is being actively edited.

**[Quality scale](https://arbital.com/p/4yg)**

* [https://arbital.com/p/4ym](https://arbital.com/p/4ym)
* [https://arbital.com/p/4gs](https://arbital.com/p/4gs)
* [https://arbital.com/p/5xq](https://arbital.com/p/5xq)
* [https://arbital.com/p/72](https://arbital.com/p/72)
* [https://arbital.com/p/3rk](https://arbital.com/p/3rk)
* [https://arbital.com/p/4y7](https://arbital.com/p/4y7)
* [https://arbital.com/p/4yd](https://arbital.com/p/4yd)
* [https://arbital.com/p/4yf](https://arbital.com/p/4yf)
* [https://arbital.com/p/4yl](https://arbital.com/p/4yl)

Tsvi BT
Jaime Sevilla Molina
Eliezer Yudkowsky
Jeremy Perret
Alexei Andreev
Eric Rogstad
Jessica Taylor
Stephanie Zolayvar
Eric Bruylant
Nate Soares
Kevin Clancy
Jack Gallagher
Silas Barta
Patrick Stevens
Joe Zeng


Complete lattice

A **complete lattice** is a [poset](https://arbital.com/p/3rb) that is closed under arbitrary [joins and meets](https://arbital.com/p/3rc). A complete lattice, being closed under arbitrary joins and meets, is closed in particular under binary joins and meets; a complete lattice is thus a specific type of [lattice](https://arbital.com/p/46c), and hence satisfies [associativity](https://arbital.com/p/3h4), [commutativity](https://arbital.com/p/3jb), idempotence, and absorption of joins and meets.

Complete lattices can be equivalently formulated as posets which are closed under arbitrary joins; it then follows that complete lattices are closed under arbitrary meets as well.

%%hidden(Proof):
Suppose that $P$ is a poset which is closed under arbitrary joins. Let $A \subseteq P$. Let $A^L$ be the set of lower bounds of $A$, i.e. the set $\{ p \in P \mid \forall a \in A. p \leq a \}$. Since $P$ is closed under joins, we have the existence of $\bigvee A^L$ in $P$.  We will now show that $\bigvee A^L$ is the meet of $A$.

First, we show that $\bigvee A^L$ is a lower bound of $A$. Let $a \in A$. By the definition of $A^L$, $a$ is an upper bound of $A^L$. Because $\bigvee A^L$ is less than or equal to any upper bound of $A^L$, we have $\bigvee A^L \leq a$. $\bigvee A^L$ is therefore a lower bound of $A$.

Now we will show that $\bigvee A^L$ is greater than or equal to any lower bound of $A$. Let $p \in P$ be a lower bound of $A$. Then $p \in A^L$.  Because $\bigvee A^L$ is an upper bound of $A^L$, we have $p \leq \bigvee A^L$.
%%

Complete lattices are bounded
========================

As a consequence of closure under arbitrary joins, a complete attice $L$ contains both $\bigvee \emptyset$ and $\bigvee L$. The former is the least element of $L$ satisfying a vacuous set of constraints; every element of $L$ satisfies a vacuous set of constraints, so this is really the minimum element of $L$. The latter is an upper bound of all elements of $L$, and so it is a maximum. A lattice with both minimum and maximum elements is called bounded, and as this discussion has shown, all complete lattices are bounded.

Basic examples
=============

Finite Lattices
------------------------

The collection of all subsets of a finite lattice coincides with its collection of finite subsets. A finite lattice, being a finite poset that is closed under finite joins, is then necessarily closed under arbitrary joins. All finite lattices are therefore complete lattices.

Powersets
------------------

For any set $X$, consider the poset $\langle \mathcal P(X), \subseteq \rangle$ of $X$'s powerset ordered by inclusion.
This poset is a complete lattice in which for all $Y \subset \mathcal P(X)$, $\bigvee Y = \bigcup Y$.

To see that $\bigvee Y = \bigcup Y$, first note that because union contains all of its constituent sets, for all $A \in Y$, $A \subseteq \bigcup Y$. This makes $\bigcup Y$ an upper bound of $Y$. Now suppose that $B \in \mathcal P(X)$ is an upper bound of $Y$; i.e., for all $A \in Y$, $A \subseteq B$. Let $x \in \bigcup Y$. Then $x \in A$ for some $A \in Y$. Since $A \subseteq B$, $x \in B$. Hence, $\bigcup Y \subseteq B$, and so $\bigcup Y$ is the least upper bound of $Y$.

The Knaster-Tarski fixpoint theorem
=============================

Suppose that we have a poset $X$ and a [monotone function](https://arbital.com/p/5jg) $F : X \to X$. An element $x \in X$ is called $F$**-consistent** if $x \leq F(x)$ and is called $F$**-closed** if $F(x) \leq x$. A fixpoint of $F$ is then an element of $X$ which is both $F$-consistent and $F$-closed.

Let $A \subseteq X$ be the set of all fixpoints of $F$. We are often interested in the maximum and minimum elements of $A$, if indeed it has such elements. Most often it is the minimum element of $A$, denoted $\mu F$ and called the **least fixpoint** of $F$, that holds our interest. In the deduction system example from [https://arbital.com/p/5lf](https://arbital.com/p/5lf), the least fixpoint of the deduction system $F$ is equal to the set of all judgments which can be proven without assumptions. Knowing $\mu F$ may be first step toward testing a judgment's membership in $\mu F$, thus determining whether or not it is provable. In less pedestrian scenarios, we may be interested in the set of all judgments which can be proven without assumption using *possibly infinite proof trees*; in these cases, it is the **greatest fixpoint** of $F$, denoted $\nu F$, that we are interested in.

Now that we've established the notions of the least and greatest fixpoints, let's try an exercise. Namely, I'd like you to think of a lattice $L$ and a monotone function $F : L \to L$ such that neither $\mu F$ nor $\nu F$ exists.

%%hidden(Show solution):
Let $L = \langle \mathbb R, \leq \rangle$ and let $F$ be the identity function $F(x) = x$. $x \leq y \implies F(x) = x \leq y = F(y)$, and so $F$ is monotone. The fixpoints of $F$ are all elements of $\mathbb R$. Because $\mathbb R$ does not have a maximum or minimum element, neither $\mu F$ nor $\nu F$ exist.
%%

If that was too easy, here is a harder exercise: think of a complete lattice $L$ and monotone function $F : L \to L$ for which neither $\mu F$ nor $\nu F$ exist.

%%hidden(Show solution):
There are none. :p
%%

In fact, every monotone function on a complete lattice has both least and greatest fixpoints. This is a consequence of the **Knaster-Tarski fixpoint theorem**.

**Theorem (The Knaster-Tarski fixpoint theorem)**: Let $L$ be a complete lattice and $F : L \to L$ a monotone function on $L$. Then $\mu F$ exists and is equal to $\bigwedge \{x \in L \mid F(x) \leq x\}$. Dually, $\nu F$ exists and is equal to $\bigvee \{x \in L \mid x \leq F(x) \}$.

%%hidden(Proof):
We know that both $\bigwedge \{x \in L \mid F(x) \leq x\}$ and $\bigvee \{x \in L \mid F(x) \leq x \}$ exist due to the closure of complete lattices under meets and joins. We therefore only need to prove that $\bigwedge \{x \in L \mid F(x) \leq x\}$ is a fixpoint of $F$ that is less or equal to all other fixpoints of $F$. The rest follows from duality.

Let $U = \{x \in L \mid F(x) \leq x\}$ and $y = \bigwedge U$. We seek to show that $F(y) = y$. Let $V$ be the set of fixpoints of $F$. Clearly, $V \subseteq U$. Because $y \leq u$ for all $u \in U$, $y \leq v$ for all $v \in V$. In other words, $y$ is less than or equal to all fixpoints of $F$.

For $u \in U$, $y \leq u$, and so $F(y) \leq F(u) \leq u$. Since $F(y)$ is a lower bound of $U$, the definition of $y$ gives $F(y) \leq y$. Hence, $y \in U$. Using the monotonicity of $F$ on the inequality $F(y) \leq y$ gives $F(F(y)) \leq F(y)$, and so $F(y) \in U$. By the definition of $y$, we then have $y \leq F(y)$. Since we have established  $y \leq F(y)$ and $F(y) \leq y$, we can conclude that $F(y) = y$.
%%

TODO: Prove the knaster tarski theorem and explain these images

add !'s in front of the following two lines
[A Knaster-Tarski-style view of complete latticess](http://i.imgur.com/wKq74gC.png)
[More Knaster-Tarski-style view of complete latticess](http://i.imgur.com/AYKyxlF.png)

Summary:
A **complete lattice** is a [poset](https://arbital.com/p/3rb) that is closed under arbitrary [joins and meets](https://arbital.com/p/3rc). A complete lattice, being closed under arbitrary joins and meets, is closed in particular under binary joins and meets. A complete lattice is thus a specific type of [lattice](https://arbital.com/p/46c), and hence satisfies [associativity](https://arbital.com/p/3h4), [commutativity](https://arbital.com/p/3jb), idempotence, and absorption of joins and meets. Complete lattices can be equivalently formulated as posets which are closed under arbitrary joins; it then follows that complete lattices are closed under arbitrary meets as well.

Because complete lattices are closed under all joins, a complete lattice $L$ must contain both $\bigvee \emptyset$ and $\bigvee L$ as elements. Since $\bigvee \emptyset$ is a lower bound of $L$ and $\bigvee L$ is an upper bound of $L$,  complete lattices are bounded.


Square root

##Introduction
When you try to get the area of a square  %%note: A shape where all sides are equal%%  and know the length of a **side** %%note: Otherwise known as its **root**%% you multiply the length of one side by itself or [square](https://arbital.com/p/4ts) it.

![area of square](http://2.bp.blogspot.com/-hkA5woVs-NQ/VRw85xkXlDI/AAAAAAAAAFc/Xc7utStLO7E/s1600/C%2BProgram%2BArea%2BOf%2BSquare.jpg)

What if you don't know the length of a side and instead only know the area of a square and want to get the length of a side?
You would have to do something to the area that transforms it in the opposite direction that squaring the length of a side does. This is just like how placing 2 apples to a pile transforms a pile of apples by increasing the amount of apples and eating 2 apples transforms the pile back to the way it was. So in other words adding is to subtracting as squaring is to.... what?

Well this kind of transformation is called a **Square Root**.


Rationality

The subject domain for [epistemic](https://arbital.com/p/) and [instrumental](https://arbital.com/p/) rationality.

Eliezer Yudkowsky
Aaron Gertler
Alexei Andreev


Central examples

The "central examples" for a subject are examples that are referred to over and over again in the course of making several different points.  They should usually come with their own tutorials.


The empiricist-theorist false dichotomy

No discussion here yet:  See https://www.facebook.com/groups/674486385982694/permalink/784664101631588/


Intension vs. extension

To give an "intensional definition" is to define a word or phrase in terms of other words, as a dictionary does.  To give an "extensional definition" is to point to examples, as adults do when teaching children.  The preceding sentence gives an intensional definition of "extensional definition", which makes it an extensional example of "intensional definition".  See http://lesswrong.com/lw/nh/extensions_and_intensions/

In the context of AI, an "intensional concept" is the code or statistical pattern that executes to determine whether something is a member of the concept, while the "extension" is the set of things that are thus determined to belong to the concept.  The intensional concept "test: does 2 evenly divide x?" recognizes the even numbers 0, 2, 4, 6... as its extension.

Given the modern level of visual recognition technology, a neural network that tries to classify cat photos vs. noncat photos would have some cat photos in its extension, but almost certainly also many things we think are 'cat photos' that would fail to be in its extension and many non-cat-photos that did end up in the extension of that particular neural network.  The intensional concept would be the classifier network itself - its weights, propagation rules, and so on.


Guarded definition

A guarded definition is one where at least one position suspects there will be pressure to stretch a concept and make it cover more than it ought to, and so they set aside a term meant to refer *narrowly* to the things inside the concept.  Thus, if a term has been designated as a 'guarded definition', stretching it to cover new and non-central members that are not *very* clearly part of the definition, and agreed to be so by those who wanted to designate it as guarded, is an unusually strong discourtesy.  If the term was originated (or its special meaning was originated) specifically in order to set it aside as a narrow and guarded term, then it is a discourse norm to respect that narrow meaning and not try to extend it.

Example:  Suppose that Alice and Bob are having a conversation about natural selection.  Alice points out that since everything occurs within Nature, all selection, including human agricultural breeding and genetic engineering, seems to her like 'natural selection', and she also argues that consumer choice in supermarkets is an instance of 'natural selection' since people are natural objects and they're selecting which foods to buy, and thus her paper on watching people buy food in supermarkets ought to be funded by a program on evolutionary biology.  If Bob and his researchers then begin using the term 'ecologically natural selection' because they think it's important to have a narrow term to refer to just birds breeding in the wild and not consumer choice in supermarkets, it is an extreme discourtesy (and a violation of what we locally take to be discourse norms) for Alice to start arguing that really supermarkets are instances of ecologically natural selection too.


Strained argument

A phenomenological feeling associated with a step of reasoning going from X to Y where it feels like somebody is stretching a hand across a cliff and not quite making it.

Example:

Alice:  "Hey, do you think that guy there is stealing that bike?"
Bob:  "They could be the bike's owner."
Alice:  "They're cutting the lock off with a hacksaw."
Bob:  "Maybe they lost the key.  They'd have to get the bike back somehow, after that."
Alice:  "So what do you think will happen if I go over there and ask to see their driver's license and maybe take a picture of them using my cellphone, just in case?"
Bob:  "Maybe they'll have a phobia of being photographed and so they'll react with anger or maybe even run away."

At the point where Bob suggested that maybe the bicycle thief(?) would have a phobia of being photographed, you might have detected a sense of *strained argument* - even by comparison to 'maybe they lost the key', which should sound improbable or like an excuse, but not have the same sense of 'somebody trying to reach across a cliff that is actually too far to cross', or overcomplicatedness, or 'at this point you're trying too hard', as "Maybe they have a phobia of being photographed".


Solomonoff induction

Solomonoff induction is an ideal answer to questions like "What probably comes next in the sequence 1, 1, 2, 3, 5, 8?" or "Given the last three years of visual data from this webcam, what will this robot probably see next?" or "Will the sun rise tomorrow?"  Solomonoff induction requires infinite computing power, and is defined by taking every computable algorithm for giving a probability distribution over future data given past data, weighted by their [algorithmic simplicity](https://arbital.com/p/5v), and [updating those weights](BayesRule-1) by comparison to the actual data.

E.g., somewhere in the ideal Solomonoff distribution is an exact copy of *you, right now*, staring at a string of 1s and 0s and trying to predict what comes next - though this copy of you starts out with a very low weight in the mixture owing to its complexity.  Since a copy of you is present in this mixture of computable predictors, we can prove a theorem about how well Solomonoff induction does compared to an exact copy of you; namely, Solomonoff induction commits only a bounded amount of error relative to you, or any other computable way of making predictions.  Solomonoff induction is thus a kind of perfect or rational ideal for probabilistically predicting sequences, although it cannot be implemented in reality due to requiring infinite computing power.  Still, considering Solomonoff induction can give us important insights into how non-ideal reasoning should operate in the real world.

Additional reading:

- https://en.wikipedia.org/wiki/Solomonoff%27s_theory_of_inductive_inference
- http://lesswrong.com/lw/dhg/an_intuitive_explanation_of_solomonoff_induction/
- http://wiki.lesswrong.com/wiki/Solomonoff_induction

Eric Bruylant
Eliezer Yudkowsky
Alexei Andreev


Perfect rolling sphere

A "perfect rolling sphere" is metaphorically an object that is too simple or idealized to exist in real life.  It derives from an old joke about a physicist who tries to bet on horse races, starting from the assumption that each horse is a perfect rolling sphere.  Nonetheless, in a lot of cases, a perfect rolling sphere is a pretty good approximation to a roughly spherical wooden ball moving down an iron trough inside an atmosphere.  More importantly, you're probably not going to be able to analyze the 'realistic' case until you understand the perfect rolling sphere.

summary(Brief): A "perfect rolling sphere" is metaphorically an object that is too simple or idealized to exist in real life.  (But which might be illuminating to consider.)

Eliezer Yudkowsky
Alexei Andreev
Eric Rogstad


Team Arbital

People behind [https://arbital.com/p/3d](https://arbital.com/p/3d):

* [https://arbital.com/p/1](https://arbital.com/p/1) (co-founder and CEO)
* [https://arbital.com/p/5](https://arbital.com/p/5) (co-founder)
* [https://arbital.com/p/2](https://arbital.com/p/2) (adviser)
* [https://arbital.com/p/76](https://arbital.com/p/76) (software engineer)

Summary: Team Arbital is the official group for all Arbital employees.


Probability theory

Probability theory is the mathematics governing quantitative degrees of belief.


Well-calibrated probabilities

A system of [probability](https://arbital.com/p/1bv) assignments is "well-calibrated" if things to which it assigns 70% probability, happen around 70% of the time.  This is the goal to which [bounded rationalists](https://arbital.com/p/) aspire - not to perfectly predict everything, but to be no more or less certain than our information warrants.