Friday, 26 January 2007

Total Functional Programming

Recently on Lambda the Ultimate there was a post called "Total Functional Programming".  The title didn't catch my eye particularly, but I tend to read the abstract of any paper that is posted there that doesn't sound terribly boring.  I've found that this is a fairly good strategy since I tend to get very few false positives this way and I'm too busy for false negatives.

The paper touches directly and indirectly on subjects I've posted about here before.  The idea is basically to eschew the current dogma that programming languages should be Turing-complete, and run with the alternative to the end of supplying
"Total Functional Programming"

At first glance this might seem to be a paper aimed at "hair-shirt" wearing functional programming weenies.  My reading was somewhat different.

Most hobbiest mathematicians have some passing familiarity with "Turing's Halting Problem" and the related "Goedel's Incompleteness Theorem".  What this paper is trying to do is say "So what!".  The basic idea is that restriction to a syntacticly restricted language can restrict the semantics in such a way that these problems do not apply.  The resulting (*huge*) class of programs can then solve almost everything that we think is important. 

The "Bazooka" of the paper is a line describing how every program for which we have a complexity upper bound, is in fact inside the restricted language.  This realisation is profound.  It means that all of the algorithms that anyone has bothered to find upper bounds for (a huge class of programs mind you!) is accessible to this technique. 

The paper even deals with infinite data (co-data) in the framework and mechanisms to properly account for streams and other (possibly) infinite structures.

The idea of working in syntactically restricted languages that are sub-Turing (for any number of reasons) is not new.  In fact I think I've mentioned DataLog (wikipedia entry) previously.  In the case of DataLog however, there are serious deficiencies in the expressive power.

Definitely the best paper I've read this year.  I'm not sure how accessible it is to lay-people, but it should be accessible to anyone with a CS B.S. or people who have some familiarity with functional programming.

Saturday, 13 January 2007

Addition in the lambda calculus

I just spent a couple of very cold weeks in Anchorage Alaska.  I went out with a bunch of friends to a bar on the night prior to leaving, where I met up with an old friend (and listened to two other friends play music).  I started describing the lambda calculus to him since it is related to my work and he was curious what I was up to.  In any case I found myself unable to produce addition of the church numerals off the top of my head, which I found pretty disturbing since it shows that I probably didn't quite understand it in the first place.  Therefor I sat down this morning to see if I couldn't reconstruct it from first principles. 

It turns out that it is relatively easy. First, let us start with a bit of notation, and a description of the lambda calculus.  Since wikipedia describes the lambda calculus in detail, I'll just show how one procedes with calculations.  As examples let us start with the church numerals

1 = (λf x.f x)
2 = (λf x.f f x)
3 = (λf x.f f f x)
...
n = (λf x.fn x)

where fn means "f f ... f" n times.

The idea is that each numeral is represented by a lambda term.  The lambda term describes the arguments (f and x in this case) and the "body" in which we substitute the arguments when the are passed in.  An example of an "application" of a lambda term to an argument would be:

1 g = (λf x . f x) g => (λx . g x)

We pass in g, and replace every occurance of f in the body with g, and delete f from the list of variables.  we can continue to apply this term to another term

(λx . g x) y => g y

Which leads us to conclude that

1 g y => g y

Ok, so now that you've seen a few calculations, let us try to construct addition.  The first thing I tried to do is to construct the function +1.  Clearly, we want +1 n => (n+1).  n+1 is (λf x. f(n+1) x) which is also (λf x. f fn x).  Since n = (λf x. fn x) we need to somehow add another f.  The trick is to get the arguments to use the same symbols, and to remove the lambda abstraction.  We can do this by applying the church numeral to the symbols we want to use, in this case f and x.

(λf x. fn x) f x => fn x

So now we know how to get rid of the lambda abstraction.  Now we can add an f on the front, which will get us closer to n+1.

f (λf x. fn x) f x => f fn x

We are almost there.  Now we need to have the λf x part at the beggining so that we look like a church numeral.

(λf x. f (λf x. fn x) f x) => (λf x. f fn x) = (λf x. f(n+1) x)

Looking great so far!  Now we just need to be able to take the whole (λf x. fn x) form as an argument.  This turns out to be really easy, we just put a variable in it's place and add it to the front of the list of lambda arguments.

(λa f x. f a f x) (λf x . fn x) => (λf x. f (λf x. fn x) f x) => (λf x. f fn x)

This shows that (λa f x. f a f x) is the +1 function! 

+1 = (λa f x. f a f x)

Ok, so now that we have +1 can we get a function that takes an n and returns +n?  This would get us a long way towards addition.  We can call this function n->+n.

Ok, so we can guess what +n will look like easily.  It's going to look just like +1 except with n leading f's.

+n = (λa f x. fn a f x)

So we should try to figure out how to take the fn out of a church numeral, and place it there.  Well, we should apply the same trick of applying the church numeral to f and x so that we can extract the body.

(λf x . fn x) f x => fn x

ok, but really we want a to follow, based on +n, so instead of using x, let us apply the form to a.

(λf x . fn x) f a => fn a

Great!  Look how close we are. now we just need to abstract the a,f,x and place f x following it.
(λa f x. (λf x . fn x) f a f x) => (λa f x. fn a f x)

Ok, so now that we know how to take an n, and able to take the entire church numeral n as an a beta reduce it to n+1, let us abstract the n as an argument

(λb a f x. b f a f x)

Let us verify quickly that this function works.

(λb a f x. b f a f x) n =>
(λa f x. n f a f x) =>
(λa f x. (λf x. fn x) f a f x) =>
(λa f x. fn a f x)

Now, we can verify that this works on m, to obtain n+m as the +n function should:

(λa f x. fn a f x) m =>
(λf x. fn (λf x. fm x) f x) =>
(λf x. fn fm x) = (λf x. f(n+m) x) = n+m

Hooray! Not only did we find n->+n, but we have obtained the function "addition" for free.  Since, once we have the +n function we can apply it to m.  So we now have the function:

add = (λb a f x. b f a f x)

I haven't tried multiplication and exponentiation yet, but you are welcome to try!