๐Ÿ“ Parametricity and Free Theorems โ€” getting theorems for free since 1989
๐Ÿ“ Type Theory & Formal Verification ๐Ÿ’ฌ 41 replies ๐Ÿ‘ 1,892 views ๐Ÿ• Last post: 3 hours ago parametricity system-F free-theorems reynolds
๐Ÿฑ
WadlerCatFan
Theorem Prover
โ˜…โ˜…โ˜…โ˜…โ˜…
Posts: 2,847 Joined: 2019-03-14 Rep: +412 Purrs/day: 7.4
#1
Quote Report โคถ Link

OK fellow type-theoretic cat girls, let's talk about one of my all-time favourite papers: Wadler's "Theorems for Free!" (1989). This paper is an absolute masterpiece and I'm still in awe of how much you can extract from a polymorphic type signature alone.

The core idea: given a most-general polymorphic type signature, you can derive a theorem that any inhabitant of that type must satisfy โ€” completely for free, without looking at the implementation.

Here's the motivating example. Suppose you have a function:

r :: forall a. [a] -> [a]

Without knowing anything else about r, parametricity tells us it must satisfy:

โˆ€ types A B, โˆ€ f : A โ†’ B: map f โˆ˜ r = r โˆ˜ map f

Why? Because r works on lists of any type a. Since it's given no operations over values of type a, all it can do is rearrange the list โ€” it has no way to inspect or create elements. So applying f before or after rearranging gives the same result. The function commutes with map.

โœจ Key Insight Given a most-general polymorphic type signature (taking as few constraints on its values as possible), we can generate for free a theorem to which any inhabitant of such a type must adhere.

The theoretical backbone: from the type of a polymorphic function we can derive a theorem that it satisfies. And this isn't just a party trick โ€” Wadler described an application of parametricity to derive theorems about parametrically polymorphic functions based on their types, and parametricity is the basis for many program transformations implemented in compilers for the programming language Haskell.

I'll walk through the full technical story in follow-up posts. But first โ€” who else here has been personally victimized by a free theorem? ๐Ÿ˜ธ

theorems before bedtime, every night without exception
๐Ÿˆ
PolyPurrfect
ฮป Abstractionist
โ˜…โ˜…โ˜…โ˜…โ˜†
Posts: 1,203 Joined: 2020-07-22 Rep: +188
#2
QuoteReport

Great thread OP! Let me add some historical context because I'm a huge Reynolds nerd.

The parametricity theorem was originally stated by John C. Reynolds, who called it the abstraction theorem. Reynolds published this in his landmark 1983 paper "Types, Abstraction and Parametric Polymorphism". Wadler's 1989 paper then popularized and extended it to the setting of Haskell-style polymorphism.

Reynolds' original framing was relational: Reynolds proved an Abstraction Theorem โ€” every term in Fโ‚‚ satisfies a suitable notion of logical relation โ€” and formulated a notion of parametricity satisfied by well-behaved models.

The key technical idea: types are interpreted as relations, not just sets.

Reynolds Abstraction Theorem: For any relation R : A โ†” A', and any well-typed term t : โˆ€X.T, (t[A], t[A']) โˆˆ โŸฆTโŸง(R) i.e. related inputs yield related outputs, for ANY relation R you care to choose.

This is where the magic comes from. You're free to choose any relation whatsoever, and the theorem still holds. Choosing relations cleverly gives you all the interesting free theorems.

In programming language theory, parametricity is an abstract uniformity property enjoyed by parametrically polymorphic functions, which captures the intuition that all instances of a polymorphic function act the same way.

โˆ€ cats. cats are cute
๐Ÿ˜ธ
LambdaCatgirl
Proof Assistant
โ˜…โ˜…โ˜…โ˜…โ˜…
Posts: 4,501 Joined: 2018-01-11 Rep: +703
#3 ๐Ÿ“Œ Notable Post
QuoteReport

Let me do the canonical walkthrough of the forall a. [a] -> [a] example properly because I've seen it explained incorrectly so many times on this forum (yes I'm looking at you, GradualTypingSkeptic).

We want to derive what any function r : forall a. [a] -> [a] must do. By the parametricity theorem, for any relation R : A โ†” B:

If xs and ys are R-related element-wise (i.e. in List(R)), then r(xs) and r(ys) are also R-related element-wise. Formally: (xs, ys) โˆˆ List(R) โŸน (r(xs), r(ys)) โˆˆ List(R)

Now pick the graph relation of an arbitrary function f : A โ†’ B:

R = {(x, f(x)) | x โˆˆ A}

Then (xs, ys) โˆˆ List(R) means exactly ys = map f xs. Substituting:

(xs, map f xs) โˆˆ List(R) โŸน (r(xs), r(map f xs)) โˆˆ List(R) โŸบ map f (r xs) = r (map f xs)

QED. For free. No peeking at the implementation required! ๐Ÿฑ

The consequences for forall a. [a] -> [a]: since any such function commutes with map, it can only select, duplicate, drop, or reorder elements from the input list โ€” it can never conjure new elements from thin air, because it has no way to construct values of type a.

๐ŸŽฏ Concrete Examples of forall a. [a] -> [a] Valid: id, reverse, tail, take n, sort (via comparison on a), cycle, [] (const empty)
Invalid: any function that creates elements, peeks at values, or changes their structure.

The intuitive explanation of this result is that r must work on lists of x for any type x. Since r is provided with no operations on values of type x, all it can do is rearrange such lists, independent of the values contained in them.

f . g = g . f (in the right monoidal category)
๐Ÿพ
ProofsArePrograms
Curry-Howard Enthusiast
โ˜…โ˜…โ˜…โ˜†โ˜†
Posts: 678 Joined: 2021-05-30 Rep: +91
#4
QuoteReport

The forall a. [a] -> [a] example is super clean, but my personal favourite demonstration of free theorems is the identity type:

Claim: any f : โˆ€X. X โ†’ X must be the identity function.

Proof via parametricity: let R = {(x, x)} (the singleton relation on an arbitrary point x). The parametricity condition says:

-- (x, x) โˆˆ R  โŸน  (f(x), f(x)) โˆˆ R
-- โŸบ  f(x) = x  for all x

Since x was arbitrary, f = id. There is only one inhabitant of forall a. a -> a!

Assume M : โˆ€X. X โ†’ X. Let t be a type and x โˆˆ [[t]]. We show that [[M]] x = x. Interpreting X by the singleton relation, we obtain [[M]] x = x. This holds for any x. Hence, [[M]] is the identity function.

Similarly for forall a b. (a, b) -> (a, b) โ€” you can prove it must be either the identity or swap. And for forall a. a -> a -> a, parametricity tells you it must be either const or flip const (i.e., it always picks the first or always picks the second argument).

โš ๏ธ Caveat: seq and general recursion These theorems hold cleanly in a total, purely polymorphic setting. In Haskell with seq or general recursion, parametricity is not "lost" when the language includes seq โ€” it is just weakened, in a precisely known way. Also, when the language includes just fix (general recursion), parametricity is weakened (not lost).
๐Ÿฑ
WadlerCatFan
Theorem Prover
โ˜…โ˜…โ˜…โ˜…โ˜…
Posts: 2,847 Joined: 2019-03-14 Rep: +412
#5
QuoteReport

Good responses everyone. Now let me talk about what makes this mechanically work โ€” the connection to naturality in category theory.

The free theorem for r : forall a. [a] -> [a]:

  map f โˆ˜ r  =  r โˆ˜ map f

...is exactly the naturality condition for a natural transformation r : List โ†’ List in the category of types and functions (Hask)! Free theorems are naturality conditions in disguise.

Natural transformation ฮฑ : F โ†’ G satisfies: ฮฑ_B โˆ˜ F(f) = G(f) โˆ˜ ฮฑ_A for all f : A โ†’ B Parametricity gives us this for FREE just from the type.

More generally, Wadler's key insight was to interpret Reynolds' theorem not only as a way of identifying different implementations of the same type, but also as a source of free theorems for polymorphic types.

The deepest version of this connection: Reynolds' abstraction theorem shows how a typing judgement in System F can be translated into a relational statement (in second-order predicate logic) about inhabitants of the type. The relational interpretation is exactly what gives you the naturality squares!

If you want to go deep on this, look up Hermida, Reddy, Robinson โ€” "Logical Relations and Parametricity". It makes the categorical picture completely explicit. Spoiler: it's enriched category theory and it's beautiful ๐Ÿพ

every commuting diagram is a theorem for free
๐Ÿ˜บ
PolyPurrfect
ฮป Abstractionist
โ˜…โ˜…โ˜…โ˜…โ˜†
Posts: 1,203 Joined: 2020-07-22 Rep: +188
#6
QuoteReport
WadlerCatFan wrote: ...free theorems are naturality conditions in disguise...

YES. And this is why GHC can use parametricity to justify compiler optimisations! The classic example is short-cut fusion (a.k.a. foldr/build fusion).

The key lemma underlying stream fusion / shortcut fusion is derivable from the free theorem for the foldr type. Because foldr has type:

foldr :: (a -> b -> b) -> b -> [a] -> b

...the free theorem tells you exactly how foldr must commute with functions, which enables GHC to rewrite:

map f . filter p . map g
-- โ†“ via free theorems (fusion)
foldr (\x acc -> if p (g x) then f (g x) : acc else acc) []

Parametricity is the basis for many program transformations implemented in compilers for the programming language Haskell. The free theorem is literally being used to prove compiler optimisations are safe. That's not just beautiful mathematics โ€” it's load-bearing infrastructure!

For more: Reynolds' Parametricity Theorem (also known as the Abstraction Theorem), a result concerning the model theory of the second order polymorphic typed ฮป-calculus (Fโ‚‚), has been used by Wadler to prove some unusual and interesting properties of programs. A purely syntactic version of the Parametricity Theorem shows that it is simply an example of formal theorem proving in second order minimal logic.

GHC RULES are free theorems you can actually run
๐Ÿฑ
LambdaCatgirl
Proof Assistant
โ˜…โ˜…โ˜…โ˜…โ˜…
Posts: 4,501 Joined: 2018-01-11 Rep: +703
#7
QuoteReport

Hot take: the most underrated free theorem is the one you get from Church-encoded data types. Consider:

type ChurchBool = forall a. a -> a -> a

-- The free theorem says: any f : ChurchBool satisfies
-- for all g : A โ†’ B, x : A, y : A:
--   g (f x y) = f (g x) (g y)

This means f must always pick the first or always pick the second argument โ€” it corresponds exactly to True and False! Free theorems essentially prove that Church encodings are uniquely inhabited by their intended meanings.

And the Church natural numbers: type ChurchNat = forall a. (a -> a) -> a -> a โ€” the free theorem for this type tells you that any such function must equal some n-fold composition. Free theorems justify the entire Church encoding programme!

forall a. (a โ†’ a) โ†’ a โ†’ a โ‰… โ„• (via free theorems) forall a. (a โ†’ a โ†’ a) โ†’ a โ†’ a โ‰… โ„• (in a different way) forall a. a โ†’ a โ†’ a โ‰… Bool

This connects back to Jean-Yves Girard and John Reynolds independently discovering the second-order polymorphic lambda calculus, Fโ‚‚. Girard additionally proved a Representation Theorem: every function on natural numbers that can be proved total in second-order intuitionistic predicate logic can be represented in Fโ‚‚. Reynolds additionally proved an Abstraction Theorem: every term in Fโ‚‚ satisfies a suitable notion of logical relation.

โœ Girard-Reynolds duality real and certified
๐Ÿพ
ProofsArePrograms
Curry-Howard Enthusiast
โ˜…โ˜…โ˜…โ˜†โ˜†
Posts: 678 Joined: 2021-05-30 Rep: +91
#8
QuoteReport

To close the loop on the "relational parametricity" formalism for newcomers reading this thread:

In programming language theory, parametricity is an abstract uniformity property enjoyed by parametrically polymorphic functions, which captures the intuition that all instances of a polymorphic function act the same way.

The formal machine is logical relations / relational parametricity. We assign to every type ฯ„ a relation [[ฯ„]]:

-- Base types: equality relation
[[Int]] = {(n,n) | n โˆˆ โ„ค}

-- Function types: maps related inputs to related outputs
[[A โ†’ B]] = {(f,g) | โˆ€(a,b) โˆˆ [[A]]. (f a, g b) โˆˆ [[B]]}

-- Universal types: quantify over ALL relations R
[[โˆ€X. T]] = {(f,g) | โˆ€A,B, โˆ€R : Aโ†”B. (f[A],g[B]) โˆˆ [[T]](R)}

The Parametricity Theorem then states: every well-typed expression e of ฮปF behaves the same as itself according to its type A โ€” that is, (e,e) โˆˆ EโŸฆAโŸงฯ. At first glance this is underwhelming โ€” of course e behaves the same as itself! The powerful part is that e behaves "according to its type A".

And it is powerful enough to provide behavioral guarantees, which Wadler christened "theorems for free".

Everything in this thread follows from choosing the right relation R at the right type variable. The whole art is picking cleverly! ๐Ÿพ

๐Ÿฑ
WadlerCatFan
Theorem Prover
โ˜…โ˜…โ˜…โ˜…โ˜…
Posts: 2,847 Joined: 2019-03-14 Rep: +412
#9
QuoteReport

Great summary from ProofsArePrograms. Wrapping up the first page: one thing I want to stress is that free theorems aren't just cute party tricks. They're genuinely useful engineering tools:

  • โœ“ Compiler optimisation โ€” GHC's rewrite rules, fusion, safe code motion
  • โœ“ API design โ€” a polymorphic type tells users what a function can't do
  • โœ“ Reasoning about abstract types โ€” representation independence follows from parametricity
  • โœ“ Verifying Church encodings โ€” uniqueness of Church numerals, booleans, etc.
  • โœ“ Category theory โ€” naturality conditions come for free

Next page I'll get into higher-rank types and whether free theorems still hold there, plus the drama around unsafeCoerce destroying parametricity. Stay tuned! ๐Ÿพ

(see page 2 for continuation)

forall f. f is a natural transformation

โœ๏ธ Post a Reply

โ†‘