igstan.ro

Mathematics is a Natural Science

Mon, 09 May 2022 00:00:00 +0000

I believe that mathematics is a science, a natural science to be more precise. By science I mean the process or method of distinguishing what's true from what's false and by natural science I mean the application of science to the natural world that surrounds us.

Most people would gladly admit that physics, chemistry and biology are natural sciences. They study nature in an attempt to find out rules that govern it, to delineate what's true from what's false by means of experiment. The common view is that what really distinguishes the aforementioned fields from the study from mathematics is that they use observation, via experiment, to determine which theories about the natural world are telling the truth and which are not.

What I'm trying to argue here is that mathematics is just as much about experiment and observation of the natural as any other science. The fundamental difference is that the world described by mathematics is the natural world of information. And just like physics can have hypotheses which will at some point be refuted or confirmed (not proved!, more on this later) by experiment, the same way mathematics presents conjectures which may at some point be proved true or false. And just like natural sciences need field scientists and experimentalists that get their hands dirty and devise and conduct experiments, the same way mathematicians will conduct experiments in the form of incomplete or failed proofs, until one of them will actually prove or disprove the mathematical proposition they set out to study. One salient point here is that there's no guarantee that the mathematician may actually find such a proof, it's a process of searching that may yield some answer or not — we have absolutely no guarantee. In a similar vein, the final proof produced by the mathematician (or even a computer) is actually a natural artefact! It is embodied in the physical universe and that's probably true even if the proof hasn't escaped the mathematician's consciousness. And I would argue that even if consciousness will ultimately turn out not to be an emergent phenomenon of the physical world, but a thing in itself, it will still be a manifestation of the natural world, the world within which we exist. So, yes, a proof is a natural artefact that we need to discover before we can say with any certainty if a mathematical statement is either true or false. But that also means that a mathematical proposition can't be just true or false, it can also be "unknown as of yet" — the law of the excluded middle.

There is however something quite different between mathematics and the other natural sciences. Mathematics is free to choose some axioms and run wild with them. Every mathematical fact is contingent on those axioms — assuming the axioms, such and such is true or false. But the other sciences don't have this luxury; they can't get to decide what are the axioms of the universe, they must assume. This small difference has interesting repercussions, one being that mathematics can prove things to be true, whereas the other sciences can only prove things to be false. Here, by "things" I actually mean universal statements, of the form "for all X", and not existential statements such as "there is X". The best the "classical" sciences can do is prove a universal statement false, but they have no hope of proving it true, because there's no way to be sure that we have taken into account all the axioms of the universe — there may be laws and situations that could render our beautiful universal equations false in a blink.

Mathematics has the luxury of saying: these are our axioms, our atoms and fundamental forces and everything derives from them. Any theorem or proposition that assumes more than the given axioms is considered nonsensical. Of course, one can always explore a new set of axioms to derive new worlds of consequences based on the respective axioms.

What really fascinates me here is that these axiom sets are formed within the universe. They are not just inventions of our mind, totally detached from what is outside us — if we can imagine it, it's natural, it's part of the universe, even if that universe is just the world of information. And I think that might explain the unreasonable effectiveness of mathematics.

I'm no mathematician, but even so, during my brief encounters I've found myself wondering how mathematicians could come up with some of their theorems. But it might not be quite that remarkable if we consider the world of possibilities engendered by some axioms as a world in itself, a world that the mathematician is more than happy to explore, to study its every corner, some banal, some not, or some spuriously so. And because human beings crave order, they might see patterns and try to generalize them, not much different than what a theoretical physicist might do, I presume. So they come up with a hypothesis, only they call it conjecture, and then go on and try to prove it. As noted above, the endeavour might be fruitful or not, or it might still be ongoing.

At this point we can try to take a few steps back, maybe a couple parsecs, and we might not see the two human endeavours as quite so dissimilar — the natural scientist will take out their magnifier, pencil and pad, explore the world abord The Beagle and come up with some theory of evolution, while the mathematician might take out, or not, a pencil and a pad and explore the curious creatures formed by the atoms of its universe, the axioms. However, at the end of the day, they will both be searching for structures in their worlds, for patterns of information, for peculiar connections between the building blocks of their respective worlds. But these are not isolated worlds, they share a common home, they're informational. And both our scientists will use computation to derive meaning.

Resources

A couple resources for anyone who hasn't seen them before:

Peter J. Denning — Computing is a Natural Science
Ian Stewart — A Subway Named Turing (I have actually read a version titled "Commuters and Computers: The Intelligent Subway", which I can't find online any longer)

join — My Favorite Unix Command, Probably

Fri, 06 May 2022 00:00:00 +0000

Consider the following case. We have two files storing some data about artists and their countries of origin:

$ ls
artists.txt   countries.txt

There are 15 artists and 6 countries:

$ tail artists.txt countries.txt
==> artists.txt <==
4 Γεωργία Νταγάκη
3 L.E.J
5 Lykke Li
1 Moonlight Breakfast
7 Irina Rimes
2 Jake Bugg
3 ZAZ
8 Lana Del Rey
8 Jackson C. Frank
4 Αλκίνοος Ιωαννίδης

==> countries.txt <==
France 3
Greece 4
Romania 1
Russia 6
Sweden 5
UK 2

The two files are related by means of the leading and trailing numbers, which represent country IDs (made up solely for the purpose of this blog post). Thus "3 L.E.J" relates to "France 3", meaning that L.E.J are from France.

Now, we don't have a lot of data here, it's just 15 artists, but it's already quite hard to see which artists are from where. If these had been records in a relational database we could have easily written an SQL INNER JOIN query to reveal the associations we're after:

SELECT artist.name, country.name
  FROM artist, country
 WHERE artist.country_id = country.id

We could maybe start an SQLite session, create the tables, import the data and finally execute the query. This may even be a viable approach, but it doesn't fit the plan for this blog post. However, if you're on an Unix system, there's a hidden gem of a command, aptly named join, that can do precisely what the above SQL query can, only it doesn't need a database, just regular files. So, how could we use join to reproduce the above query?

Inner Joins — Where are These Artists From?

$ join -t: -22 -o1.2 -o2.1 \
    <( sort         artists.txt   | sed 's/ /:/' ) \
    <( sort --key=2 countries.txt | sed 's/ /:/' ) \
    ;

Moonlight Breakfast:Romania
Adele:UK
Dido:UK
Jake Bugg:UK
Clio:France
Jain:France
L.E.J:France
Stromae:France
ZAZ:France
Γεωργία Νταγάκη:Greece
Αλκίνοος Ιωαννίδης:Greece
Lykke Li:Sweden

Left Joins - Artists without a Country

$ join -a1 -t: -22 -o1.2 -o2.1 \
    <( sort     artists.txt   | sed 's/ /:/' ) \
    <( sort -k2 countries.txt | sed 's/ /:/' ) \
  | grep --extended-regexp ':$' \
  ;

Irina Rimes:
Jackson C. Frank:
Lana Del Rey:

Right Joins - Countries without Artists

$ join -a2 -t: -22 -o1.2 -o2.1 \
    <( sort     artists.txt   | sed 's/ /:/' ) \
    <( sort -k2 countries.txt | sed 's/ /:/' ) \
  | grep -E '^:' \
  ;

:Russia

Full Joins — Artists and Countries, Both Unlinked

$ join -a1 -a2 -t: -22 -o1.2 -o2.1 \
    <( sort     artists.txt   | sed 's/ /:/' ) \
    <( sort -k2 countries.txt | sed 's/ /:/' ) \
  | grep -E '(^:)|(:$)' \
  ;

:Russia
Irina Rimes:
Jackson C. Frank:
Lana Del Rey:

Tabular Display

$ join -a1 -a2 -t: -22 -o2.1 -o1.2 \
    <( sort     artists.tx    | sed 's/ /:/' ) \
    <( sort -k2 countries.txt | sed 's/ /:/' ) \
  | sed -E 's/^:/𝐍𝐔𝐋𝐋:/; s/:$/:𝐍𝐔𝐋𝐋/' \
  | ( echo "  #:COUNTRY:ARTIST"
      echo "===:=======:======"
      sort -k1 -t: | awk -vOFS=: '{ print sprintf("%3d", NR), $0 }'
    ) \
  | column -ts: \
  ;

  #  COUNTRY  ARTIST
===  =======  ======
  1  France   Clio
  2  France   Jain
  3  France   L.E.J
  4  France   Stromae
  5  France   ZAZ
  6  Greece   Γεωργία Νταγάκη
  7  Greece   Αλκίνοος Ιωαννίδης
  8  Romania  Moonlight Breakfast
  9  Russia   𝐍𝐔𝐋𝐋
 10  Sweden   Lykke Li
 11  UK       Adele
 12  UK       Dido
 13  UK       Jake Bugg
 14  𝐍𝐔𝐋𝐋    Irina Rimes
 15  𝐍𝐔𝐋𝐋    Jackson C. Frank
 16  𝐍𝐔𝐋𝐋    Lana Del Rey

Refactored

# Our internal delimiter, needed because the artist names in our records
# contain spaces. This conflicts with the default separator(s) of all the
# commands below: spaces or tabs. It should be safe to embed in regexes.
declare -r delim=:

redelimited() {
  # Replaces the *first* space with our delimiter. This works wonderfully
  # for the datasets we have, but it would fail miserably if added a country
  # whose name contains a whitespace, such as "Sierra Leone".
  #
  # So... exercise for the reader: handle whitespaces in country names.
  sed "s/ /$delim/" "$1"
}

sort-by() {
  sort -k"$1" -t"$delim"
}

full-join() {
  join -a1 -a2 -t"$delim" -22 -o2.1 -o1.2 \
    <( redelimited artists.txt   | sort-by 1 ) \
    <( redelimited countries.txt | sort-by 2 ) \
    ;
}

show-nulls() {
  # Prepend/append 𝐍𝐔𝐋𝐋 to rows starting/ending with our delimiter.
  sed -E "s/^($delim)/𝐍𝐔𝐋𝐋\1/; s/($delim)$/\1𝐍𝐔𝐋𝐋/"
}

headers() {
  echo "  #${delim}COUNTRY${delim}ARTIST"
  echo "===${delim}=======${delim}======"
}

number-rows() {
  awk -vOFS="$delim" '{ print sprintf("%3d", NR), $0 }'
}

tabulate() {
  column -ts"$delim"
}

full-join | show-nulls | sort-by 1 | ( headers; number-rows ) | tabulate

Unicode vs UTF-8

Tue, 06 Jun 2017 00:00:00 +0000

I couldn't sleep and my mind wandered around, finally reaching a place where I would explain Unicode to an undefined audience. It was almost an hour of laying purposeless in bed and the explanation seemed rather nice and short, so I thought I'd get out of bed and put it down here.

Unicode vs UTF-8

There was a time when I had no idea what Unicode actually was. All I knew was that it's some sort of technology that makes computers understand how to display my first name, Ionuț. I also heard about UTF-8, which presumably did the same thing, but better. Naturally, I wondered what was the difference between them. I remember reading Joel Spolsky's article The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets, but for some reason it didn't click. This is my take on explaining the difference, 14 years later.

Charsets and Code Points

Imagine that we're gathering all the world's written chracters that exist or have ever existed. They would be Latin characters, Arabic, Japanese, emojis or even Klingon ones. Next, we put all of them in a single big ordered list. By doing this we implicitly assign each character a number — its index in that list. This very big list is called a charset and the indeces are called code points. In other words, a charset is a mapping from code points to characters.

The next step is to get all software manufacturers to use this charset. Whenever thay want to accept or output a certain character in their software packages, they should be representing them in the computer language using the corresponding code point of our charset, which is just a number.

Encodings

Computers, however, don't directly understand mathematical numbers. What they can do is interpret sequences of 1 and 0 as numbers. So we need yet another standard that puts everyone on agreement on how to represent these numbers in the binary language understood by a computer. This additional transformation is called an encoding. Put otherwise, an encoding specifies how to translate code points to sequences of 1 and 0.

32-bit Encoding

Where's the main difficulty with representing numbers in binary format? It's in determining where to stop reading a sequence of bits and then treat it as a mathematical number.

We could use 8 bits, i.e., a byte. However, with 8 bits we can only represent 256 numbers. That's clearly not enough for all the characters we've gathered, we have more than just 256. What next?

We can use 16 bits in a row to represent a number. That's a little better, we can now represent numbers up to 65535. Should it be enough? Well, it accomodates all the characters we've gathered so far and it allows some future ones. But just to be safe, let's use 32 bits in a row to represent numbers. This will allow our charset to acommodate as many as 4,294,967,296 characters. Done deal, each character will be represented in the computer code using 32 bits for each character.

16-bit Encoding

Are we done yet? Not quite, because the software manufacturers will notice that their users will mostly use characters that are assigned numbers less than 128, which means they could fit on 7 bits. Having them represented on 32 bits means a lot of wasted resources. So we come up with a better encoding, one that requires just 16 bits to represent a number.

As we've seen above, 16 bits allow us to represent numbers up to 65535. What shall we do for numbers greater than this one? We can apply a clever little trick. We can reserve a few 16-bit combinations which used alone don't encode any code point, but when seen in pairs are treated as 32-bit numbers. We decide to reserve 2048 such 16-bit combinations, where 1024 of them can only appear as the first component of the pair and the other 1024 can only appear as the second component. These are called surrogate pairs.

This trick means that we'll be able to represent 65536 - 2048 = 63488 characters using 16-bit numbers and an additional 1024 * 1024 = 1048576 characters using surrogate pairs. It all totals up to 1,112,064 characters. This is less than the 32-bit encoding, but we'll decide it's a good trade-off and move on. In addition, we'll also limit the maximum number representable in the 32-bit encoding to that of the 16-bit encoding, to keep things consistent. We'll call the first encoding UTF-32 and the second one UTF-16.

8-bit Encoding

Finally, we figure out there's an even better way to encode code points less than 128 using just 8 bits. So we come up with an encoding that uses an increasing number of 8-bit blocks, i.e., bytes, as we need to represent bigger numbers. This is also called a variable-length encoding because the number of bytes used to represent a character varies.

The main idea behind this encoding is to add a mark at the beginning of the first byte that says how many bytes the sequence is composed of.

 Code Point Range | Applicable Bit Sequence
------------------|---------------------------------------
 0 - 127          | 0xxxxxxx
 128 - 2047       | 110xxxxx  10xxxxxx
 2048 - 65535     | 1110xxxx  10xxxxxx  10xxxxxx
 65536 - 1114111  | 11110xxx  10xxxxxx  10xxxxxx  10xxxxxx

So, if we want to represent a number between 0 and 127 we can just output a 0 bit followed by the 7 bits that encode the number. For numbers between 128 and 2047 we need two bytes. The first one starts with the 110 bit sequence, while the second with the 10 bit sequence. The x characters can be used to fill in the bits of the number we want to represent. Similarly for the other two cases. We'll call this encoding UTF-8.

Quick Glossary

To summarize all the things we've discussed so far, here's a quick list of concepts:

charset: a mapping from numbers to characters;
Unicode: the best such charset we have today;
code point: an index into the charset;
encoding: a recipe describing how to represent code points as computer bits;
UTF-32: encoding where each code point is represented using 32-bit sequences;
UTF-16: encoding where code points are represented using 16-bit sequences;
UTF-16 surrogate pairs: pairs of 16-bit sequences that are used to encode code points that don't fit in a single 16-bit sequence. There are only 1024 surrogate pairs, i.e., 2048 such 16-bit sequences.
UTF-8: encoding where code points can be represented using a variable number of 8-bit sequences, from 1 to a maximum of 4.

Conclusion

As you've probably understood by now, UTF-8 is just a concrete realization of a more abstract concept called Unicode, so there's no point in comparing them.

A Safe Type-Indexed Set for Standard ML, Errata

Wed, 12 Apr 2017 00:00:00 +0000

It turns out that my contraption on how to make a type-indexed set safe in Standard ML doesn't actually work. It gives the illusion that the type system will prevent that kind of usage errors, but it actually won't. You still need discipline. Here's a very short example that breaks the solution. It's based on the code that's available in this gist:

- use "set.sml";
-
- fun reverseOrd ord (a, b) =
…   case ord (a, b) of
…     LESS => GREATER
…   | EQUAL => EQUAL
…   | GREATER => LESS;
-
- val set1 = ListSet.empty { wrap = Fn.id, compare = Int.compare };
-
- val set2 = ListSet.empty { wrap = Fn.id, compare = reverseOrd Int.compare };
-
- ListSet.union set1 set2; (* it typechecks... :( *)

So, if someone creates their own record, without going through an asRecord call, then the problem reappears. It can appear even without an asRecord call if the tagging type isn't hidden properly, i.e., made opaque via an opaque signature ascription: :>.

Is there an actual solution then? I've spent some brain cycles thinking about this and my answer so far is that there isn't. The main, technical reason is that SML provides type generativity only at the functor-level, not at the function-level. Here's what I mean by this.

Type Generativity

Functors in SML are generative, which means that exposed opaque types are different between different function invocations. We need a signature first, otherwise we can't hide the exposed types.

signature FOO =
  sig
    type t

    val foo : t
  end

Now we can write a zero argument functor that produces FOO structures where t is hidden/opaque due to :>:

functor Foo() :> FOO =
  struct
    type t = unit

    val foo = ()
  end

Let's try to use it now:

- structure F1 = Foo();
- structure F2 = Foo();
-
- val a = ref (F1.foo);
val a = ref - : F1.t ref
- a := F2.foo;
stdIn:17.1-17.12 Error: operator and operand don't agree [tycon mismatch]
  operator domain: F1.t ref * F1.t
  operand:         F1.t ref * F2.t
  in expression:
    a := F2.foo

As you can see, this is a type error, the type F1.t is seen as different from the type F2.t. This feature, where a functor produces different opaque types with each invocation is called generativity. There's no way to have non-generative functors in SML.

Back to the problem. What does generativity have to do with our initial problem? The crux of it is that we need to attach unique types to each ordering function we might come up with and the only way to produce unique types in Standard ML is via generativity, i.e., functors. So we need to pass the ordering function as an argument to some functor and have the functor produce a uniquely typed ordering function.

In conclusion, if we want typesafe union operations on sets, we need functors. And thus, my previous blog post is useless. The only thing it helped me with is that I learned a bunch of new stuff. Sorry for the confusion.

A Safe Type-Indexed Set for Standard ML

Sat, 08 Apr 2017 03:00:00 +0000

Update

The solution I present in this post turns out not to be safe. See the subsequent post for an explanation. What follows is the original article.

I recently discovered, while preparing the slides for my Modularity à la ML talk, a solution to a problem I've come across a while ago while implementing a type-indexed set data structure in Standard ML.

Functor-Based Data Structures

First of all, let's see how sets are represented in the SML/NJ library, which is what I've used initially. The SML/NJ library provides a collection of common data structures and algorithms that aren't part of the standard Basis library mandated by Standard ML. For set data structures, it exposes an ORD_SET signature that's meant for sets implemented using a predefined ordering on the contained elements (there is support for hash sets, too). Two functors produce structures that satisfy the ORD_SET signature: BinarySetFn and SplaySetFn. Here's a sample of using BinarySetFn with integer elements:

structure Main =
  struct
    (*
     * Create an IntSet module by instantiating the BinarySetFn
     * functor with a structure that provides ordering on integers.
     *)
    structure IntSet = BinarySetFn(struct
      type ord_key = int
      val compare = Int.compare
    end)

    fun main () =
      let
        val set1 = IntSet.fromList [1, 2, 3]
        val set2 = IntSet.fromList [2, 3, 4]
        val set3 = IntSet.intersection (set1, set2)
      in
        IntSet.toList set3 (* [2, 3] *)
      end
  end

The problem with this approach is that it's functor-based, which means that one has to call the BinarySetFn functor for each and every type of element that they want to put in a set data structure. You want a set of strings? Then you call the BinarySetFn functor with a structure implementing ordering on strings. You want a set of sets? Same thing: call the functor with an ordering on sets.

This is annoying because it's just too verbose and it's due to the fact that structures are not first-class values. They're part of the module language of Standard ML, which lives separately from the core language of values. Is there a better way? Maybe.

Type-Indexed Data Structures

At some point I decided I wanted to play around with a different design for such data structures that was not functor-based. The idea was to pass the ordering function as a parameter to the builder functions of the structures. Here's what I mean by that. The current ORD_SET signature looks similar to this:

signature ORD_KEY :
  sig
    type ord_key
    val compare : ord_key * ord_key -> order
  end

signature ORD_SET :
  sig
    structure Key : ORD_KEY

    type set
    type item = Key.ord_key

    val empty : set
    val singleton : item -> set
  end

So the ordering function is part of a sub-structure of the signature: ORD_KEY. What if instead we'd do this and eliminate the ORD_KEY signature entirely?

signature ORD_SET =
  sig
    type 'a set

    val empty : ('a * 'a -> order) -> 'a set
    val singleton : ('a * 'a -> order) -> 'a -> 'a set
  end

Now, the two "factory" functions will take a polymorphic ordering as an argument. The type parameter 'a will be the type of set elements. Presumably this ordering function will be stored inside a data structure that implements that abstract set type. Let's see how this could look like with a naive list-based set.

structure ListSet :> ORD_SET =
  struct
    type 'a set = {
      compare : 'a * 'a -> order,
      elems : 'a list
    }

    fun empty compare =
      {
        compare = compare,
        elems = []
      }

    fun singleton compare elem =
      {
        compare = compare,
        elems = [elem]
      }
  end

The set is represented using a record that stores the elements as a list, together with the ordering function for those elements. We can now add a couple other functions for inserting elements into an existing set and for obtaining the list representation back, which is now held hidden.

signature ORD_SET =
  sig
    (* continued *)
    val toList : 'a set -> 'a list
    val insert : 'a set -> 'a -> 'a set
  end

structure ListSet :> ORD_SET =
  struct
    (* continued *)
    fun toList { compare, elems } =
      elems

    fun insert { compare, elems } elem =
      let
        fun loop elems =
          case elems of
            [] => [elem]
          | head :: tail =>
              case compare (head, elem) of
                LESS => head :: (loop tail)
              | EQUAL => elems
              | GREATER => elem :: elems
      in
        {
          compare = compare,
          elems = loop elems
        }
      end
  end

I won't go in detail over the implementation of insert, but, in short, what it does is that it keeps the elements list sorted so that it's easier to find whether an element already exists in the list. It's still naive for a real set implementation that will most likely use a balanced tree, but it'll do. Let's now see how to use it:

- val intSet = ListSet.empty Int.compare;
- val strSet = ListSet.empty String.compare;
- val set1 = ListSet.insert intSet 1;
- val set2 = ListSet.insert set1 2;
- ListSet.toList set2; (* [1, 2] *)
- ListSet.insert strSet 2;
stdIn:16.1-16.24 Error: operator and operand don't agree [overload conflict]
  operator domain: string
  operand:         [int ty]
  in expression:
    (ListSet.insert strSet) 2
- ListSet.insert intSet "2";
stdIn:1.2-2.10 Error: operator and operand don't agree [tycon mismatch]
  operator domain: int
  operand:         string
  in expression:
    (ListSet.insert intSet) "2"
-

The Problem

As you can see, we didn't have to instantiate any functors here. The set is polymorphic just like a normal list and it's typesafe — one cannot put a string inside an int set or vice-versa. This is nice, but we're just about to hit the actual problem. Let's think how we would go about implementing a common set operation: union.

signature ORD_SET =
  sig
    (* continued *)
    val union : 'a set -> 'a set -> 'a set
  end

structure ListSet :> ORD_SET =
  struct
    (* continued *)
    fun union set1 set2 =
      let
        val { compare = compare1, elems = elems1 } = set1
        val { compare = compare2, elems = elems2 } = set2
      in
        raise Fail "not implemented"
      end
  end

I'm not going to fill in the actual implementation because what we have is enough to illustrate the problem. The issue we're facing is that we have two sets, each providing its own compare function and there's absolutely no guarantee that these two functions behave the same. For example, the following code typechecks, but it should't, because we're trying to merge sets with different orderings:

- fun reverseOrd ord (a, b) =
=  case ord (a, b) of
=    LESS => GREATER
=  | EQUAL => EQUAL
=  | GREATER => LESS;
-
- val set1 = ListSet.empty Int.compare;
- val set2 = ListSet.empty (reverseOrd Int.compare);
- val set3 = ListSet.union set1 set2; (* shouldn't compile, but it does *)

The same problem can manifest when the ordering function isn't stored as part of the set representation, but it's instead passed as an extra argument to each of the set operations, either explicitly or implicitly, i.e., using type-classes. It's actually even more hideous in that case because it creates problems even for the insert operation (imagine two insert calls on the same set instance, but supplying different orderings to each call). This is a problem that can appear in Scala, for example, whose type-class emulation does not guarantee coherence, i.e., a given type can only have a single associated instance of a particular type-class.

So, is there a solution to this problem in SML? Initially I thought there isn't, but a few days ago I realized there might be one, albeit not that simple.

The Solution

To recap, the problem appears for this call, where we can't statically guarantee that the two set instances carry the same ordering:

ListSet.union set1 set2

One idea of ensuring this property is to somehow tag the orderings through the type system; associate each ordering function with a unique type. We can achive this by wrapping an ordering function in a structure that opaquely exposes the type of the function. And two opaque types are equal only if they're part of the same signature. So, the first step towards opaque types is a signature, which will let us hide the type in the structure(s):

signature TAGGED_ORD =
  sig
    type t
    val compare : t * t -> order
  end

This looks exactly as the ORD_KEY signature defined by the SML/NJ library, but we're not done yet. Let's try to use it in a way that wasn't meant to be:

structure IntOrd :> TAGGED_ORD =
  struct
    type t = int
    val compare = Int.compare
  end

Notice the :> part where we're specifying that the structure must comply with the signature. The difference between : and :> is that :> will keep the representation of types hidden. In our particular case, it renders the IntOrd structure useless. We cannot use it at all:

- IntOrd.compare (1, 2);
stdIn:35.1-35.22 Error: operator and operand don't agree [overload conflict]
  operator domain: IntOrd.t * IntOrd.t
  operand:         [int ty] * [int ty]
  in expression:
    IntOrd.compare (1,2)

This says that IntOrd.compare only accepts types IntOrd.t, not int. We know they're the same, but the compiler does not because of the opaque signature ascription :>. What's to do? We need to add a new function to the signature that will let us lift int values to IntOrd.t values:

signature TAGGED_ORD =
  sig
    type t
    val wrap : int -> t
    val compare : t * t -> order
  end

structure IntOrd :> TAGGED_ORD =
  struct
    type t = int

    fun wrap a = a
    val compare = Int.compare
  end

We can now order integers again:

- IntOrd.compare (IntOrd.wrap 1, IntOrd.wrap 2);
val it = LESS : order

What the TAGGED_ORD.t represents now is basically the singleton type of the compare function defined in the structure. There will be no other type that will be equal to it, except itself.

We're still missing a thing, however. The type-indexed ListSet module will now need not only a compare function, but also a wrap function. And because modules aren't first-class values in Standard ML, we cannot pass our TAGGED_ORD structures to the ListSet.empty function. We need a way to pack those two functions together in a record and pass that instead. So, our final TAGGED_ORD signature looks like this:

signature TAGGED_ORD =
  sig
    type t
    type a
    val wrap : int -> t
    val compare : t * t -> order
    val asRecord : { wrap : a -> t, compare : t * t -> order }
  end

Notice that we had to introduce another type, a, that will represent the type of values we're lifting from. Now IntOrd becomes this (notice that the a type is exposed, but t is not, through what it's called a translucent signature ascription):

structure IntOrd :> TAGGED_ORD where type a = int =
  struct
    type t = int
    type a = int

    val wrap = Fn.id
    val compare = Int.compare
    val asRecord = { wrap = wrap, compare = compare }
  end

We're now finally ready to change our ORD_SET signature and ListSet structure to work with this new type of "tagged" orderings:

signature ORD_SET =
  sig
    type ('a, 'ord_t) set

    type ('a, 'ord_t) compare = {
      wrap : 'a -> 'ord_t,
      compare : 'ord_t * 'ord_t -> order
    }

    val empty : ('a, 'ord_t) compare -> ('a, 'ord_t) set
    val singleton : ('a, 'ord_t) compare -> 'a -> ('a, 'ord_t) set
    val toList : ('a, 'ord_t) set -> 'a list
    val insert : ('a, 'ord_t) set -> 'a -> ('a, 'ord_t) set
    val union : ('a, 'ord_t) set -> ('a, 'ord_t) set -> ('a, 'ord_t) set
  end

The 'ord_t type parameter in type ('a, 'ord_t) set represents the unique type associated with a wrap/compare record. So the set type isn't indexed only on the type of the containing elements, but also on the type of the ordering function. This new type parameter will ensure the type safety we're after.

Here's how the ListSet structure needs to change now:

structure ListSet :> ORD_SET =
  struct
    type ('a, 'ord_t) set = {
      wrap : 'a -> 'ord_t,
      compare : 'ord_t * 'ord_t -> order,
      elems : 'a list
    }

    type ('a, 'ord_t) compare = {
      wrap : 'a -> 'ord_t,
      compare : 'ord_t * 'ord_t -> order
    }

    fun empty { wrap, compare } =
      {
        wrap = wrap,
        compare = compare,
        elems = []
      }

    fun singleton { wrap, compare } elem =
      {
        wrap = wrap,
        compare = compare,
        elems = [elem]
      }

    fun toList { wrap, compare, elems } =
      elems

    fun insert { wrap, compare, elems } elem =
      let
        fun loop elems =
          case elems of
            [] => [elem]
          | head :: tail =>
              case compare (wrap head, wrap elem) of
                LESS => head :: (loop tail)
              | EQUAL => elems
              | GREATER => elem :: elems
      in
        {
          wrap = wrap,
          compare = compare,
          elems = loop elems
        }
      end

    fun union set1 set2 =
      let
        val { wrap = wrap1, compare = compare1, elems = elems1 } = set1
        val { wrap = wrap2, compare = compare2, elems = elems2 } = set2
      in
        raise Fail "not implemented"
      end
  end

The most important difference is in the insert function, where we need to call wrap on the elements that we want to pass to compare:

case compare (wrap head, wrap elem) of

The same thing would have to happen in the implementation of union as well, but we don't have one yet. All we're interested in is whether illegal calls are refused by the type checker. To check that, let's define yet another ordering on integers, a reverse ordering:

structure RevIntOrd :> TAGGED_ORD where type a = int =
  struct
    type t = int
    type a = int

    fun reverseOrd ord (a, b) =
      case ord (a, b) of
        LESS => GREATER
      | EQUAL => EQUAL
      | GREATER => LESS

    val wrap = Fn.id
    val compare = reverseOrd Int.compare
    val asRecord = { wrap = wrap, compare = compare }
  end

Finally, we can verify whether we can union two sets based on different orderings:

- val set1 = ListSet.empty IntOrd.asRecord;
val set1 = - : (a,IntOrd.t) ListSet.set
- val set2 = ListSet.empty RevIntOrd.asRecord;
val set2 = - : (a,RevIntOrd.t) ListSet.set
- ListSet.union set1 set2;
stdIn:110.1-110.24 Error: operator and operand don't agree [tycon mismatch]
  operator domain: (a,IntOrd.t) ListSet.set
  operand:         (a,RevIntOrd.t) ListSet.set
  in expression:
    (ListSet.union set1) set2
-

Success! Another illegal state made unrepresentable! Also, notice how the error message lets us know that we've been trying to use difference orderings (IntOrd.t vs RevIntOrd.t):

  operator domain: (a,IntOrd.t) ListSet.set
  operand:         (a,RevIntOrd.t) ListSet.set

That's all, folks! You can find the final version of the code in this gist.

Modularity à la ML (slides)

Sat, 08 Apr 2017 00:00:00 +0000

A few words regarding modularity in Standard ML and Scala. These are the slides for the presentation I held yesterday as part of Scalar Conf 2017, Warsaw, Poland.

On Determinism and Free Will

Thu, 05 May 2016 00:00:00 +0000

For the past two or three weeks I've been somehow obsessed with the idea that there's no free will, that humans possess no free will. This idea didn't just pop up in my head, it's the result of all the other ideas I've heard or read before. This time it clicked, though, and it clicked so hard that I feel I'm in the midst of an existential crisis. In high-school it was whether there's any purpose in life and I managed to overcome that one quite easily in retrospect. Now it's harder, but maybe there's nothing I can do about it, as I shall explain immediately. What follows is mostly a brain dump in an attempt to defuse my head.

Determinism vs Free Will

My current belief is that the universe is a deterministic entity, that the present is entirely determined by its past, that the past is determined by its past and so on, until the beginning of time. I'm also reticent of believing that there's anything else "outside" the universe, for example a God, heaven, angels and so on, although I kind of wish it were true.

Now, if the universe is all there is to it and we're also part of the universe, our minds included, then it must be that all that we feel we are is a deterministic result of the initial conditions of the universe. Every thought we had, have or will have can virtually be traced back to the beginning of the universe. Every decision we make, every action we take is a result of a previous configuration of the universe we're occupying. Where's free will then? Whose free will even?

It seems that the feeling of free will is one of those traits that natural selection has favored, but which bear no actual relation to the reality. We think we have free will because otherwise we'd get depressed and we'd commit suicide, I presume.

However, it gets meta. I believe these thoughts I've put down here were meant to exist from the very beginning of time. There's no contribution of any entity to anything. The particles I'm made of are now in a state that can influence other particles into some other states and them, in turn, influence others, i.e., any reader of this. A continuous movement of star dust in the universe, but certain clumps form our selves. Or rather, form machines complex enough to build a certain, most likely imprecise model of themselves, a model that sees the machine as having a special sauce — consciousness.

As with anything in this universe, my thoughts are bound to evolve over time. I might read about some discovery or opinion that will change how I see things right now. But will these thoughts change me at this moment? Technically, they won't cause anything to happen that wasn't meant to happen. Whether I'll decide to ignore them completely or get depressed, it was meant to be that way.

These aren't new thoughts, of course. There have been many more before me that have thought of this problem, but it's still an open one. It's called the free will problem and appears because there seems to be an incompatibility between determinism and free will. One quote that resonates with me, attributed by Wikipedia to Arthur Schopenhaur reads:

Man can do what he wills but he cannot will what he wills.

Atheists vs Free Will

A few years ago I had a small argument with a work colleague where I argued that what a person can achieve in life is limited by the biological substrate, to which he disagreed. At the time I didn't completely realized the conflict between determinism and free will. What really surprised me about that discussion was when my colleague said that I have a religious view of the world. A deterministic existence would be one where God dictates what you can or cannot do, whereas an existence where free will existed didn't require a God. It struck me as totally paradoxical. To me, free will requires something outside the universe, like a God or a soul, but a deterministic, machine-like existence doesn't require a God.

The reason I've recounted this is because I've recently come across a video where someone says basically the same thing — as an atheist he believed in free will and thought that only religious people would believe in a deterministic life. Isn't it ironic? It is to me.

I, Robot

You know, some movies related to artificial intelligence, the strong one, raise questions on how a robot might feel if it finds out that it's just a robot, that it's not the real thing. I'd like to see a movie where human beings realize that they're less than they think they are — just machines. Maybe someone has made such a film already?

I normally have a resources section at the end of my blog posts, with links to resources I've used while doing my research. I won't do this now because the research is still in progress and I'm pretty sure you can find them yourself just as easily.

Calculating an Object Graph's Size on the JVM

Tue, 23 Sep 2014 00:00:00 +0000

Here's a little something I learned today. Recently, I had the task of adding an in memory cache for an exchange rate web service to the project I'm working on. I've used Guava's CacheBuilder and set some random values for its maximum size and expiry periods. And the I got wondering, what would be the amount of memory this cache will need when full?

I had a faint idea, from Twitter, that a tool called jol (Java Object Layout) might be exactly what I needed, but I wasn't exactly sure. I investigated and, sure enough, it is a very good tool for finding not only the memory size of an object graph, but also how objects are laid out in memory on different JVMs. This later feature is useful when optimizing for CPU caches, but I've got no experience with this yet.

Back to the subject of this blog post — finding the total amount of memory for an object graph — things are actually quite simple.

Add Dependency

First, you want to add jol as a dependency in your favourite build system. I'm using sbt here.

libraryDependencies += "org.openjdk.jol" % "jol-core" % "1.0-SNAPSHOT" % "compile"

As you may have noticed, I've declared this dependency as compile time only. We need it during interactive development, but not (yet) at runtime.

Start Exploring

Next, fire up sbt console and try these commands:

import org.openjdk.jol.info.GraphLayout

println(GraphLayout.parseInstance("USD").toFootprint)
println(GraphLayout.parseInstance("USD").toPrintable)
println(GraphLayout.parseInstance("USD" -> "EUR").totalSize)

Here's the output I got when inspecting a simple tuple value.

scala> println(GraphLayout.parseInstance("USD" -> "EUR").toFootprint)
scala.Tuple2 instance footprint:
 COUNT   AVG   SUM DESCRIPTION
     2    24    48 [C
     2    24    48 java.lang.String
     1    24    24 scala.Tuple2
     5         120 (total)

In the first column we have an instance count per class: two instances of type [C or Array[Char], two instance of type String and one instance of type Tuple2.

The third column shows how much memory all the instance of a particular type occupy. The char arrays and the strings occupy 48 bytes each. The tuple takes 24 bytes. They all add up to a total memory size of 120 bytes.

The average column says what amount of memory is consumed, on average, by each instance. Why on average though? I don't have an answer to this yet, but I presume is becuse of byte alignment rules.

Another question might be why does a single String instance occupy 24 bytes? This is where another utility, called ClassLayout comes in handy:

scala> println(ClassLayout.parseClass(classOf[String]).toPrintable)
java.lang.String object internals:
 OFFSET  SIZE   TYPE DESCRIPTION                    VALUE
      0    12        (object header)                N/A
     12     4 char[] String.value                   N/A
     16     4    int String.hash                    N/A
     20     4    int String.hash32                  N/A
Instance size: 24 bytes (estimated, the sample instance is not available)
Space losses: 0 bytes internal + 0 bytes external = 0 bytes total

The output says precisely how the object fields will be lay out in the computer memory and how much space they take up.

The first item, the object header, contains class type information. It starts at offset 0 and takes up 12 bytes.

The second item is the first object field. In this case we have an array of chars, which is used to store the actual characters that make up the string. It starts at offset 12 and occupies 4 bytes. Similar for the subsequent two fields.

If you'd like to know more, the project's source tree contains a directory of samples demonstrating some of its features. It's a good place to learn more about jol.

References

Run a single specs2 example from sbt

Wed, 13 Aug 2014 00:00:00 +0000

Every now and then I need to run an single failing specs2 test from the sbt command line. Because it happens so seldom, though, I keep on forgetting how to do it. Google isn't very helpful either because I have to remember the specs2 terminology, i.e., specification vs example, when searching.

In this blog post I'll work with the following test:

package com.example.quux

import org.specs2.mutable._

// This class represents a **specification**.
class ExampleSpec extends Specification {
  "this is a group of example" should {
    "this is example one" in {
      true must be(true)
    }

    "this is example two" in {
      true must be(true)
    }
  }
}

Running a Single Specification

If you're interested in running all the examples in the ExampleSpec specification, the following sbt command will do.

sbt> testOnly com.example.quux.ExampleSpec

Or, using a wildcard for the package name, which is what I personally prefer.

sbt> testOnly *ExampleSpec

Running a Single Example

If, on the other hand, you'd like to run just the example named this is example two, you have to make use of the -ex specs2 flag. Please note, this argument only works with the specs2 testing framework. ScalaTest supports a different set of arguments.

sbt> testOnly -- -ex "this is example two"

This works, but the test reporter will show even the names of the groups that haven't been run, which makes it hard to figure out where's the actual failed test. For this reason, the command I'm actually using is akin to this one:

sbt> testOnly *ExampleSpec -- -ex "this is example two"

NB: you have to wrap the example name in double quotes. Single quotes won't do it.

Contravariant Functors — An Intuition

Thu, 31 Oct 2013 00:00:00 +0000

Today I came across a blog post listing different types of functors (expressed using Scala), and once again I found a mention of contravariant functors. I heard about them in the past, saw their signature, but... no epiphany.

In this post I'd like to gain an intution for what a contravariant functor is. I'll leave it for another post to understand the other scary one, the exponential functor (shouldn't be much harder I presume, seems to be composed of two other functors — one covariant, the other contravariant).

Here's the signature given for contravariant functor in the above blog post:

trait Contravariant[F[_]] {
  def contramap[A, B](f: B => A): F[A] => F[B]
}

It's pretty mind bending (for me) to understand how that f function can possibly get a handle over a value of type B when B can't be found in the argument list, only in the return type (as opposed to A). What's going on then?

A Concrete Example

After a brief search on the internet I found out that a good example of a contravariant functor can actually be seen in the Scala standard library. Ordering.by is a contramap incarnation, specialized for instances of Ordering.

def by[T, S](f: T => S)(implicit ord: Ordering[S]): Ordering[T]

The signature looks a bit different that the one used by Tony Morris, but they're the same conceptually (the former is just more general). What it says is that if you know how to compare elements of type S and you have a function relating elements of type T to elements of type S, then you know how to compare elements of type T.

There are two conceptual mappings here. At a higher-level you want to map Ordering[S] to Ordering[T], but in order to achieve that an opposite mapping is required — from T to S — hence "contra".

Generalizing we get that contramap is a function which says that if you give it some abstraction over a concept A, i.e., F[A], and a function that maps from a different concept B to the concept A, then it is able to give you back an abstraction over B, i.e., F[B]. Pretty abstract maybe, but bear with me.

Intuition

Here's a simple real world example inspired by this thread.

Everybody knows how to compare numbers, right? 2 is smaller than 3, 3 is greater than 2. Also, everybody knows what money are. It's a concept we've been using for ages now, and it's pretty trivial to draw a correspondence between money and numbers. We'd map a one dollar bill (100 cents) to the number 100 for example.

Because we know these two things: 1. how to compare numbers; 2. how to map from money to numbers, we can actually compare money. Obvious, right? Well, calling contramap using those two facts as arguments will provide you the knowledge of how to compare money. That's exactly the knowledge that a contravariant functor encodes, but in a more general way, not only for numbers, money and comparisons.

As I said, the final goal is to actually map F[A] => F[B], but in order to achieve that we need a function that maps B => A, i.e. the opposite (contra) direction. It's basic abstraction. You build new abstraction out of old ones by specifying relationships between concepts.

Translating to Scala

Let's put the above example in Scala code. First, let's represent Money:

case class Money(amount: Int)

Scala already knows how to compare Ints and based on that knowledge we want to specify how Money should be compared, i.e., derive Ordering[Money] from Ordering[Int]. To achieve that, we need a way to specify which sub-component of Money is our desired Int, which is trivial in this example:

val contramapFn: Money => Int = (money) => money.amount

Having defined that, and given that an implicit instance for Ordering[Int] is already in scope, all we need now is to call the contramap function defined on the Ordering object, namely, the by method. The resulted Ordering[Money] is marked as being implicit because we want the compiler to use it when we use the comparison method <.

implicit val moneyOrd: Ordering[Money] = Ordering.by(contramapFn)

Now we can easily compare Money instances (we still need an implicit conversion from Ordering to Ordered in scope, hence the first import):

scala> import scala.math.Ordered._
import scala.math.Ordered._

scala> Money(13) < Money(20)
res0: Boolean = true

scala> Money(23) < Money(20)
res1: Boolean = false

Implementing `by`

How is Ordering.by implemented though? One possible implementation looks like this (the real one is a bit more complicated due to optimizations):

def by[T, S](f: T => S)(implicit ord: Ordering[S]): Ordering[T] =
  new Ordering[T] {
    def compare(x:T, y:T) = ord.compare(f(x), f(y))
  }

As you can see, it creates an Ordering instance whose compare method simply delegates to the compare method of the known Ordering[S], but not before transforming the x and y parameters by passing them through f.

This simple implementation also gives an idea of how f is able to receive an argument of type T. It creates a new "function" that will receive Ts and inside that "function" will we have access to these Ts to pass to f. When you need a value of a specific type and you don't have it, what do you do? You return a function that asks for one. In this particular case we don't have a proper function, but an object (an instance of Ordering), which is nothing more than a collection of functions, so the reasoning holds.

Conclusion

There are a few more concrete implementation of contravariant functors in the pages I've linked to in the resources section below, but the principle remains the same — you build new abstractions out of old ones by providing a mapping function from the new concept being abstracted to the old one that has already been abstracted.

igstan.ro

Mathematics is a Natural Science

Resources

join — My Favorite Unix Command, Probably

Inner Joins — Where are These Artists From?

Left Joins - Artists without a Country

Right Joins - Countries without Artists

Full Joins — Artists and Countries, Both Unlinked

Tabular Display

Refactored

Unicode vs UTF-8

Unicode vs UTF-8

Charsets and Code Points

Encodings

32-bit Encoding

16-bit Encoding

8-bit Encoding

Quick Glossary

Conclusion

Further Reading

A Safe Type-Indexed Set for Standard ML, Errata

Type Generativity

A Safe Type-Indexed Set for Standard ML

Update

Functor-Based Data Structures

Type-Indexed Data Structures

The Problem

The Solution

Modularity à la ML (slides)

On Determinism and Free Will

Determinism vs Free Will

Atheists vs Free Will

I, Robot

Calculating an Object Graph's Size on the JVM

Add Dependency

Start Exploring

References

Run a single specs2 example from sbt

Running a Single Specification

Running a Single Example

Contravariant Functors — An Intuition

A Concrete Example

Intuition

Translating to Scala

Implementing by

Conclusion

Resources

Implementing `by`