Wanted to start taking notes on both F# and MailboxProcessor and compare it to Erlang. Part 1 of the blog is to talk about MailboxProcessor and what it gives you in F#. In the next couple of weeks, talk about Erlang and how it manages messages. Maybe if I stop being lazy, I will write a comparison to show the two. I think this will be exciting.
At work there has been a team working with Erlang for some project. While I won’t go into the project, what I have heard is that Erlang is FAST! By looking at some companies already using Erlang, like WhatsApp, they can process a huge amount of messages, going back and forth between people. While this sounds like super awesomeness, what is the real deal behind the throughput? I started looking into F# lately, to see if there was similar mechanisms, and what I found is what I will talk about in this post.
F# is a .Net language that was brought around from devs at Microsoft about 2005. The community liked F# for being from OCaml roots, and if you wanted to, you could write an OCaml program and run it in the F# compiler or an OCaml compiler. While you could do this, the area F# shines in is interoperability with .Net applications. So, F# could take advantage of all the power of .Net, including TPL or Task Parallel Library. The one problem is that TPL is heavy. Costs at least 1 MB to spin up a thread in .net. So, if we are comparing F# with Erlang, this doesn’t look good for us, but is there something else we can look at? Say MailboxProcessor?
I know a horrible name, but looking at the history, it makes sense. I’m sending messages to and from other processes. Now if you look at Erlang, you get the same kind of roots, sending messages to and from other processes. What MailboxProcessor is doing is interesting: It takes a bunch of messages, dumps them on an internal queue, and processes them one by one. But, what is making it able to accept these requests, behind the scenes? Well, if you have talked to anyone doing Erlang, what you will find out is everything is a small message. As long as you have a small message, things are quick. The trick behind MailboxProcessor is just that, small quick messages.
F# has ThreadPools running behind the scenes, and if you are using async keyword, you have those threads I was talking about earlier. But F# handles them differently, but listening for requests, and then storing them on a queue for more work. While that is going on, another thread is busy handling these request, passing them on to the next process. This allows F# to spin up a bunch of threads initially, and reuse them as things are being processed in the system. All those messages you have queued up, they are not on a thread, but somewhere in memory waiting to be processed.
Of course, I am just a little of the way through learning about MailboxProcessor and I wanted to check out more about it, before I start looking into Erlang. If you have some thoughts, comments or comparisons, please let me know.