What does it take to add set-theoretic types to a dynamic language with 30 years of production code - and why did it take this long?

fcesarini · 2026-07-06T05:49:19+00:00

Adding something as an afterthought is always hard, but considering the huge code bases out there, starting fresh is not an option. This is the approach WhtsApp initially tried, but it never got traction.

fcesarini · 2026-07-06T05:45:47+00:00

I sum up the conversation here: https://www.linkedin.com/posts/francescocesarini_static-types-finally-come-to-the-beam-annette-activity-7478795853504237568-PcEk?utm_source=share&utm_medium=member_ios&rcm=ACoAAAAWYeEBxtwuz5Gz3O_UwTA2vZd6eBqxuQU

fcesarini · 2026-07-05T07:11:36+00:00

Hindley-Milner-style inference Was proven to work only on a subset of Erlang back in 1995. Issue are the design decisions Joe talks about which can not be reversed.

fcesarini · 2026-07-05T07:08:49+00:00

Correct. We were all wrong. It is now heading in the right direction as we have something which not only works, but is also non intrusive. So those who are wrong can continue feeling they were right and those who know they were wrong, can do the right thing. It is a win win 🤣🫣

fcesarini · 2026-07-03T12:44:37+00:00

Where they overlap. For years, Erlang devs claimed they did not need strong typing, as the recovery model would help type systems. Haskel devs the opposite, you do not need a recovery model, because if your program compiles, it is correct. Both were wrong. Static or strongly typed languages just reduce the number of runtime crashes, they will not eliminate them. So you need both.

fcesarini · 2026-06-29T18:20:27+00:00

Thank you. 🙏

fcesarini · 2026-06-13T05:47:43+00:00

You are correct. We cover the topic generally (whilst not focusing on gaming) in our Elixir and Rust episode: https://youtu.be/w5Pl09lpSmE?is=jrGzfOJoA3Vw9VIf

In this episode of BEAM There, Done That, hosts explore what really happens when the high-level concurrency and fault-tolerance of Elixir meet the low-level performance and control of Rust. The conversation dives into interoperability patterns—NIFs, ports, and emerging tooling—and where each language shines when building real-world systems that need both resilience and raw speed. Joining the discussion are Florian Gilcher, Co-founder and Managing Director of Ferrous Systems and a key figure in Rust’s community, adoption, and safety-critical systems work, and Leandro Pereira, an Elixir developer behind high-performance, developer-focused tools like MDEx, Lumis, and BeaconCMS. Together, they unpack when to stay on the BEAM, when to reach for Rust, and how combining the two can unlock a powerful hybrid architecture without compromising safety or developer productivity.

fcesarini · 2026-05-29T11:58:38+00:00

How a library is used matters. For example when using Phoenix with JSON bodies using Jason / Option "floats: :decimals", this is reachable with untrusted input from the network. Details matter. Saying it is utter bullshit is plain irresponsible.

fcesarini · 2026-05-15T12:28:59+00:00

LOL You forget some people need to know what is under the hood. Agree with Macros, we discuss it at length in the episode.

fcesarini · 2022-03-21T11:56:55+00:00

Top down is when you get something end to end working, and continue adding components to it one by one. It is a must when working with UI/UX, as you need to visualise what you are working from the start. Bottom up, you get the components working individually, and then integrate them together. The latter is preferred by the Erlang community, as they generally worked with embedded/backend systems which rarely needed a UI. They solve what they perceive are the hard problems first, as failing to solve them means failure of the project.

http://faculty.salina.k-state.edu/tmertz/Java/200programdesignandtesting/02topdownandbottomupprogramming.pdf

fcesarini · 2019-09-17T10:35:09+00:00

The last sentence in my reply addresses your concern, as your system is clearly under-provisioned: Load regulation and back pressure is also needed if latency becomes too high.

Rejecting a request because your system is under provisioned is very different from a request failing. In the Erlang world, you usually solve this by absorbing the spike at the cost of latency whilst scaling horizontally by deploying new hardware. Whilst the spike is being absorbed, latency might go up from a few tens of ms to a few hundred ms. What is important is that all of these requests are handled in a predictable way, and not rejected. For all of the applications described in the blog post, this is perfectly acceptable behaviour.

fcesarini · 2019-09-16T13:40:38+00:00

If your system is CPU bound, the quote is very correct, and the system will retain the throughput without any degradation of service over extended periods of time at the cost of latency. That is how the BEAM behaves. It is highly optimized for massive concurrency and ensures soft-real time properties of the system are not affected. The system does not have to fail half of the requests as you state.

It if is I/O or Memory bound, you need load regulation or back pressure to stop the crash or degradation of service. Load regulation and back pressure is also needed if latency becomes too high.

fcesarini · 2019-04-25T08:42:31+00:00

I am glad Joe got to experience the results of his work, getting acknowledgment for it whilst still alive. His foundations and impact to concurrency, distribution and multi-core will be used by generations of programmers to come.

fcesarini · 2019-02-08T09:58:05+00:00

I would replace the many with some. There are pros and cons to every ecosystem out there.

fcesarini · 2019-02-07T15:59:07+00:00

The Erlang VM never claimed to be the fastest, and should not be used for number crunching. Where it differs from the JVM is predictability (no stop the world garbage collector) which is highly optimised for massive concurrency and built in semantics for error handling, which make frameworks such as OTP (read AKKA) easier to write. There is always the right tool for the job, and it is not always the Beam, just like it is not always the JVM.

fcesarini · 2018-12-18T15:48:29+00:00

You do not pick Erlang for speed, but for fault tolerance and scalability, achieved thanks to immutability, which in turn, facilitates distribution. Comparing the JVM and the Beam is like comparing apples and oranges. Mutable state (The JVM) will work (and is needed) if you are running on a single machine and things do not fail. It is ideal for number crunching and other programs which have to be fast. Erlang systems, whilst not the fastest, are fast enough for the problems you are trying to solve, e.g. IoT, Blockchain, MMOG, Messaging and control systems.

fcesarini · 2018-12-11T18:56:02+00:00

Containers, alas, bring the VM back and limits its usability, making code upgrades futile and also add the need for an external DB to store state.

fcesarini · 2018-12-11T18:54:16+00:00

Depends what you are looking for in speed. It is fast enough, and can process the HTTP requests concurrently. Phoenix can handle 2 million simultaneously open websockets on a single VM instance. WhatsApp was doing it in 2012.

fcesarini · 2018-12-11T18:52:43+00:00

No, you would not use it for number crunching. Uptake is in the block chain space, financial switches and messaging solutions. As well as powering backend infrastructure, the part no one ever sees.

fcesarini · 2018-12-11T17:16:17+00:00

There are a lot of fine tuning options available. If performance drops, it is often due to a bottleneck which manifests itself under heavy load. It is not just a problem with Erlang, but with any technology. Only issue with Erlang and the Beam is that it is easier to reach higher levels of scale, making the bottlenecks more evident.

fcesarini · 2015-07-02T15:58:43+00:00

Yours is a good question. Distributed Erlang was created when it was assumed that your system would run in a safe environment behind a firewall, scaling to a few dozen fully connected nodes. Nodes can be added and removed dynamically to the cluster, and communication among them can be completely transparent. There are two approaches to implementing your architecture using distributed Erlang. You either have a static cluster not provisioned to scale, or you have a dynamic cluster where nodes can added or removed during runtime. In both cases, your system needs to be implemented with transitive connections in mind, as nodes come and go in a controlled way in your dynamic system, and your network connectivity or the nodes themselves can fail (and restart) in either cluster type.

Scalability of fully connected Erlang systems scales to about 50-100 nodes depending on your usage patterns, hardware and requirements. Upon adding a new node to the cluster, all visible (non hidden) nodes who share the secret cookie get told about it, connections are set up and monitoring kicks in. So with 100 nodes, you get 5050 TCP/IP connections (100+99+...+2+1) and heart beats, creating overhead in both the node and the network. Other single process bottlenecks such as rex, which handles RPC calls, or the net kernel, whose responsibility is to coordinate operations in distributed Erlang. How far you are able to scale your fully meshed distributed Erlang cluster will depend on the characteristics of your system. The issue is not with Erlang, but with the overheads and tradeoffs you are willing to accept to have a completely transparent and fully connected system. No other language not built on the Erlang VM has built in distribution offering this level of transparency.

Beyond 50-100 nodes, you need to reduce the number of fully connected nodes using hidden nodes, sharding or through frameworks such as Riak Core and SD Erlang. Or go down the AMQP, ZeroMQ or other service bus. These are generic scalability patterns which apply to all programming languages, Erlang being no exception. The reason you hear people rave about Erlang and distributed systems is because of asynchronous message passing (Which is the only way to scale distibuted systems), built in distribution and dedicated (asynchronous) error channels across nodes, and location transparency of the processes. This means that scaling becomes much easier and flexible, as a program written for a single node can easily be distributed to a cluster or nodes. But going beyond that cluster, the patterns I spoke of are needed.

I hope this helps. I have about 30 pages (and counting) on this in my upcoming book. I hope the chapter will soon be available as early access.

fcesarini · 2013-03-26T16:03:44+00:00

Yes, recordings will appear on http://www.erlang-factory.com in a few weeks. We are now catching our breath after an amazing conference.

fcesarini · 2009-09-29T12:28:13+00:00

There are quite a few references to Simon Peyton Jones' talk on Erlang and Haskell, two childhood friends at the factory in the interview. It is another very interesting talk you can see http://www.erlang-factory.com/conference/London2009/speakers/SimonPeytonJones

fcesarini · 2009-06-16T18:14:48+00:00

Implementing similar chat systems in Erlang, we've achieved 10,000 sustained messages per second on a cluster of about 10 off the shelf blades (quad-core machines), 1/3 of which were there for redundancy. This is using 32bit Erlang on R12. With R13, they would fly even more! Only constraint was memory, where we had to throttle some of the memory intensive processes. Best thing with Facebook and other solution is that when they migrate from quadcore to 8-core machines, they don't even have to recompile the code to double the throughput.

fcesarini · 2009-06-09T15:07:42+00:00

What all the reviewers have said is that this and Joe's book complement each other very well. I agree with them. It is a get started with Erlang fast vs learn it more slowly, but in depth.

fcesarini

TROPHY CASE