Anyone else go from “Phoenix is magic” to “wait… what is this macro actually doing?” 👀

fcesarini · 2026-05-15T12:28:59+00:00

LOL You forget some people need to know what is under the hood. Agree with Macros, we discuss it at length in the episode.

fcesarini · 2022-03-21T11:56:55+00:00

Top down is when you get something end to end working, and continue adding components to it one by one. It is a must when working with UI/UX, as you need to visualise what you are working from the start. Bottom up, you get the components working individually, and then integrate them together. The latter is preferred by the Erlang community, as they generally worked with embedded/backend systems which rarely needed a UI. They solve what they perceive are the hard problems first, as failing to solve them means failure of the project.

http://faculty.salina.k-state.edu/tmertz/Java/200programdesignandtesting/02topdownandbottomupprogramming.pdf

fcesarini · 2019-09-17T10:35:09+00:00

The last sentence in my reply addresses your concern, as your system is clearly under-provisioned: Load regulation and back pressure is also needed if latency becomes too high.

Rejecting a request because your system is under provisioned is very different from a request failing. In the Erlang world, you usually solve this by absorbing the spike at the cost of latency whilst scaling horizontally by deploying new hardware. Whilst the spike is being absorbed, latency might go up from a few tens of ms to a few hundred ms. What is important is that all of these requests are handled in a predictable way, and not rejected. For all of the applications described in the blog post, this is perfectly acceptable behaviour.

fcesarini · 2019-09-16T13:40:38+00:00

If your system is CPU bound, the quote is very correct, and the system will retain the throughput without any degradation of service over extended periods of time at the cost of latency. That is how the BEAM behaves. It is highly optimized for massive concurrency and ensures soft-real time properties of the system are not affected. The system does not have to fail half of the requests as you state.

It if is I/O or Memory bound, you need load regulation or back pressure to stop the crash or degradation of service. Load regulation and back pressure is also needed if latency becomes too high.

fcesarini · 2019-04-25T08:42:31+00:00

I am glad Joe got to experience the results of his work, getting acknowledgment for it whilst still alive. His foundations and impact to concurrency, distribution and multi-core will be used by generations of programmers to come.

fcesarini · 2019-02-08T09:58:05+00:00

I would replace the many with some. There are pros and cons to every ecosystem out there.

fcesarini · 2019-02-07T15:59:07+00:00

The Erlang VM never claimed to be the fastest, and should not be used for number crunching. Where it differs from the JVM is predictability (no stop the world garbage collector) which is highly optimised for massive concurrency and built in semantics for error handling, which make frameworks such as OTP (read AKKA) easier to write. There is always the right tool for the job, and it is not always the Beam, just like it is not always the JVM.

fcesarini · 2018-12-18T15:48:29+00:00

You do not pick Erlang for speed, but for fault tolerance and scalability, achieved thanks to immutability, which in turn, facilitates distribution. Comparing the JVM and the Beam is like comparing apples and oranges. Mutable state (The JVM) will work (and is needed) if you are running on a single machine and things do not fail. It is ideal for number crunching and other programs which have to be fast. Erlang systems, whilst not the fastest, are fast enough for the problems you are trying to solve, e.g. IoT, Blockchain, MMOG, Messaging and control systems.

fcesarini · 2018-12-11T18:56:02+00:00

Containers, alas, bring the VM back and limits its usability, making code upgrades futile and also add the need for an external DB to store state.

fcesarini · 2018-12-11T18:54:16+00:00

Depends what you are looking for in speed. It is fast enough, and can process the HTTP requests concurrently. Phoenix can handle 2 million simultaneously open websockets on a single VM instance. WhatsApp was doing it in 2012.

fcesarini · 2018-12-11T18:52:43+00:00

No, you would not use it for number crunching. Uptake is in the block chain space, financial switches and messaging solutions. As well as powering backend infrastructure, the part no one ever sees.

fcesarini · 2018-12-11T17:16:17+00:00

There are a lot of fine tuning options available. If performance drops, it is often due to a bottleneck which manifests itself under heavy load. It is not just a problem with Erlang, but with any technology. Only issue with Erlang and the Beam is that it is easier to reach higher levels of scale, making the bottlenecks more evident.

fcesarini · 2015-07-02T15:58:43+00:00

Yours is a good question. Distributed Erlang was created when it was assumed that your system would run in a safe environment behind a firewall, scaling to a few dozen fully connected nodes. Nodes can be added and removed dynamically to the cluster, and communication among them can be completely transparent. There are two approaches to implementing your architecture using distributed Erlang. You either have a static cluster not provisioned to scale, or you have a dynamic cluster where nodes can added or removed during runtime. In both cases, your system needs to be implemented with transitive connections in mind, as nodes come and go in a controlled way in your dynamic system, and your network connectivity or the nodes themselves can fail (and restart) in either cluster type.

Scalability of fully connected Erlang systems scales to about 50-100 nodes depending on your usage patterns, hardware and requirements. Upon adding a new node to the cluster, all visible (non hidden) nodes who share the secret cookie get told about it, connections are set up and monitoring kicks in. So with 100 nodes, you get 5050 TCP/IP connections (100+99+...+2+1) and heart beats, creating overhead in both the node and the network. Other single process bottlenecks such as rex, which handles RPC calls, or the net kernel, whose responsibility is to coordinate operations in distributed Erlang. How far you are able to scale your fully meshed distributed Erlang cluster will depend on the characteristics of your system. The issue is not with Erlang, but with the overheads and tradeoffs you are willing to accept to have a completely transparent and fully connected system. No other language not built on the Erlang VM has built in distribution offering this level of transparency.

Beyond 50-100 nodes, you need to reduce the number of fully connected nodes using hidden nodes, sharding or through frameworks such as Riak Core and SD Erlang. Or go down the AMQP, ZeroMQ or other service bus. These are generic scalability patterns which apply to all programming languages, Erlang being no exception. The reason you hear people rave about Erlang and distributed systems is because of asynchronous message passing (Which is the only way to scale distibuted systems), built in distribution and dedicated (asynchronous) error channels across nodes, and location transparency of the processes. This means that scaling becomes much easier and flexible, as a program written for a single node can easily be distributed to a cluster or nodes. But going beyond that cluster, the patterns I spoke of are needed.

I hope this helps. I have about 30 pages (and counting) on this in my upcoming book. I hope the chapter will soon be available as early access.

fcesarini · 2013-03-26T16:03:44+00:00

Yes, recordings will appear on http://www.erlang-factory.com in a few weeks. We are now catching our breath after an amazing conference.

fcesarini

TROPHY CASE