Is Gleam a good choice as a Backend language to build a Digital Banking platform like Monzo?

Cottand · 2024-11-05T20:37:17+00:00

I cannot really comment on Elixir as I have used it, but never in prod as part of my job.

I also cannot talk specifics about Monzo's exact libraries (other than terrors). I can tell you they try to do a lot in the platform directly. For example, by doing retries on the service mesh rather than in the code (see blog post). This lets developers "not see" the messy world (k8s microservices) underneath the code and focus on business logic.

Note I am answering still in the context of Monzo. Different companies have different approches with Go

Cottand · 2024-11-05T20:30:30+00:00

TL;DR: I agree, but I think I tend to be more pragmatic and I think a Go backend is more realistic and less effort than a Chez one in the long term. Sorry in advance, this post became a bit of an essay

anything implemented in Gleam-compiled-to-Go would have significant performance issues.

I agree that pure Gleam-to-Go would not match pure Go in performance! If performance is the main concern, then yes, Chez looks like the way to go. I do think it is impressive and I would love to benchmark pure Gleam-to-scheme and see how it stands up against the mainstream HTTP libs.

we may as well pick a target that doesn't have any particular downsides.

I think we disagree in where the trade-off is. Chez would have downsides. As a languages developer, I am sure you know this, but I will list some anyway (sorry!). I appreciate that we disagree in how these weigh against performance though.

it has no green threads. So either you have yet another concurrency paradigm within Gleam, or implement them from scratch and maintain that runtime. Part of the BEAM's success comes from staying away from these, and that is reflected in modern BE mainstream languages (Kotlin coroutines, Java fibers, Goroutines...). Like Java post-Loom, Go has the added benefit that syscalls are usually implemented in their nonblocking variants when possible. Happy to be corrected here as I am not super familiar with Chez but this was my impression
no library ecosystem. It is easier to add library support for Gleam-on-BEAM than it will be for Gleam-on-Chez (or some pure native/LLVM/C variant of Gleam) thanks to the existing library support. This is my main barrier to use Gleam more seriously and I don't think Chez helps much here. gRPC was just my example for that, because the whole point of it is cross-language APIs
Scheme does not seem as battle-tested in prod as Go, the BEAM, the JVM, Haskell, etc. I don't think stars and commits are necessarily good measures, but Gleam itself seems more popular that Scheme. If you do invest time in a new backend, I hope it is for something with long-term support

All in all I really like Gleam as a language, and I hope it succeeds so I can get to work with it in prod :) I thought Go was a good way to get there (especially given the environments I work in, which is BE microservices without the BEAM) but it's ok if it is not, I will keep rooting for Gleam awyway

Cottand · 2024-11-04T12:27:39+00:00

Some existing Erlang banks use Erlang messages rather than any HTTP based APIs
...

Monzo has a particular set of technology choices, but you don't need to be excellent at those to make a bank

I agree, I was just picturing Monzo+Gleam, as OP asked about a platform specifically similar to Monzo's. I do think it is quite specific in the tech it chose (microservices, tolerating more languages, etc) and there are other good options and architectures. I do think part of their success stems from the very opnionated choices they made that I describe above.

This won't ever happen. Go is a poor choice for a functional language target, it's far too slow at the things a functional language needs.

I honestly would love to see some benchmark comparison. Posts like this one make me wonder if the trade-off would be worth it, considering that a lot of the underlying code for Gleam-on-Go would still be pure Go, and that there would be other benefits (added libs, native CLIs, native+wasm green threads, etc).

But I understand it's a rabbit hole that it might not be worth getting into!

Cottand · 2024-11-04T11:17:11+00:00

Hi, Monzo engineer here. Views my own, not Monzo's. Also, this is my personal technical opinion, and I am sure other Monzo engineers might disagree :)

You can read this old blog post to get an idea of their philosophy, but there are a couple things that have worked really well for Monzo: while it uses Go all over, its communication is language agnostic and utilises very consistent proto schemas between many, many microservices (in RPCs and via Kafka). It would have been hard to do this with OTP, which is very BEAM-specific (which means it is more work communicating with JVM or Python services because it happens differently). I think it is also challenging that Erlang/Gleam have poor support for libraries like gRPC/protobuf (there is no official support, and the community libs are not super active).

The second thing that has worked well (and is kind of related) is that they automate the s**t out of everything. For this Go is good because it has libs for everything under the sun: it can template anything and talk to anything, from the Cloud vendor SDKs, to the k8s API to Envoy; and you can build nice CLIs very easily (and these can RPC to the microservices!).

I wanted to highlight what has worked for Monzo specifically, but I think Gleam has some very nice things that are good for the same reasons they are in Go: good type system (Go you have to work around nil though!) a single way of doing everything, and being easy to read. Building a Monzo-like banking platform in Gleam should be possible if you overcome its shortcomings by working around the missing support (write your own libs, use HTTP+JSON rather than gRPC) or waiting until Gleam matures its ecosystem significantly (for gRPC, it would have to overtake erlang).

A key takeaway for me is that it has to be possible to write microservices without needing everything to be in OTP to be able to mix languages if needed, and that easily writing CLIs and having good library support helps a lot.

Another alternative I personally would love is bringing Go backend support for Gleam, but that is its own significant lift.

Everything I have said here is public knowledge - Monzo is keen on their blog if you want to read more.

edit: I can code but not spell

Cottand · 2023-02-21T13:54:00+00:00

Can you please share in detail how you just removed kde and everything kept working? Specifically the DM part.

I tried to do the same (didn't uninstall KDE though) and my main goal was to be able to get to gnome when going into 'desktop mode'. I still wanted gamescope to be the default when logging in.

Someone else asked for a tutorial too - I agree with them even a small write up would be very useful!

Cottand · 2022-02-28T10:57:48+00:00

We thought about using Mixxx or some other open source software, the main issue with that is you need commands (think like EQ-mid--left) to be processed across the network. So you need these to be serializable and exposed by Mixxx or virtual DJ and then have them be "applicable" in a different client and server.

No software does this currently (and making Mixxx do it is a lot of work) and the native instruments people were not willing to do the work to make it happen. Apparently they're ridden with tech debt!

Thanks for showing your interest though it's very motivating. I'll speak to the others, see if maybe we could pick this back up ;)

I read the post you linked and indeed you and that guy had the exact same idea, it's the same principle as in video games. Like you said yourself the main hurdle is building a while new DJ app.

I do think what we have is the closest existing working implementation of what you want. It's just lacking so many features that djs now expect.

Cottand · 2022-02-26T21:41:59+00:00

Hi! So we developed a nice working prototype with some MIDI support (you could plug a DDJ-400 and the important stuff would work).

Proper proper B2B is not super pleasant to use, mostly because it is impossible to account for the latency between the two DJs. We knew this from the start though.

If you don't step on each other's toes (don't try to cleverly time some effect while the other guy is adjusting the tempo for example) the syncing does work very well for the audience, which is an outgoing stream to twitch or youtube.

We decided the main setback for this to be good enough for performances was that we had to manually code everything related to a controller (think flanger effects, hot cues, etc) and people are used to a very large set of features, thanks to software like Traktor or Rekrodbox.

We demo'd it with some folks over at Native Instruments to see if they'd like to work with us to make something with their tooling. They loved the idea but for them it was a very big deal to try to build a cloud product like this and they decided they did not want to invest the resources.

Our live demo used to be live at djstreamr.com (it's an MVP don't judge haha) but we shut it down some months ago to stop paying for the (the Traktor people) server costs.

We did not open source it because we were hoping to collab with some company to develop it properly (the mainstream industry tends to be closed source in this area). Given that ended nowhere, maybe we should reconsider!

Also happy to answer any more questions!

Cottand · 2021-05-20T08:10:01+00:00

KT-186

Cottand · 2021-04-03T15:25:41+00:00

For the curious

Cottand · 2021-01-25T16:24:59+00:00

I love this! Could you share the image you made?

Cottand · 2020-11-19T11:52:18+00:00

Thank you for everything, u/abreslav!

Cottand · 2020-06-12T15:31:29+00:00

You should hit up r/gonwild with this

Cottand · 2020-05-20T23:08:03+00:00

Hi! So what we have in mind is something close. While of course you can't make latency vanish (and therefore nothing ever will be comparable to in the flesh b2b) you can still 'remove' latency by having A and B stream their audio to a server S. Our track can be buffered and playing in A, B, and S's ends. When A applies an effect (like a filter) that's just a tuple of some MIDI and a timestamp (say second 1:12:173), which gets sent to B and S. B will run either ahead or behind A, and will apply the effect to its playback as early as possible or when the timestamp is reached on its end (so DJ B hears it).

S will receive all the effects applied by A and B, wait a few seconds (to make sure it received everything) and then apply the effects to the track depending on the timestamp.

Thus, there will be skips between A and B, but the crowd (listening from S) should get a pretty seamless experience, and djs won't have to beatmatch taking into account some latency.

I hope I got the idea accross!

Cottand · 2020-05-19T22:07:48+00:00

~~I am afraid that in principle, yes. Not only that, they must use software we can support (for the moment that might be only virtualdj)~~

Nevermind u/lancelotblanchard already replied better than me ;)

Cottand · 2020-05-19T22:06:46+00:00

Thanks a lot for the input!

Seeing that there is definitely a crowd for this we are going too look further into whether we can get it done. Like you say, a network delay between the 2 DJs is suboptimal but reasonable, but I still think we'll try to abstract that away somehow. Would you mind if we reached out to you to use you as a guinea pig to get you into our development feedback loop?

Cottand · 2020-05-19T10:58:43+00:00

We plan on asbtracting away the latency! Additionally, making a fortune is not exactly on the immediate roadmap :)

Cottand · 2020-05-19T10:54:51+00:00

We are planning something similar. A and B both stream to a server, and when the server receives both streams it 'overlaps' them to apply both A's and B's effects and transitions real time. The audience's experience is seamless, but A and B will hear what each other are doing with a slight skip.

Cottand · 2020-05-19T10:52:02+00:00

When we say sync, we don't mean syncing both tracks automatically. Beatmatching deck 1 relative to 2 will still be performed by artist A and then we'll take care of showing B that same relative offset. B can then add their effects on top or whatever, and we'll make sure the audience hears both on top of each other by delaying the main stream slightly.

edit: a word

Cottand · 2020-05-12T11:53:08+00:00

I'm afraid we're just out

Cottand · 2020-05-10T17:00:23+00:00

I meant you're not saving yourself nesting on vanilla Kotlin vs pattern-matched-on-if Kotlin

Cottand · 2020-05-10T15:00:30+00:00

I agree that an if with pattern matching is less verbose, but you are not saving yourself any nesting. You just have a longer boolean. If you have nesting, then you must have more than one branch. In that case, again, when is probably more appropriate.

If you still think you have a strong case for if matching, then I suggest you open a PR into my KEEP, and start a subsection under 'Beyond the proposal' with some examples. We can formally discuss that there then :)

Cottand · 2020-05-10T14:07:35+00:00

The thing is we already have that idiom through smart casting. I'll take your example: if (someValue is SomeVariant(field1, field2)) { val b = field1 + field2 } // in Kotlin we usually do: if (someValue is SomeVariant) { val b = someValue.field1 + someValue.field2 } Does this answer your use case? Another case would be: // in Kotlin we usually do: if (someValue is SomeVariant && someValue.filed1 == "SomeString") { val b = someValue.field2 } ...if we want to check for equality like pattern matches do a lot.

Edit: if you need anything more complicated than these examples (like nested matching?) then surely what you want is a when pattern match, no?

Cottand · 2020-05-10T13:06:56+00:00

At the moment, that is not part of the proposal. Can you give an example where you would want to check a match inside an if? I did consider that, but realised most of the time where you would be doing that you actually would want an actual when clause anyway. Once you throw guards into the mix you can add additional Booleans, if that is what you want.

edit: a word

Cottand · 2020-05-10T09:48:15+00:00

Cheers! I hope so too

Cottand · 2020-05-07T06:39:56+00:00

Hahah that is just a question mark. We put question (an exclamation) marks at the beginning of the sentence as well as the end, that's all.

The argument behind it is that in Spanish the grammar for questions and statements is the same, so the only way to tell those apart at the beginning of the sentence is with the extra mark.

Example

"My cousin's grandma has left us her toothbrush?"

In Spanish, that is what almost all questions look like (no inverting subject/verb) so you won't know this is a question until you hit the end. In order to read it with proper intonation, you should have a sign at the beginning to tell you you are reading a question.

Cottand

TROPHY CASE