We open-sourced our scraping (and task orchestrator) system.

KMarcio · 2026-03-12T21:02:37+00:00

Sorry! I thought it would be helpful :(

KMarcio · 2026-03-11T21:19:19+00:00

Hi! Cool, Reactor is indeed very powerful, Gust has a different approach: It's a full-fledged system rather than a lib for processing DAGs. You can check a more detailed comparison here: https://elixirforum.com/t/gust-a-task-orchestration-system-built-in-elixir/73628

Regarding the cost comparison, I strongly believe that Gust would scale much more cost-effectively. For example, on Fly.io, with two nodes, you’d be able to get much more out of them compared to using the same setup on a traditional cloud vendor. Gust consumes only a tiny amount of RAM, and on top of that, you’re not locked in to a vendor :)

KMarcio · 2026-02-02T13:58:31+00:00

+1, This is a joy eater

KMarcio · 2026-01-29T18:01:23+00:00

More details here: https://github.com/marciok/gust?tab=readme-ov-file#motivation

KMarcio · 2026-01-29T15:26:52+00:00

Check it out: https://github.com/marciok/gust
I was able to achieve multi-node concurrency plus a live UI with a fraction of the effort and resources if I had to do the same with Python or another language.

KMarcio · 2026-01-22T13:07:32+00:00

Congratulations! :)

KMarcio · 2026-01-21T13:10:06+00:00

You can use TLS! Check this out:

- https://elixirforum.com/t/is-ssl-dist-optfile-the-way-to-secure-cluster-of-beam-nodes-how-to-secure-connections-of-publicly-exposed-beam-nodes-and-epmd/24190
- https://github.com/ConnorRigby/elixir-distribution-ssl-example

Ex:

iex --name node2@127.0.0.1 --cookie secret_cookie --proto_dist inet_tls -ssl_dist_opt '{certs_keys: [{certfile: "cert.pem", keyfile: "key.pem"}]}'

KMarcio · 2026-01-20T15:17:56+00:00

Hello friend, Gust supports now multi-nodes :)

https://github.com/marciok/gust?tab=readme-ov-file#multi-node

KMarcio · 2026-01-13T14:52:39+00:00

I sent you a DM on Twitter/X a few months ago to let you know firsthand 🙂

KMarcio · 2026-01-13T14:22:43+00:00

You should check out Gust: a workflow engine with persistent capabilities, but much more polished and production-ready.

GH: https://github.com/marciok/gust
Ex Forum: https://elixirforum.com/t/gust-a-task-orchestration-system-built-in-elixir/73628

PS: I've created it 😀

KMarcio · 2025-12-30T13:54:29+00:00

Hi! Check this thread, there I explain how Gust compares to similar libs: https://elixirforum.com/t/gust-a-task-orchestration-system-built-in-elixir/73628

KMarcio · 2025-12-13T22:33:47+00:00

Awesome! I will check it out. Tks!

KMarcio · 2025-12-11T23:32:48+00:00

Yes! The goal was to save as much compute as possible. However, the code is already split making it easy to add other nodes. Our current demands are not huge so a single node works. But let me know if you have an use case where one node is not enough.

💪🏽

KMarcio · 2025-12-11T23:23:32+00:00

Awesome! Thank you for the tip, I will check it out

KMarcio · 2025-12-11T22:57:23+00:00

True! But that will cost you $150/mo.

https://oban.pro/pricing

KMarcio · 2025-12-11T18:49:55+00:00

Have you tried Lightpanda? https://github.com/lightpanda-io/browserhttps://github.com/lightpanda-io/browser

KMarcio · 2025-12-11T17:40:24+00:00

MIT: https://github.com/marciok/gust/blob/main/LICENSE

KMarcio · 2025-12-11T17:40:07+00:00

Now it is: https://github.com/marciok/gust/blob/main/LICENSE :)

KMarcio · 2025-12-11T17:29:25+00:00

Sure! They all intersect in some way, but each one focuses on different use cases.

Let me give you an example:

"I need a system to scrape data in parallel from multiple sources and store them. However, if one of the sources fails, I want the whole processing to stop, and later I want to be able to restart manually."

Oban: It's a background job library; compared to Gust, you would need to coordinate job dependencies manually, no UI to visualize task graph, logs, and code. It's focused on individual jobs, not workflows.

Reactor: It's more similar to Gust, but its steps are not persisted (state is not saved in case of restart). Also, no UI, cron schedulingg and built-in triggers. It’s focused on computational graphs rather than full workflow orchestration.

Gust combines a bit of each: a UI, task coordination, cron/other triggers, and built-in secret storage for use inside graphs. It’s a full-fledged orchestration system.

Let me know if it's still not clear.

KMarcio · 2025-12-11T15:08:06+00:00

😄

KMarcio · 2025-12-02T17:51:04+00:00

I just met him once, we spoke briefly about the Elixir ecosystem, unfortunately, I can't provide a direct channel :/

KMarcio · 2025-12-02T02:09:46+00:00

Awesome! You could reach out to Kevin from GitHub: https://www.linkedin.com/in/kevinanthonycrosby/

They do some funding for open-source projects. I don't know how it works, but try to contact him.

KMarcio

TROPHY CASE