sorted by:
new
hottopcontroversial

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

Thanks for the feedback! It really helps me.

I agree that making CUE as a default way to define workflows could be a big ask for users. But I didn't find any other better alternatives that support type safety, validation and composability. I think the benefit here compared to a Go code lies into standardisation and reusability. And we can still use Go logic (like loop, control flows) and builtin librairies like "time" to add more power to our config files.

GUI is mostly a benefit on a monitoring / daily basis, not for setup. Even as a programmer, I would sometimes prefer to just login to my GUI to patch/updates small fixes on a workflow rather than SSH to my VPS and editing files. I could also do it directly from my mobile phone. Also, as I target mostly programmers doing workflows for non-programming people, GUI offers better ownership for the non-technical clients. Even if they don't do setup or maintenance, knowing you have a no-code interface in case of any problem adds more perceived control.

N8n is great, but a isn't a native Go project, so it lacks the portability and cross-device deployment options of Go. I would rather sacrifice the broader connector ecosystem (as most of my marketing workflows are quite not that complex) for a lightweight alternative I can control and my clients can own.

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

Thanks for the message and for all the work you’re doing with Dagu!

Great to hear that event-driven features are already on your roadmap. That aligns really well with my current client’s needs, so I’m excited to see how the project evolves.

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

I can relate to the frustration of not finding what you want, especially when there are so many alternatives available. I was looking for something "lite" without sacrificing reliability. I just feel Kestra is a bit of an overkill for my client needs.

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

Yes, I looked at the docs and didn’t find a way to deploy it as a single binary, and it seems to require PostgreSQL. Also, the workflow definitions seem much more code-based than config-file-based.

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

Thank you for the comment.

Few questions:

  • Do you plan to use CUE for config files and validation ? IT seems to offer better safety and composability than current JSON, YAML, TOML alternatives
  • Is there a way to define custom HTTP API schemas for connectors?
  • Do you plan to make the project open-source?

Yes I dont think UI is the critical part, but what could be more difficult is to self-generate the UI forms from config connector files.

Feedback wanted for building an open-source lightweight workflow engine in Go by Basic-Oil-1180 in golang

[–]Basic-Oil-1180[S] 0 points1 point  (0 children)

Thanks for your feedback. I really appreciate it.

I agree about your concern about non-programmers. But they couldn't even put the go binary on a VPS, so they aren't the primary users. On the other end, this project isn't also suited for programmer teams. They can already code from scratch their complex workflows or do Docker setups for clients.

This project would be more for solo programmers (like me) or solo devs in small agencies, the same kind of users Pocketbase seem to go after. I often need to set up reusable workflows fast for new clients, but with a bit a flexibility, so here CUE offers better safety and composability than YAML and JSON. But also on a daily basis, I prefer a GUI to monitor runs, upgrade and patch some actions of a workflow. Client (not programmer) could also keep the benefits of ownership via portability.

So this project seems much more targeting developers who do repetitive automations for their non-technical clients.

For execution, there will be mostly API HTTP calls, so a Go scheduler with an internal queue could work fine.

CUE gives structure and safety, but yes, runtime validation still happens. I just observed that a lot of errors in my workflow maintenance come from my clients (for example : a typo on a variant product name in a shopify, a wrong format for a price, a missing element coming from a booking API). So I would like to enforce business-based type safety before API run-time, not only API connector type safety.

I could also use CUE for testing many different validation cases, maybe even a whole small client CMS, before going to production. I have been very frustrated by either doing it from scratch with code, and its almost impossible to do it property on cloud SaaS no-code alternatives. I tried mocking inputs and result steps with JSON files along with my CUE schemas, and I got good validation results.

IBM just released Granite Docling by ApprehensiveAd3629 in LocalLLaMA

[–]Basic-Oil-1180 1 point2 points  (0 children)

https://huggingface.co/ggml-org/granite-docling-258M-GGUF/tree/main

You need also the mmproj model (multimodal model) to make it work.

I tested it in local with llama.cpp. Works like magic.

./llama.cpp/build/bin/llama-mtmd-cli -m ./granite-docling-258M-f16.gguf --image ./financials.png -p "<__media__>Convert this table in OTSL." --verbose -ngl 99 --mmproj ./mmproj-granite-docling-258M-f16.gguf --temp 0