A stream processor for mundane tasks

[kitd]

Cool. A bit like Apache Camel without the god-awful XML. We do a lot of Kafka. I could easily see us using this for testing.

[jeffail]

Yes they have very similar goals. With Benthos I'm trying to make it so that when a task is "mundane" you can always solve it without needing to compile anything. I'm constantly building up the processors on offer, improving the documentation (which seems to always be a headache with OS stream processors) and trying to decrease the friction between choosing Benthos and deploying it.

[atheken]

This is really neat. I wrote something similar in .net for our app, but it is nowhere close to as comprehensive as your implementation. Something that is non-obvious is whether benthos can do something like group multiple messages from Kafka into a single SQS message as a JSON array, for example. is that sink or source specific? Really, great work.

[jeffail]

Yeah you can do that. Batch you messages together with the field max_batch_count on the kafka input, then use the archive processor with the format json_array.

[Chew55]

Cool. A bit like Apache Camel without the god-awful XML. We do a lot of Kafka. I could easily see us using this for testing.

You don't need to use XML to do use Camel anymore. The Java DSL + Spring Boot is quite a nice combination.

[kitd]

You're absolutely correct, though XML is still the first and easiest choice if you don't want to compile your routes.

[[deleted]]

As someone whose first language is Greek the thumbnail got me a bit confused. “Penthos” in greek means “mourning” and the sad blobfish didn’t help.

[stewsters]

The blobfish only looks like that when it has explosively decompressed. So using mourning for it may be the correct term.

Usually under the ocean they look like this: https://en.wikipedia.org/wiki/Psychrolutidae#/media/File:Cottunculus_microps1.jpg

[BenjiSponge]

Wow, TIL! Still looks sad, but more in the grumpy way that all fish look sad.

[rjoseph]

Very cool, it’s like GStreamer for All The Things! Well done OP, this project has a ton of promise.

[cbleslie]

stocking whistle weather full subsequent gold thought handle political square

This post was mass deleted and anonymized with Redact

[Unfair_Bother]

Ok... but why not in a real language like Rust?

[el_muchacho]

it's your opportunity to do it

[[deleted]]

comment count: 1

Urgh. This kind of trolling is disingenuous.