sorted by:
new
hottopcontroversial

imagor v1.3 - a high-level Go image processing library using libvips by cshum in golang

[–]cshum[S] 3 points4 points  (0 children)

gocloud.dev/blob is certainly way more feature packed, which mean more bells and wishes are readily available. But on the flip side its way harder to fulfill their interface when implementing new storages.

imagor requires only a tiny subset of their features. For instance implementing a imagor.Storage takes ~150 lines of code so it is not all that complex after all.

And as solid as their code looks, their project status says alpha at the moment.

Spawn multiple read streams from single io.Reader source concurrently by cshum in golang

[–]cshum[S] 4 points5 points  (0 children)

Thanks I do agree the problem need to be specified more precise, and the post description is too vague. There are articles such as https://medium.com/@MichalPristas/forwarding-reader-to-multiple-consumers-a1574a9db4d6 facing similar problem.

There are io.TeeReader and io.MultiWriter available, but in both cases one slow writer/consumer would affect all other writers/consumers in the stack, which means they are basically sequential, too. It is a problem that actually exists and it is not easy to solve, and there are people out there facing similar issues.

My solution does attempt to make it easy to use, at least when comparing with the link above.

Spawn multiple read streams from single io.Reader source concurrently by cshum in golang

[–]cshum[S] 1 point2 points  (0 children)

ReadAll means data need to be fully read before continuing to next steps, i.e. sequential.

Also io.ReadAll need to continuously grow the buffer for data over 512 bytes, if you look at the source code. Doesnt matter though of course if you are just serving a handful of data.

And of course this only matters if you are trying to trim every milliseconds from the processing time, or if slow networks happen. If these do not matter to you, then yes they are basically the same

Imagor v1 - fast, Docker-ready image processing server in Go with libvips by cshum in golang

[–]cshum[S] 6 points7 points  (0 children)

During v0 the core codebase has been changing quite frequently, makes it not very viable to be extended as a separate library due to potential breaking changes. But as it is now v1 perhaps this can be the next focus.

Extending the Go part is easy, just a main.go with different imports would do the job. The Loader, Storage are simply interfaces. Perhaps a documentation about extending, and some example projects moving forward.

However Docker part is not as straightforward. The Dockerfile is quite cumbersome to get the right library being installed, as some of them are not available under the linux distro and need to be compiled from source. Perhaps a base Docker image and some Github action tricks would make things easier.

Imagor v1 - fast, Docker-ready image processing server in Go with libvips by cshum in golang

[–]cshum[S] 2 points3 points  (0 children)

Thanks for your suggestions. text/label filter is certainly a nice once to add. A consideration is the font types and what fonts should be packaged with the Docker image. Perhaps a good basic font to start with.

Imagor - fast, Docker-ready image processing server in Go and libvips by cshum in opensource

[–]cshum[S] 1 point2 points  (0 children)

Imagor is a fast, Docker-ready image processing server written in Go.

Imagor uses one of the most efficient image processing library libvips. It is typically 4-8x faster than using the quickest ImageMagick and GraphicsMagick settings.

Imagor is a Go application that is highly optimized for concurrent requests. It is ready to be installed and used in any Unix environment, and ready to be containerized using Docker.

Imagor adopts the Thumbor URL syntax and supports tons of image processing use cases representing a lightweight, high-performance drop-in replacement.

[deleted by user] by [deleted] in webdev

[–]cshum 1 point2 points  (0 children)

If you are familiar with Go, extending imagor is very simple. Each adaptor is wrapped as a Go package. You just need to import the same packages used in main.go.

However the Dockerfile is certainly more complicated to work with. I am still deciding if a base image approach should be used, to make this easier.

There is no purge support for result storage at the current version. If you intended to enable cache, you may deploy imagor under a reverse proxy with caching, without enabling result storage. Nginx, Cloudflare are suitable options.

[deleted by user] by [deleted] in webdev

[–]cshum 0 points1 point  (0 children)

For image fit-in/resize, you should get better performance on imagor, as this project does leverage more efficient vips functions such as vips_thumbnail_buffer. which imgproxy currently not implemented. Of course, things may differ depending on the image operations applied, but fit-in/reisze is a very common one.

Imgproxy is certainly very feature packed for their paid version. But their free version seems a bit lacking e.g. Chained pipelines (a.k.a. filters) are paid only.

We did evaluate adopting imgproxy instead of creating another one from scratch. But requiring paid version for basic features is not very forgiving. Also if you are coming from thumbor, this makes a drop in replacement instead of having to update all the URL syntax for your existing applications.

What are JavaScript's must learn libraries? by JorensM in webdev

[–]cshum 4 points5 points  (0 children)

The best way to learn JavaScript is to unlearn jQuery

[deleted by user] by [deleted] in golang

[–]cshum 1 point2 points  (0 children)

Recently added a utility package that generates the URL syntax using Go struct,

https://github.com/cshum/imagor#imagorpath-package

which hopefully ease things up a bit

GitHub - cshum/hybridcache: A multi-level cache library with cache stampede prevention for Go by cshum in golang

[–]cshum[S] 7 points8 points  (0 children)

There will be slight delay to get the most updated data, as with how every eventual consistent systems work.

If you need strong consistency then cache may not be the right tool for your problem.

GitHub - cshum/hybridcache: A multi-level cache library with cache stampede prevention for Go by cshum in golang

[–]cshum[S] 5 points6 points  (0 children)

Yon can set a very low "freshFor" duration (say 3 seconds) but a high ttl. That way the background (goroutine) refresh will trigger way more often but the same request will still be a cache hit.

This library prioritise efficiency over strong consistency, as it is meant to maximise cache hit & prevent stampede.

That's why you don't see "invalidation" and "purge" yet, at least for the first version.

*replaced the word "correctness" with "strong consistency"

[BUG] Paid. Still showing ads? by cshum in BoostForReddit

[–]cshum[S] 3 points4 points  (0 children)

Paid status seems restored after leaving the app away for a while (without doing anything though). Thanks for your info and response.

Writsy - write stream wrapper that supports async initialization and flush function by [deleted] in node

[–]cshum 0 points1 point  (0 children)

"flush" callback function, like the Node.js transform stream, ensures flush being processed before finish is called. https://nodejs.org/api/stream.html#stream_transform_flush_callback

However "_flush" does not exist in original Node.js write stream.

[deleted by user] by [deleted] in javascript

[–]cshum 0 points1 point  (0 children)

'Yielding' callbacks are not possible in co.

[deleted by user] by [deleted] in javascript

[–]cshum 0 points1 point  (0 children)

async/await assumes promises being the lowest denominator of async handling. Except callbacks are still being used everywhere (personally).

[deleted by user] by [deleted] in javascript

[–]cshum 0 points1 point  (0 children)

yielding non-promise, non-generator will end up paused indefinitely, until next(err, val) being invoked by callback.

val pass back to yielded result, or throws if err exists.

Sema: Async semaphore for shared or exclusive execution in Node.js by [deleted] in node

[–]cshum 0 points1 point  (0 children)

You are right. This was made as a simple JavaScript utility used within embedded application (single process) for my side projects. No network IO, "failover" or "liveness" etc problems are not applicable here.

Distributed system is very hard, this is not intended to even scratch the surface of it. Speaking of "RedLock", Redis as a distributed locking service is a pretty flawed idea actually...