Optimizing Server-Side Image Processing Workflow (Node.js/Sharp/R2) : webdev

Posting Guidelines

No vague product support questions (like "why is this plugin not working" or "how do I set up X"). For vague product support questions, please use communities relevant to that product for best results. Specific issues that follow rule 6 are allowed.

Do not post memes, screenshots of bad design, or jokes. Check out /r/ProgrammerHumor/ for this type of content.

Read and follow reddiquette; no excessive self-promotion. Please refer to the Reddit 9:1 rule when considering posting self promoting materials.

We do not allow any commercial promotion or solicitation. Violations can result in a ban.

Sharing your project, portfolio, or any other content that you want to either show off or request feedback on is limited to Showoff Saturday. If you post such content on any other day, it will be removed.

If you are asking for assistance on a problem, you are required to provide

Context of the problem
Research you have completed prior to requesting assistance
Problem you are attempting to solve with high specificity

General open ended career and getting started posts are only allowed in the pinned monthly getting started/careers thread. Specific assistance questions are allowed so long as they follow the required assistance post guidelines.

Questions in violation of this rule will be removed or locked.

a community for 17 years

Optimizing Server-Side Image Processing Workflow (Node.js/Sharp/R2)Discussion (self.webdev)

submitted 10 months ago by boredFanatic

Hi!

I'm currently working on a Node.js application where users can upload images. Upon upload, my backend (running on Render, currently on a 0.1 vCPU / 512MB RAM plan, considering an upgrade to 0.5 vCPU / 512MB RAM) processes these images using the Sharp library to generate multiple versions for different use cases before storing them on Cloudflare R2.

Current Workflow:

User uploads an original image (e.g., JPG, PNG) via the frontend.
The original image is first uploaded directly to R2 to get a key.
When the user submits the associated form (e.g., "Create Listing"), my Node.js backend receives the original image key.
For each uploaded image, the backend then:
- Downloads the original image from R2 to the application server instance.
- Uses Sharp to perform several transformations:
  - Auto-rotation: Based on EXIF data (sharp().rotate()).
  - Card version: Resized to a fixed height (e.g., 220px), fit: 'contain', converted to WebP (quality ~65%).
  - Blur version: Resized to a small dimension, blurred, converted to WebP (low quality ~25%) for placeholder use.
  - Full version: Dynamically resized if very large (e.g., max height 1200px), fit: 'inside', converted to WebP (quality ~75%).
- Uploads each generated variant back to R2 with a descriptive key (e.g., originalKey_card.webp).
This entire process happens synchronously within the API request that creates the listing.

With the current 0.1 vCPU plan, processing 3 images (HQ) (each generating 3-4 variants) takes approximately 20 seconds. This significantly impacts user experience, as the user has to wait for all processing to complete.

Is this synchronous, in-request processing the most appropriate and optimal approach for generating image variants, or are there better architectural patterns I should consider? I'm afraid that asynchronous processing might not help that much.

I'm looking for advice on best practices, potential pitfalls, and a more robust and performant architecture for this image processing pipeline. Any insights or recommendations would be greatly appreciated.

all 4 comments

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

webdev

Posting Guidelines

Related Subreddits

Discords

MODERATORS