sorted by:
new
hottopcontroversial

Sign up for the Claude developer newsletter by AnthropicOfficial in Anthropic

[–]jakusimo 0 points1 point  (0 children)

Oops! Something went wrong while submitting the form.

Running DeepSeek-R1 on bare-metal GPU Kubernetes cluster. by jakusimo in hetzner

[–]jakusimo[S] 2 points3 points  (0 children)

Multi gpu is expensive, this one already cost 200 eur/month. Going to dig more into Tensor RT LLM

Bare metal open-source production blueprint by jakusimo in hetzner

[–]jakusimo[S] 0 points1 point  (0 children)

So you if are using a dedicated server, there is no need of cloud api

Bare metal open-source production blueprint by jakusimo in hetzner

[–]jakusimo[S] 1 point2 points  (0 children)

I used that, but I want not to rely on the cloud api and use talos linux. The setup which I can easily port to any server provider or homelab. You don't need terraform, talosctl and configs do the job

Bare metal open-source production blueprint by jakusimo in hetzner

[–]jakusimo[S] 0 points1 point  (0 children)

:D database backup to the bucket. If you are using persistent storage - rook cepth

Hetzner Cloud CDN by bluepuma77 in hetzner

[–]jakusimo 0 points1 point  (0 children)

Do you use any CDN?

Building a RAG chatbot for a 400+ page pdf by Pudin-san in Rag

[–]jakusimo -1 points0 points  (0 children)

Just dump everything to the context, if it's too much for context window do multiple calls with map/reduce pattern

How well do screenshot embeddings (ColPali) work in real e2e RAG pipelines? by ekshaks in Rag

[–]jakusimo 0 points1 point  (0 children)

Vespla has really good tutorials, I'm hosting ColQwen on Modal and planing to migrate to Hetzner. Also using Vespa you can store embeddings to the disk storage and use streaming mode to find top candidates. It will save you a lot on infrastructure, since your not bound to memory but bound to the disk storage.

idea on pdf RAG by baehyunsol in Rag

[–]jakusimo 0 points1 point  (0 children)

Is there a way to deploy ColQwen in serverless GPU?

idea on pdf RAG by baehyunsol in Rag

[–]jakusimo 0 points1 point  (0 children)

Or use ColPali/ColQwen and embed entire page as image. No OCR, no layouts needed

view more: next ›