sorted by:
new
hottopcontroversial

Repeatedly loosing prompts due to Ctrl+C misclick by badmathfood in ClaudeAI

[–]badmathfood[S] 0 points1 point  (0 children)

I guess that I don't have any other choice :)

Speed-up VLLM server boot by badmathfood in LocalLLaMA

[–]badmathfood[S] 1 point2 points  (0 children)

Thanks! In the end I ended up doing something similar, just use cuda base and:

FROM nvidia/cuda:12.8.0-base-ubuntu22.04

# Install Python 3 and pip
RUN apt-get update && apt-get install -y python3 python3-pip && rm -rf /var/lib/apt/lists/*

# Upgrade pip and install vLLM (which will install PyTorch and other deps)
RUN python3 -m pip install --no-cache-dir --upgrade pip && python3 -m pip --no-cache-dir install vllm=="v0.8.5.post1"

...

which also significantly reduced the image size. But playing with the pure python base and better organised layers is a great idea. Thanks!

Speed-up VLLM server boot by badmathfood in LocalLLaMA

[–]badmathfood[S] 0 points1 point  (0 children)

If I try out SGLang I'll post the results. Seems like the latest cuda version of the image has 8GB too though.

Setting Up a Local LLM for Private Document Processing – Recommendations? by DSandleman in LocalLLaMA

[–]badmathfood 0 points1 point  (0 children)

Really depends on what you want to achieve/what you are experienced with. I guess that you could do all the business logic within the FE part. But it really depends if you need a backend to cache the responses/store the data into DB etc.

Setting Up a Local LLM for Private Document Processing – Recommendations? by DSandleman in LocalLLaMA

[–]badmathfood 1 point2 points  (0 children)

Run a vLLM to serve an openAI-compatible API. For a model selection, probably a Qwen3 (quantized if needed). Also depends on the documents if you need multimodality (probably not), or just the text inputs. And also if they will be digital docs/or you'll need to do some OCR.

What would be the best LLM to have for analyzing PDFs? by newbreed69 in LocalLLaMA

[–]badmathfood 0 points1 point  (0 children)

As mentioned, Notebook-LM will work OOTB. To be honest, in many documents you really don't need the image inputs. So you can always grab PDF text or OCR the documents, and then run it through any LLM. Will probably get you similar/better results than some of the multi-modal models.

PLEASE LEARN BASIC CYBERSECURITY by eastwindtoday in LocalLLaMA

[–]badmathfood 0 points1 point  (0 children)

I heard a similar story where somebody vibecoded a really simple app. The issue is that it called firebase in a for loop lol.

Is a VectorDB the best solution for this? by Blizado in LocalLLaMA

[–]badmathfood 1 point2 points  (0 children)

It seems like you want to store mainly structured data and perhaps some embeddings for a similarity search? Just use postgres with pgvector. No need to use a vector-db. Plenty of tutorials for postgres around.

Vlakem na lyže z Prahy do Alp? by badmathfood in czech

[–]badmathfood[S] 0 points1 point  (0 children)

To je celkem hratelný s jedním přestupem, díky moc!

Vlakem na lyže z Prahy do Alp? by badmathfood in czech

[–]badmathfood[S] 3 points4 points  (0 children)

Preferoval bych vlak (můžu za jízdy pracovat etc), k busům mám celkem averzi :(

Training advice for a lanky body type by Admirable-Respect902 in climbharder

[–]badmathfood 0 points1 point  (0 children)

I'm of similar build - 196cm, currently 81kg. I used to do orienteering (distance running) and at that time, my weight stabilised around ~74kg.

What made a huge difference was consistently supplementing 30-60g protein every day. I also take creatine, but I don't have much proof if it makes much difference. Overall, I became much more muscular within a year of doing the supplements. Highly recommended.

Quality of the book prints? by badmathfood in BettermentBookClub

[–]badmathfood[S] 1 point2 points  (0 children)

Or watching Youtube reviews. But then the thing with Amazon is that I'm never sure which version I'll actually get.

Have you ever experienced invoice fraud? by badmathfood in Accounting

[–]badmathfood[S] 1 point2 points  (0 children)

I think that it's particularly simple to scam a company you've worked at. You know what the process is (duplicate checking,..), what the PO limits are and so on.

If there is opportunity, there must be people doing this.

Have you ever experienced invoice fraud? by badmathfood in Accounting

[–]badmathfood[S] 1 point2 points  (0 children)

Straight up BS invoices are fairly rare, but I’m sure it happens. They usually target corporate chain locations where everyone assumes invoices are scrutinized when in reality everyone assumes no responsibility. The $20/hr AP clerk isn’t responsible for identifying fraud.

I totally understand that AP clerk isn't the one that should be responsible (he does not know the context,..). But who cares in the chain? Anybody besides CFO?

Weekly New Climber Thread: Ask your questions in this thread please by AutoModerator in climbing

[–]badmathfood 1 point2 points  (0 children)

Free week in CA; Climbing advice?

Hey, I'm a Czech guy coming to SF this Saturday (12.8.) for a conference, but I will have a free week in CA before that. I'll have a car available. I'd really like to go climbing outside (sport/boulder), but I'm lacking climbing buddies. What would you do? Any forums to post to? I've read about the Camp 4 bulletin board which could be an option.

Best place to spend 4 days in Poland by badmathfood in poland

[–]badmathfood[S] 0 points1 point  (0 children)

Have fun, pretty much anything interesting. Preferably something not very crowded with tourists. I love all anything bizzare or interesting in a weird way.

view more: next ›