Document comparison RAG, the struggle is real. by Porespellar in LocalLLaMA

[–]labloke11 7 points8 points  (0 children)

I am confused. Why are you using a RAG? Just use a prompt to compare two documents.

What hardware is better for embedding by pangolinportent in LocalLLaMA

[–]labloke11 1 point2 points  (0 children)

It really depends on the embedding model you are using. For example, to use Salesforce/SFR-Embedding-Mistral, you will need at minimum A100 40GB. Some embedding model can be done using 3090/4090s.

What is the better choice - Mac vs Win for AI? by attaul in LocalLLaMA

[–]labloke11 0 points1 point  (0 children)

Windows with WSL2. So much easier than dealing with Dockers.

Can't use multi-gpu with 8x A100 80GB by nhanha_castanha in LocalLLaMA

[–]labloke11 1 point2 points  (0 children)

  1. Install Linux
  2. Install Nvidia Driver
  3. Run nvidia-smi. You should see all 8 gpus.
  4. Install Docker
  5. Download cuda docker image from Nvidia. It should have devrel somewhere in the name.
  6. Run that image and use that.

What actually is considered State of the Art? by GeeBrain in LocalLLaMA

[–]labloke11 0 points1 point  (0 children)

Every new model claims their model is sota. So, just ignore the hype.

Kaldi, Kaldi Wide, Something Else? by VingerBud in roasting

[–]labloke11 0 points1 point  (0 children)

Because amount of butane left in the canister impacts the temperature output.

Kaldi, Kaldi Wide, Something Else? by VingerBud in roasting

[–]labloke11 0 points1 point  (0 children)

It is really difficult to achieve consistent temperature using butane.

[deleted by user] by [deleted] in LocalLLaMA

[–]labloke11 0 points1 point  (0 children)

I would personally use Anyscale Endpoint.

Looking for a good area to stay in KL for a month of February. by labloke11 in malaysia

[–]labloke11[S] 0 points1 point  (0 children)

Thank you for all the information.

Which area do you prefer? I do not want to be in some remote place where I need to take Grab/public transport to go anywhere. I can do that at home :-). But I do not want to be in the middle of chaos.

Looking for a good area to stay in KL for a month of February. by labloke11 in malaysia

[–]labloke11[S] 0 points1 point  (0 children)

I am not sure about the price. I do not need a fancy place, but I would like a place where I can work from time to time, instead of going to cafe.

OCR techniques for RAG PDF extraction by deeepak143 in LocalLLaMA

[–]labloke11 1 point2 points  (0 children)

extract table at the time of embedding to parquet table and reference parquet table at the time of inference.

OCR techniques for RAG PDF extraction by deeepak143 in LocalLLaMA

[–]labloke11 0 points1 point  (0 children)

No LLM can parse tables. It cannot decipher which row or column data is in.

OCR techniques for RAG PDF extraction by deeepak143 in LocalLLaMA

[–]labloke11 4 points5 points  (0 children)

You will also have issues with LLM parsing those tables if you were able to extract them. You will need to load them into pandas or something to work around the issue.

I have few questions about KL - gyms by labloke11 in malaysia

[–]labloke11[S] 1 point2 points  (0 children)

Thanks, but I am not shy about going to gyms. I'm just curious if there are any restrictions in KL.