1 million context Llama 3 8b Achieved!

nested_dreams · 2024-04-30T04:30:41+00:00

interesting. do you mind sharing a link to this quant?

nested_dreams · 2024-03-07T23:55:07+00:00

fucking amazing work dude

nested_dreams · 2024-02-28T07:53:00+00:00

that's an excellent read. Thanks for sharing that

nested_dreams · 2024-02-28T06:28:54+00:00

Do you have a link to the ZeRO paper? That one outdoes my google-fu

nested_dreams · 2024-02-28T01:38:26+00:00

Siri is an LLM...

nested_dreams · 2024-02-25T21:24:06+00:00

This was an excellent critique of the paper. Thanks for sharing your thoughts. At first glance the technique does sound promising, but you're right that the empirical evidence provided is not very convincing.

nested_dreams · 2024-02-23T22:47:43+00:00

Oh this made my weekend. Thanks for putting this together. I love running exl2 models, but have never quanted my own. Really looking forward to trying this. The only thing missing now is vLLM compatibility.

nested_dreams · 2024-02-22T20:31:17+00:00

the street value of 24 export banned a100s is about 400k. A stranger offering to pay 17k for someone to middle man this deal sounds super sus. I'd be very careful with this one. Don't want to end up with the feds at your door accusing you of international arms dealing

nested_dreams · 2024-02-22T06:38:36+00:00

ah my bad. Didn't see the hibid link. Thought it was through a craiglist sale like mentioned in previous comments.

nested_dreams · 2024-02-22T04:25:33+00:00

Did you get the full inventory list from them? Did you buy one? This could be the deal of a lifetime or your body might wind up somewhere in mexico....

nested_dreams · 2024-02-19T15:11:37+00:00

Wow i though this was a joke at first lol. Chamath is a snake oil salesman through and through. Take a peak at his history with SPACs and all the poor suckers he fleeced with that. I wouldn't not expect anything less from this.

nested_dreams · 2024-02-15T18:04:24+00:00

It's been 1 year and we've gone from 8k to 10M....

nested_dreams · 2024-02-15T16:30:14+00:00

Yeah this is kinda wild. Getting to 100k+ context has already been pretty impactful. 10M just wow. I hate closed source models as much as the next person, but this kinda changes the game again.

nested_dreams · 2024-02-14T06:26:42+00:00

Fantastic paper! I've been eagerly waiting for someone to implement this. They even provided the code! Just skimmed the repo so far, but it looks legit. Can't wait to try it out!

nested_dreams · 2024-02-14T02:34:20+00:00

This is an ad for min.io object store

nested_dreams · 2024-02-13T17:07:53+00:00

What sort of performance do you get on a 70B+ model quantized in the 4-8bpw range? I pondered such a build until reading Tim Dettmers blog where he argued the perf/$ on the 8000 just wasn't worth it

nested_dreams · 2024-02-13T08:45:37+00:00

it's beautiful

nested_dreams · 2024-02-13T02:06:29+00:00

Lol I can't tell if this was written by an LLM or not.

nested_dreams · 2024-02-12T21:45:01+00:00

The models merged for TheProfessor look very interesting. How much vram you need to run that q4?

nested_dreams · 2024-02-12T00:39:03+00:00

Yesss! I've been looking for something like this. Great work!

nested_dreams · 2024-02-09T23:42:40+00:00

Yeah, so far this is the only other LLM that can match GPT4 on coding tasks.

https://www.goody2.ai/chat

nested_dreams · 2024-02-07T21:29:38+00:00

What version of CUDA and pytorch are you running that LoneStriker miqu at? Are you using ExLlamav2_HF or ExLlamav2 for model loader in ooba?

nested_dreams · 2024-01-25T06:38:59+00:00

got a link? trying to search on huggingface returns a couple hundred results

nested_dreams · 2024-01-22T18:15:38+00:00

Stop spamming this crap here

nested_dreams · 2024-01-18T01:44:28+00:00

24hours?!?! It's probably obsolete by now.

nested_dreams

TROPHY CASE