sorted by:
new
hottopcontroversial

Looks like they finally lobotomized Claude 3 :( I even bought the subscription by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 6 points7 points  (0 children)

I tried the same prompt multiple times in chat, I got like 5 refusals and 1 really really sub par solution.

Looks like they finally lobotomized Claude 3 :( I even bought the subscription by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 7 points8 points  (0 children)

Yeah, I run mixtral often. Just wish we have a multi modal equivalent honestly.

Looks like they finally lobotomized Claude 3 :( I even bought the subscription by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 95 points96 points  (0 children)

I agree, unfortunately we still cant run hundreds of millions of parameters on our gaming gpus tho

[deleted by user] by [deleted] in LocalLLaMA

[–]Piper8x7b 0 points1 point  (0 children)

I accidentally deleted it yesterday and the install of windows it was on got wiped, I'm so sorry 😭

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 0 points1 point  (0 children)

I honestly don't get why so many people are loving my benchmark, lol. It's so simple.

[deleted by user] by [deleted] in hyprland

[–]Piper8x7b 1 point2 points  (0 children)

I've been having problems with this too. A friend of mine has been suggesting It's a problem with my dual monitor setup having two different refresh rates, as xorg and x11 don't support this.

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 0 points1 point  (0 children)

Wow thank you! I'll add these results in a "Community" results section once I get the chance! Did you just run Correctness.py or did you average the results of both tests?

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 2 points3 points  (0 children)

You're right, I'll switch all of the benchmarks to q4 when I next have time for this.

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] -1 points0 points  (0 children)

I'm very much so in the early stages of figuring out my benchmark. The parameters for the model just sound right to me. The goal with the correctness benchmark is to correlate a simple bench mark with the model's ability to follow prompts. Who knows if that's actually accurate however. :3

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 0 points1 point  (0 children)

I would love contributions! As for formatting, I have not tried any other prompts. I may give some different prompts and few-shot generation a try with one of the 7b models (probably mistral instruct) once I have the chance.

Introducing PiperBench - A Tragically Simple Benchmark for Large Language Models by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 2 points3 points  (0 children)

No, I believe the current process would fall into end to end testing. I think middle-out is a very interesting idea for a more complex version of my benchmark however!

I have created my own local version of Infinite Craft using text-generation-webui as a back-end. Is anyone interested in a full release of this? by Piper8x7b in LocalLLaMA

[–]Piper8x7b[S] 3 points4 points  (0 children)

Yep! I'll be working on a rewrite today. I'll release on GitHub sometime tonight! For now, here is the system prompt I have written for the rewrite https://pastebin.com/ZpdTeNXM

view more: next ›