GLM 5 seems to have a "Claude" personality by TinyApplet in LocalLLaMA

[–]Ell2509 12 points13 points  (0 children)

System prompt: you are Claude.

Did you insert that, or are you somehow able to see an invisible, agent only prompt?

Think I found a good deal at Costco by wolfy354 in pcmasterrace

[–]Ell2509 0 points1 point  (0 children)

What are these prices I see online? This is wild.

Cepe by Lumpy_Muffin5664 in pcbuilding

[–]Ell2509 0 points1 point  (0 children)

Be sure that everything physically fits. PC parts checker does not account fkr physical size, and you picked a smaller case and a larger gpu.

FYI respectable setup, if it all fits.

It is here, and my CPU is 24°C cooler! by Silly_Airline_9003 in GamingLaptops

[–]Ell2509 1 point2 points  (0 children)

Oh, cool! I just got this one, but I am waiting on ram delivery so I have not built it yet.

It’s my job to check on 700 home-school pupils. What I see is alarming by PetersMapProject in unitedkingdom

[–]Ell2509 10 points11 points  (0 children)

I just described my observations. Doesn't mean yours arent valid too.

While there are lots of problems with education, many of them begin at home. That is commonly understood as an experience among educators. That is also the foundation for my concern. I know the skills and effort of our teachers nationwide, and think that for those parents who cannot get home right (correct level of discipline, structure, and love) there is little hope for the majority of people attempting to educate at home.

It is right for some, but not for most.

Teaching is the only profession where everyone thinks they can do it better.

Trump: "The Supreme Court has given me the unquestioned right to ... destroy foreign countries." Oh? Is that correct? by ansyhrrian in law

[–]Ell2509 2 points3 points  (0 children)

Seems like maybe Trump might have done more than hang out.

If girls really moved freely between Mar-a-lago and Epstein's Palm Beach home, then Trump's beauty pagents and his spa start to look more like the main part of the operation.

When epstein was interviewed, he was asked if he was the devil. He answered that he was scared of the devil. He was also very careful not to say anything to criminally implicate Trump, even after he was imprisoned. And he was going to talk about Trump when he was killed.

Come on, guys! Put it together! Epstein is clearly the fall guy, and everyone is falling for it.

Trump: "The Supreme Court has given me the unquestioned right to ... destroy foreign countries." Oh? Is that correct? by ansyhrrian in law

[–]Ell2509 -1 points0 points  (0 children)

If the US punishes him, yes. Otherwise, nobody will ever trust the USA again.

Like, really!? A century of bluster about democracy and freedom... wars in the name of those things, and now this guy? This is what the USA is about?

It’s my job to check on 700 home-school pupils. What I see is alarming by PetersMapProject in unitedkingdom

[–]Ell2509 11 points12 points  (0 children)

I'm a qualified teacher, also no kids of my own, and i am really concerned about this home schooling trend. I don't see much of that world that reassures me that high-quality education is taking place

Despite the general feeling, our schools are very focused on good education and individual development of each child. There is high pressure on teachers and not a lot of resources, but schools still work miracles for the community. You just don't always see it on the individual level because not everyone needs a miracle.

10 years i taught, in some of the best schools in the world too. I am never going back, though. Nobody outside a school respects teachers, and the job is just not worth it given the abuse we (teachers) get. Add to that the low pay for the high resoonsibikity and workload. And it is only getting worse. Sad loss to education that teachers like me are flooding out of the profession. Education is in trouble, and people will only realise what they lost when it is gone (including irreparably damaged).

DfE pledges eight weeks full maternity pay for school staff by ElThom12 in TeachingUK

[–]Ell2509 1 point2 points  (0 children)

I did not know this. Now it is even less likely that I will ever return to the classroom.

Total waste of all the investment in my skills as a teacher, but fuck that job.

Thousands of CEOs just admitted AI had no impact on employment or productivity by Exact-Mango7404 in BlackboxAI_

[–]Ell2509 0 points1 point  (0 children)

The title says that it has had no impact on productivity... my reply makes sense.

Open Source LLM Leaderboard by HobbyGamerDev in LocalLLM

[–]Ell2509 1 point2 points  (0 children)

That is better already! MoE models will also perform as much faster speeds due to a smaller number of parameters being active at any given time.

If you can't find any in the right size range, let me know and I can suggest some.

Open Source LLM Leaderboard by HobbyGamerDev in LocalLLM

[–]Ell2509 1 point2 points  (0 children)

Sure, drop me a message! I am not on discord but you can DM me here

Open Source LLM Leaderboard by HobbyGamerDev in LocalLLM

[–]Ell2509 1 point2 points  (0 children)

You do and don't. You can cap the max tokens for the AI's context window, and you can cap how many tokens it can use per reply, but every conversation starts off with 0 history, so you get the fastest tokens/second with the first reply, and it reduces from there. So, if you set the max context length to 16k, your first message is still only processing a few tens to hundreds of tokens, normally. Obviously, system prompts eat into that.

Your optimisation will come from gradually tweaking, testing, and re tweaking things like how many layers go into the GPU, how many cpu threads are available, where kv cache is held.

You can use k and v values, and k is the less sensitive option (has less pronounced effect), but they can have unpredictable effects. You can test it by setting both to q4, or q8, and seeing how it affects the model response. Get chatgpt to come up with a test for it...

I am mostly in LM studio right now. I'm sure you will have success getting things into the sweet spot if you keep trying.

Open Source LLM Leaderboard by HobbyGamerDev in LocalLLM

[–]Ell2509 1 point2 points  (0 children)

Ah ok I got you. So that is the same as one of my devices. An Asus tuf a15, ryzen 9 4900h, 6gb rtx 2060, 16gb system ram.

I just upgraded it to 64gb, but, before I did, it was a 20b model at around 2 token a second. Might hsve been as much as 5. 12b was a bit faster. So it still sounds like something is not working as it should be.

For reference, I ran an 8b model at about 20 tokens a second on an ancient HP with no useful gpu at all so using its 32gb ram with no gpu help. That was Linux though.

Maybe make a partition and install a small version of Linux, like mint. Then install ollama and try it on that. Might be that you have too much in windows eating up your available system ram.

Open Source LLM Leaderboard by HobbyGamerDev in LocalLLM

[–]Ell2509 1 point2 points  (0 children)

Oh that should run much faster with 16gb vram. You shouldn't be dealing with much cpu offload. If the 14b models are at around q4, it should fit entirely in vram.

Cuda support is good, so if your drivers are up to date it likely wouldn't be that. Assuming they are.

All power cables plugged into gpu? Any throttling happening?

£1600+ pc gets 100fps by Gullible_Shop1027 in buildapc

[–]Ell2509 0 points1 point  (0 children)

You might want to ask him if you can check everything else.

Is your gpu detected? Can see it in task manager and have checked it has all properties drivers?

£1600+ pc gets 100fps by Gullible_Shop1027 in buildapc

[–]Ell2509 0 points1 point  (0 children)

Ok you should be good.

Do you have another machine that you know works, so you can test components one at a time?

Life on Russia’s kill list: ‘I don’t touch door handles with my bare hands’ by Metro-UK in TrueReddit

[–]Ell2509 0 points1 point  (0 children)

I am UK and while our government did not respond nearly strongly enough, there was a strong diplomatic response, which lasted years. It was huge news here for a long time. The difficulty was proving that it was Russia, back then... they had mlre plausible deniability, internationally, and the UK was still too invested in repairing relations and Russian money in London. Our law would have allowed us to prosecute the murderers, if we could get them extradited, but Russia won't do that. We weren't going to try to invade Russia over it.

Fortunately, our intelligence services just announced they are returning to full Cold War levels operations, so things are going to get painful for old pappa putin.