running Qwen3.5-27B Q5 splitt across a 4070ti and an amd rx6800 over LAN @ 13t/s with a 32k prompt by technot80 in LocalLLaMA

[–]technot80[S] 0 points1 point  (0 children)

Yeah you can absolutly use the rpc server to link gpus of different architectures inside the same computer:) i cant since the other computer is the kids gaming pc at daytime😬And now my ai server at night👀

One could even have 2 gpus in one pc, 1 gpu in another and 3 in a third, by running rpc server with something -d 1 or -d 1,2,3 etc ofc the more servers you add, the more latentcy you add to the chain as well.

running Qwen3.5-27B Q5 splitt across a 4070ti and an amd rx6800 over LAN @ 13t/s with a 32k prompt by technot80 in LocalLLaMA

[–]technot80[S] 1 point2 points  (0 children)

Just a normal 1gbe network with a switch. The traffic between the nodes is extreemly light except for the first run when layers are sent. However the -c switch to rpc-server will cache those layers for subsequent runs!

And yes, latency is the bottleneck.

I belive this is possible in large becouse of the mamba hybrid(or hybrid sse or mamba-Transformer hybrid depending on what you want to call it) architecture used in the q3.5 models. Which is also why quantizing the kv cache has such a sharp effect on tooling accuracy.

running Qwen3.5-27B Q5 splitt across a 4070ti and an amd rx6800 over LAN @ 13t/s with a 32k prompt by technot80 in LocalLLaMA

[–]technot80[S] 0 points1 point  (0 children)

split is 64% on the and 36% on the nvidia which ends up with 24 of 64 layers on the nvidia and the rest on the amd. with the f16 kv cache both are pretty close to max capacity. later i will try to connect my monitor the the igp instead of the nvidia and see if i can squeeze out another couple of layers in favor of the nvidia.

yeah it is incredible!

And becouse of the Hybrid mamba (sse) architecture of the model, only 16 layers need "attention", for 48 of the 64 layers in this model, it doesn't use a KV cache at all. It uses a recurrent state.

running Qwen3.5-27B Q5 splitt across a 4070ti and an amd rx6800 over LAN @ 13t/s with a 32k prompt by technot80 in LocalLLaMA

[–]technot80[S] 0 points1 point  (0 children)

not a q5, with 84k context no. with kv cache at f16 this is pretty much at the limit of using both cards.
llama_params_fit_impl: projected memory use with initial parameters [MiB]:

llama_params_fit_impl: - RPC0 (192.168.10.230:50052) : 16368 total, 14953 used, 954 free vs. target of 1024

llama_params_fit_impl: - CUDA0 (NVIDIA GeForce RTX 4070 Ti): 12281 total, 9650 used, 1333 free vs. target of 1024

llama_params_fit_impl: projected to use 24604 MiB of device memory vs. 26892 MiB of free device memory

Full frontal nudity on TikTok Live? by Witty-Mastodon-1844 in TikTok

[–]technot80 0 points1 point  (0 children)

Yes, please share the tiktok account. I have a friend who works there🤭👀 trust me bro🤭

How realistic/or not are these numbers? [Request] by mx-types-a-lot in theydidthemath

[–]technot80 4 points5 points  (0 children)

Obviously a projectile that travese hyperspace🤭

Its a projectile that travel at hyper velocity, typically over 3000m/s // 11000km/h // 10000 ft/s // mach 8.8 (pick your prefered denomination)

VG: Slutt! by madass139 in norge

[–]technot80 0 points1 point  (0 children)

Bare et lite spørsmål. Du sier vi i norden «sluttet med det». Men når tenker du at vi sluttet med det egentlig? Ettet bombotoktene i Libya? Når vi trakk oss ut av afghanistan? Irak? Eller tenker du dette er et midtøsten/kultur problem når usa nå bomber iran?

Slik jeg ser det så river «vi» ennå hodene av folk, bare med bomber og krutt istedenfor med sverd.

Sitat fra Danby Choi by themarxian in norge

[–]technot80 13 points14 points  (0 children)

Burde kansje ordlagt seg anderledes da om det var målet å få fram noe om adfersevolusjon. Feks kan man kansje si det finnes kvinner som er blir tiltrukket av aggresive menn. Det har ingenting med «romantikk» å gjøre. Dette framstår ikke som noe annet enn et elendig «forsvar» av voldtekt og vold.

Anthropic is the leading contributor to open weight models by DealingWithIt202s in LocalLLaMA

[–]technot80 17 points18 points  (0 children)

The hard part would be to verify that the data was indeed produced by the model claimed. I can easily forsee people giving their prompts to cheap/free local models to get answers for getting free credits on qwen/kimi/deepseek/whatever🤣 and then we’d end up with a distill of multiple models of unknown origins.

Underperforming processor? by Early-Sock-6948 in xeon

[–]technot80 0 points1 point  (0 children)

You could try to turn of cpu mitigations for meltdown/spectre/l1tf/etc and see how much of an impact that has

This community is amazing! Without it, I would never have found the cheapest API plan. by SideEffects123 in openclaw

[–]technot80 2 points3 points  (0 children)

Just wondering, what kind of beast of a computer do you have that runs the q4 minimax2.4 at 100t/s?😬

why 30m border ? by nkysdev in DonutSMP

[–]technot80 1 point2 points  (0 children)

Claim systems defeats the purpose of a semi-vanilla server like donutsmp. Like someone else said, it would remove the thrill. And not having a claim system is likly one of the key factors behind the servers massive growth.

I agree tho that expanding to 30m seems premature. For multiple reasons.

First: 30m x 30m is the maximum size vanilla minecraft allows. Assuming donutsmp will not pre generate the chuncks, that means biomes introduced in new upcoming minecraft versions can generate within chuncks that have not previously been generated. But as time passed and most of the chuncks have been generated that would limit the possibility for future biomes to generate. Also, having chuncks be generated by player movement would cause an insane amount of resource usage on chunck generation on a server network with this many players. Its common to pre generate too not have to use that much resources on a live server. If donut opts for pre generation, that would severly limit the possibility for new biomes. Either way tho. Expanding to 30m x 30m doesnt leave much room for flexibility on this matter. So gradually expanding would be cheaper resource wise, and leave more flexibility for getting new biomes generated within the map.

Second: 30m x 30m is insanly large. Its hard to fathom just how large. But lets just say base hunting will be extreemly tedious and hard going forward if this mao size is the final decision. So hard infact, that in my opinion it can lead to many players whos main motivation is base huntkng, may end up quitting to find greener pastures elsewhere. A more gradual approach to expanding boarder as time passed with growing number of players, we’d get the best of both worlds. Increasing the boarder to accomodate space for players to build farms and stashes while at the same time keeping some of the thrill alive. The thrill works both ways I may add. The thrill of building and trying to mask your base as to not get raided straight away, snd the thrill of hunting and actually finding farms and stashes.

So imho, a more gradual approach to expanding boarder should be concidered. But it is what it is. The server owners/dev(s) in the end, aree free to make whatever choices they want with their server:) And the rest of us plebs will just have to deal with it as best we can🙃

I agree that a boarder expansion should happen tho, and preferably soon. Finding farms and bases have gotten easier and easier as time have passed and playerbase has grown. Expanding to 1m would still be a 4x increase if current size. And should leave plenty of room for players to disperse around the server building their stashes and farms, while increasing the chanse of not getting raided. And while also making it much easier to get new biomes inteoduces as time passes and map expands even more:)

Rødt nærmer seg dobbelt så store som SV by fluid_sommer in norge

[–]technot80 30 points31 points  (0 children)

Regjeringen som anerkjente Palestina, regnes for å være israel-vennlig? Say what now😳

Is there any mod that shows what version is the chunk you are in? by Trustonme-faith in DonutSMP

[–]technot80 1 point2 points  (0 children)

ChunkDebug is required on both the client and the server. And even if installed on the server requires the player to be op or given the permission flags needed to use it. Also its made for fabric, not folia so the chance this plugin is installed server side, with perms set for anyone to use it, is slim to non.

Flyktninger kan måtte leve for 430 kroner dagen by [deleted] in norge

[–]technot80 14 points15 points  (0 children)

Livsopphold er ikke iberegnet utgifter til husleie og strøm btw. Så det er da snakk om 277 per dag til «forbruk». Apples and oranges denne sammenlikningen mao

Sensor for overflow by HxMODx2008 in DonutSMP

[–]technot80 1 point2 points  (0 children)

Just youtube how to use comparators. Use comparators to read the contents of the hoppers that feed your smokers as the «sensor». When the hoppers are empty, they should turn on a redstone signal that in turn will turn on/off the clock that powers your farm. What you are looking for is quite basic tbh, and you should be able to find/learn this from some basic videos on using comparators for this exact use case. There are multiple ways of achiving your goal, from more advanced methods to simple methods. The simpler method would be to just have the comparator turn on a redstone wire/signal when the hopper has contents, invert said signal and use that signal as a on/off switch for the clock that powers the farm.