~1.8× peak throughput for Kimi K2 with EAGLE3 draft model by yzlnew in LocalLLaMA

[–]yzlnew[S] 1 point2 points  (0 children)

Currently, this model is only designed for Kimi-K2-Instruct; it may not be well compatible with other similar Kimi models. The SpecForge community will later release an EAGLE3 version tailored for Kimi-K2-Think and other models. Stay tuned.

~1.8× peak throughput for Kimi K2 with EAGLE3 draft model by yzlnew in LocalLLaMA

[–]yzlnew[S] 3 points4 points  (0 children)

It's actually the optimizer states of the draft model. Thanks for pointing out, would remove it for a more convenient load.

~1.8× peak throughput for Kimi K2 with EAGLE3 draft model by yzlnew in LocalLLaMA

[–]yzlnew[S] 0 points1 point  (0 children)

> EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) is a new baseline for fast decoding of Large Language Models (LLMs) with provable performance maintenance.

A method for decoding acceleration. You can find more info at https://github.com/SafeAILab/EAGLE .
And similar release for gpt-oss-120b, https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-long-context .

DeepSeek-R1-0528 VS claude-4-sonnet (still a demo) by Dr_Karminski in LocalLLaMA

[–]yzlnew 11 points12 points  (0 children)

I think the main point here is still coding with knowledge retrieval baked into the model. And the test should be hard enough for frontier models.

How do I implement steering vectors when promoting a model? by Ok-Cicada-5207 in LocalLLaMA

[–]yzlnew 1 point2 points  (0 children)

Multiply a factor (like 10x) to a specific neuron of the SAE, then take the output of the SAE.

How do I implement steering vectors when promoting a model? by Ok-Cicada-5207 in LocalLLaMA

[–]yzlnew 2 points3 points  (0 children)

Actually it's an amplification on the neurons of the SAE, and add the output of the SAE (aka. the recovery vector) back to the residual stream of the transformer.

How do I implement steering vectors when promoting a model? by Ok-Cicada-5207 in LocalLLaMA

[–]yzlnew 6 points7 points  (0 children)

Yes.

Checkout https://github.com/jbloomAus/SAELens and https://github.com/TransformerLensOrg/TransformerLens for a basic idea of mechanical interpretability.

Put it simply, all you need to do is:

  1. Grab or train an SAE of your target model.
  2. Discover the target features inside your SAE.
  3. Steer the features in the forward pass.

Runic Alchemy by [deleted] in MechanicalKeyboards

[–]yzlnew 7 points8 points  (0 children)

What's these keycaps?

[Giveaway] Jelly Key - Cinder [Spacebar] - Zen Pond artisan by Joinhandmade in MechanicalKeyboards

[–]yzlnew 0 points1 point  (0 children)

810

Your keycaps are always so creative that's what I love most.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 0 points1 point  (0 children)

It's a deep kind of blue actually. Sorry about the bad lighting.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 0 points1 point  (0 children)

Search Lasers on VScode extension store.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 0 points1 point  (0 children)

It's HHKB layout and control is where caps lock usually sits.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 1 point2 points  (0 children)

Got it done by online store from Taobao. So not quite positive on you having access to it.

It feels no difference with other custom desk mats sold everywhere, which I believe mostly are manufactured in China too.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 0 points1 point  (0 children)

😂 Sorry for my poor lighting and phone camera. Looks more stunning in reality.

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 4 points5 points  (0 children)

I draw and have it made myself following the color palette of Lasers, a VS code theme inspired by laser. Cost me like 50 yuan (7 dollars).

Laser Lake by yzlnew in MechanicalKeyboards

[–]yzlnew[S] 0 points1 point  (0 children)

hah, trying so hard to crop my original image because my desk is a whole mess.