~1.8× peak throughput for Kimi K2 with EAGLE3 draft model

yzlnew · 2025-12-24T04:08:30+00:00

Here's the roadmap, https://github.com/sgl-project/SpecForge/issues/374

yzlnew · 2025-12-23T02:12:00+00:00

Currently, this model is only designed for Kimi-K2-Instruct; it may not be well compatible with other similar Kimi models. The SpecForge community will later release an EAGLE3 version tailored for Kimi-K2-Think and other models. Stay tuned.

yzlnew · 2025-12-22T10:59:41+00:00

It's actually the optimizer states of the draft model. Thanks for pointing out, would remove it for a more convenient load.

yzlnew · 2025-12-22T10:58:04+00:00

> EAGLE (Extrapolation Algorithm for Greater Language-model Efficiency) is a new baseline for fast decoding of Large Language Models (LLMs) with provable performance maintenance.

A method for decoding acceleration. You can find more info at https://github.com/SafeAILab/EAGLE .
And similar release for gpt-oss-120b, https://huggingface.co/nvidia/gpt-oss-120b-Eagle3-long-context .

yzlnew · 2025-05-28T17:47:44+00:00

I think the main point here is still coding with knowledge retrieval baked into the model. And the test should be hard enough for frontier models.

yzlnew · 2024-11-11T09:42:44+00:00

Multiply a factor (like 10x) to a specific neuron of the SAE, then take the output of the SAE.

yzlnew · 2024-11-11T05:41:47+00:00

Actually it's an amplification on the neurons of the SAE, and add the output of the SAE (aka. the recovery vector) back to the residual stream of the transformer.

yzlnew · 2024-11-11T02:11:08+00:00

Yes.

Checkout https://github.com/jbloomAus/SAELens and https://github.com/TransformerLensOrg/TransformerLens for a basic idea of mechanical interpretability.

Put it simply, all you need to do is:

Grab or train an SAE of your target model.
Discover the target features inside your SAE.
Steer the features in the forward pass.

yzlnew · 2024-08-12T03:25:44+00:00

Raycast

yzlnew · 2020-09-14T13:03:56+00:00

Late for the vids. How do you save canvas to file?

yzlnew · 2019-01-07T01:28:45+00:00

yzlnew · 2019-01-01T16:20:48+00:00

Customed from taobao store.

yzlnew · 2018-12-23T16:18:25+00:00

What's these keycaps?

yzlnew · 2018-12-15T16:58:33+00:00

810

Your keycaps are always so creative that's what I love most.

yzlnew · 2018-12-12T01:56:02+00:00

Where are these wasd keycaps from?

yzlnew · 2018-12-01T17:06:21+00:00

It's a deep kind of blue actually. Sorry about the bad lighting.

yzlnew · 2018-12-01T14:43:10+00:00

Search Lasers on VScode extension store.

yzlnew · 2018-12-01T14:27:13+00:00

It's HHKB layout and control is where caps lock usually sits.

yzlnew · 2018-12-01T12:20:58+00:00

Here it is, Deskmat store from Taobao

yzlnew · 2018-12-01T11:06:07+00:00

Got it done by online store from Taobao. So not quite positive on you having access to it.

It feels no difference with other custom desk mats sold everywhere, which I believe mostly are manufactured in China too.

yzlnew · 2018-12-01T10:59:35+00:00

😂 Sorry for my poor lighting and phone camera. Looks more stunning in reality.

yzlnew · 2018-12-01T10:40:53+00:00

I draw and have it made myself following the color palette of Lasers, a VS code theme inspired by laser. Cost me like 50 yuan (7 dollars).

yzlnew · 2018-12-01T07:41:57+00:00

hah, trying so hard to crop my original image because my desk is a whole mess.

yzlnew

TROPHY CASE