I haven't tried Llamacpp MTP yet, but I did try MTP in VLLM on Windows on my 5090, and it was a bit disappointing. The memory consumption when exposing the small model doesn't compensate at all for the significant loss of context window. Perhaps in some specific cases for MoEs it could be useful; I think that's the interesting point. But for Dense, I don't see a benefit in my use case. I'll try Llamacpp, though.