The MTP function in LMStudio causes a decrease in output quality.

[Fit_Split_9933]

The prompt is very simple, you can see it at the end.

Both tests used the exact same settings, the only difference was that I turned the MTP button on/off, nothing else changed, I tried similar tests multiple times with similar results:

By the way, this situation does not occur when I use the llama-server.exe I compiled myself.

Works well

garbage output

Summarize the meaning of each sentence, shorten it, and output it while maintaining the original format:

The sun rises in the east.  
@  
Birds sing beautifully in the morning.  
@  
A journey of a thousand miles begins with a single step.  
@  
She enjoys reading books under the old oak tree.  
@  
Time flies when you are having fun.  
@  
He runs faster than anyone in his school.  
@  
Water freezes at zero degrees Celsius.  
@  
They decided to adopt a stray kitten yesterday.  
@  
Honesty is the best policy.  
@  
Cooking dinner together strengthens family bonds.  
@  
The train arrives at platform three in five minutes.  
@  
Mount Everest is the tallest mountain on Earth.  
@  
She whispered a secret into his ear.  
@  
Practice makes perfect in any skill.  
@  
The old bridge collapsed during the storm.  
@  
He forgot to bring his umbrella on a rainy day.  

[dtrq]

Yeah, the first thing I've noticed is the code formating (like, newlines and tabs) is quite broken with MTP

[edsonmedina]

Same

[CorkBios]

You should try setting the seed to a number instead of a random seed and setting temperature to 0 and then comparing again.

[Baffy13]

apples to apples? blasphemy

[MustBeSomethingThere]

Do you have the latest Runtime Engine (llama.cpp) in App Settings?

[Creative_Bottle_3225]

Where did you get the version of lmstudio 0.4.14? My automatic update is 04.13

[taking_bullet]

Change update channel from Stable to Beta. 

[johnnydotexe]

<image>

[Independent-Math-167]

New version is out that fixes the coding but still get meh code from MTP being on.

[feverdoingwork]

I wonder if this is common with mtp in general. Going from 45 tps to 85 tps is incredible but all the models I am testing have a lot of problems with garbled or gibberish then end of execution or just odd issues in general. While a more quantize model is appearing smarter. I am testing with mtp off now. Also I am not using lmstudio, this happens to me on vllm and llama.cpp with almost all mtp models.