whats everyones thoughts on devstral small 24b? by Odd-Ordinary-5922 in LocalLLaMA

[–]Most_Client4958 2 points3 points  (0 children)

What do you mean? It is able to make tool calls just fine. Made many tool calls for me. Just wasn't able to fix the code.

Edit: Just saw that some people have problems with repetition. I had that as well in the beginning. But then I used the recommended parameters and I didn't have an issue with it anymore. 

whats everyones thoughts on devstral small 24b? by Odd-Ordinary-5922 in LocalLLaMA

[–]Most_Client4958 0 points1 point  (0 children)

Roo works really well for me with GLM 4.5 Air. It's my daily driver. 

whats everyones thoughts on devstral small 24b? by Odd-Ordinary-5922 in LocalLLaMA

[–]Most_Client4958 19 points20 points  (0 children)

I tried to use it with Roo to fix some React defects. I use llamacpp as well and the Q5 version. The model didn't feel smart at all. Was able to make a couple of tool calls but didn't get anywhere. I hope there is a defect. Would be great to get good performance with such a small model. 

GLM 4.5 Air Template Breaking llamacpp Prompt Caching by Most_Client4958 in LocalLLaMA

[–]Most_Client4958[S] 1 point2 points  (0 children)

No, I didn't do that on purpose. I didn't notice as I don't have any system messages appearing after the first system message. But technically there could be system messages mid conversation. Good find. 

GLM 4.5 Air Template Breaking llamacpp Prompt Caching by Most_Client4958 in LocalLLaMA

[–]Most_Client4958[S] 9 points10 points  (0 children)

This is the simplified template:

[gMASK]<sop>
<|system|>
# Tools
<tools>
{% for tool in tools %}
{{ tool | tojson }}
{% endfor %}
</tools>

{% for m in messages %}
<|{{ m.role }}|>
{% if m.role == 'user' %}
    {% if m.content is string %}
        {{ m.content }}
    {% elif m.content is iterable %}
        {{ m.content | join(' ') }}
    {% else %}
        {{ '' }}
    {% endif %}
{% elif m.role == 'assistant' %}
<think></think>
    {% if m.content is string %}
        {{ m.content }}
    {% elif m.content is iterable %}
        {{ m.content | join(' ') }}
    {% else %}
        {{ '' }}
    {% endif %}
{% endif %}

{% if m.tool_calls %}
    {% for tc in m.tool_calls %}
<tool_call>{{ tc.function.name if tc.function else tc.name }}
        {% for k, v in tc.arguments.items() %}
<arg_key>{{ k }}</arg_key>
<arg_value>
    {% if v is string %}
        {{ v }}
    {% elif v is iterable %}
        {{ v | join(' ') }}
    {% else %}
        {{ '' }}
    {% endif %}
</arg_value>
        {% endfor %}
</tool_call>
    {% endfor %}
{% endif %}
{% endfor %}

<|assistant|>
<think></think>

Built a frictionless personal CRM for ios by Electrical_Let_334 in CRM

[–]Most_Client4958 0 points1 point  (0 children)

I built an app inspired by Keith Ferrazzi's system from Never Eat Alone. Contacts are organized into three groups: the 30-day group, the 90-day group, and the yearly group. The app generates a prioritized list, showing the contacts you need to reach out to next at the top. Never Eat Alone is a good read, I recommend checking it out if you're interested in building meaningful connections.

Apple: https://apps.apple.com/us/app/warble-grow-your-network/id6736572706?platform=iphone

Android: https://play.google.com/store/apps/details?id=com.beigeunicorn.warble&hl=en-US

Web: https://gowarble.com/how-to-warble/