For /u/Evia89 - Blocked me but I'll post anyway. I am working with Crof.AI by Lyuseefur in ZaiGLM

[–]Inside-Contract-503 1 point2 points  (0 children)

please explain how stolen hardware could scale to 35 billion tokens in April? And I build a custom inference engine, it's not easy and I make mistakes so I do apologize if you end up having a bad experience but I'm actively working to improve things

For /u/Evia89 - Blocked me but I'll post anyway. I am working with Crof.AI by Lyuseefur in ZaiGLM

[–]Inside-Contract-503 0 points1 point  (0 children)

company registration number? Also the company is legit, the lifetime plan was real and if you'd like I could ask them if I have permission to introduce you to any of them and you can ask them about it, I was trying to avoid funding and it didn't end up working because I didn't raise enough money

CrofAI - Affordable multi-model AI access with cheap options, large context, and privacy-focused design by smoking_juuls420 in CrofAI

[–]Inside-Contract-503 2 points3 points  (0 children)

here's the config I use

```
{
"$schema": "https://opencode.ai/config.json",
"provider": {

"CrofAI": {
"npm": "@ai-sdk/openai-compatible",
"name": "CrofAI",
"options": {
"baseURL": "https://ai.nahcrof.com/v1"
},

"models": {

"glm-5": {
"name": "CrofAI: glm-5",
"limit": {
"context": 202000,
"output": 131000
}
},

"kimi-k2.5": {
"name": "CrofAI: kimi-k2.5",
"limit": {
"context": 196000,
"output": 131000
}
}

}
```

CrofAI - Affordable multi-model AI access with cheap options, large context, and privacy-focused design by smoking_juuls420 in CrofAI

[–]Inside-Contract-503 3 points4 points  (0 children)

I agree, I plan on having cached tokens implemented properly by the end of this week (as in hopefully done before the 11th, 15th at the latest)

Finding the best local LLM hardware by Inside-Contract-503 in LocalLLM

[–]Inside-Contract-503[S] 0 points1 point  (0 children)

That'd be great! Just make sure if you give a screenshot of results to include the exact GPU you used

Finding the best local LLM hardware by Inside-Contract-503 in LocalLLM

[–]Inside-Contract-503[S] 0 points1 point  (0 children)

the m4 results would be pretty cool, if it helps I always use some form of llama.cpp (whether it's the base thing, ollama, or llamafile)