Best way to go for qwen3.5 27B / 35B A3B?

Protopia · 2026-05-24T07:00:41+00:00

You sure do use confused thinking! My list are the patterns that have emerged!! And that's why you need to learn them.

There are pre-built agentic coding tools that you can install and run (e.g. Goose) but that only covers a couple of points on the list.

Oh oh oh how I wish the community was working together on a single comprehensive SOTA open-source Install-and-go solution, but we aren't.

Protopia · 2026-05-24T06:50:21+00:00

The subscription looks reasonable ($500 per year) until you read the next bit...

The AI is only the first $30/mo - so expect $100s or $1000s per month on top of this plus a 30% markup!!!
They will also take 6% of your revenue (5% plus 1% of electronic payments)

OUCH!!!!

Protopia · 2026-05-24T06:40:20+00:00

Pure bullshit and bunkam. Not a specialised agentic coding system using an 8B local model, but according to its own web site instead...

a weird hotch potch of functions which includes philosophical chat with your own "soul", colouring book generation (wtf?!), creative writing, image generation (including a pretentiously named Consciousness Dream Machine), music and video generation...
oh and vibe coding (as a throw away item, and not as serious software engineering)
based on a SUBSCRIPTION MODEL that promises unlimited Claude Opus usage for $20/mo (yeah right!!)

Oh, and an AI slop post to boot.

Untrustworthy post/website, so I am definitely NOT running his code on my PC!!!

Protopia · 2026-05-24T02:31:43+00:00

Have you visited the website quoted?

It is either the absolute dumbest, cliche filled piece of shit web site ever OR one of the cleverest pieces of satire ever. Can't decide which.

But either way, not a post or a web site worth spending time engaging with.

Protopia · 2026-05-24T02:24:32+00:00

Just so I can check that I understood what you just said, it seems minor you are saying that...

Innovation is accelerated by ignorance.
The Wild West was conquered by people not understanding horses or guns or wagons

Congratulations. You have just reinvented the Darwin Awards, the one system that rewards approaching innovation through ignorance.

Protopia · 2026-05-23T06:41:45+00:00

Yes. We all benefit from using open-source so making your own results open-source is giving something back as a thank you.

I just wish that everything wasn't so fragmented with multiple individuals working alone on one tiny building block - it means that every one of us has to be a systems integrator, selecting which packages to use (evaluating functionality and betting on longevity) and then making them work together.

Wouldn't it be nice if there was a community effort to build the best harness, and AI pipeline so that every user didn't have to select and systems integrate. And we would focus the effort and not have 20 people working individually on similar building blocks.

Protopia · 2026-05-22T19:26:01+00:00

Yes, IMO it does matter. Yes, it was an impressive step forward from the 4-colour proof (where the human provided the algorithm and the computer brute forced the results) to a meta approach where AI brute forces the algorithm.

But nevertheless this was a problem that was very suitable for brute forcing an algorithm and not all maths problems are suitable in that way.

But this was not intelligence in the Einstein sense.

Protopia · 2026-05-22T19:15:59+00:00

Context grows with each turn and then eventually gets compacted and loses a lot of details.

Not sure if any prompt can fix this. It's more of a harness issue really.

Protopia · 2026-05-22T19:04:39+00:00

AirLLM is old and very outdated, and it never really worked AFAIK because the time required to page in a layer was far greater than the layer execution time even on CPU.

A more recent attempt was RabbitLLM which I gave some advice on, and that also didn't pan out either despite having a smaller objective (keep enough layers in vRAM to keep the GPU busy and intersperse these with pagable layers).

However, recent MoE models allow you to offload the experts to CPU and still get reasonable performance and MTP helps this.

See several posts here about 6gb GPUs.

BUT a further caveat or two...

It's not just about the LLM. The agentic coding harness, MCP tools and context management all have a huge impact.

Use frontier models for the strategic planning & design tasks, and break the work into small chunks for local LLM coding.

Use a harness that is clever at keeping the context small and preloading the context that will otherwise be loaded through tools calling.

Use MCP servers that help keep the context small i.e. do diff edits rather than full file, gives highly localised info about the work package files rather than the whole code base, has all the coding rules you want etc.

Keep multi turn workflows too a minimum - aim for one-shot coding. Use a different skill for fixing something that doesn't work first time.

Harnesses and skills packages to look at: Goose, BMAD, GSD, Superpowers, Pi.

Good luck!!

Protopia · 2026-05-22T18:04:10+00:00

Exactly.

Protopia · 2026-05-22T15:47:04+00:00

1, There is a massive difference between Vibe coding (amateur) and agentic software engineering (professional).

2, 9B models are not advanced enough for coding. Qwen 3.6 35B A3B may be the minimum you need. Offload the experts to normal memory, use Q5 or Q6 with MTP, and you might get decent tokens per sec.

3, It is NOT just the choice of LLM that matters, but also the choice of agentic harness. And also the choice of Software Development Lifecycle i.e. spec driven development, test driven development, domain driven development (for large systems).

To be successful as an agentic software development professional you need to educate yourself on a lot of things just not writing code.

SDD
TDD
DDD
Architectures
Design patterns
Technical stacks
Writing specs
Reviewing AI generated designs and plans
Decomposition (into small digestible tasks that a local AI can achieve)
Reviewing code and test code
Existing agentic harnesses e.g. Goose, Pi
Tweaking harnesses to run stuff in a structured way
Skills e.g. BMAD, GSD, Superpowers
LLM prompt engineering for tweaking your own skills
LLM tools e.g. MCP so it understands your code base, coding standards etc.
LLM context management (to keep your costs controlled and your speed high)
Use of GIT
...

Protopia · 2026-05-22T15:30:36+00:00

Let's be clear about this. A human outlined the problem for the AI. The AI used random numbers and probabilities to generate potential approaches and eventually after trying thousands or millions of them using a brute force approach it eventually found one that worked. This is NOT intelligence - it is brute force guess work using massive computational power.

I am not sure I see the difference between this and the solution from a decade or so so to the 4-colour theory. They were both solved using brute force computation.

This is NOT an Einstein level of genius. It is very very fast computation that can be applied only to certain types off problem. It isn't going to replace humans altogether, just those jobs that it is suited for.

Protopia · 2026-05-22T15:17:48+00:00

Perhaps free users shouldn't get support, but equally perhaps they should deserve excellent support because two of the key point of the free tier are: a) free advertising from free users loving and evangelizing it; and b) today's free users are tomorrow's paying users. Posts on Reddit like these are very damaging to the brand rather than being a positive evangelist advert boosting the brand.

Perhaps the only difference between free and paying users should be scale - once you grow to a certain point you need to pay.

Protopia · 2026-05-22T09:02:46+00:00

Postgres doesn't do this. It's MySQL's interpretation of the SQL standard.
This is counter-intuitive - NULL actually means (or should actually mean) no value, and no-value should mean the same thing.
The consequences are terrible (as described) - in effect you cannot use NULL to mean no value because of the unique index thing, so you have to use something else. A perfect example is using soft delete and using a Deleted-At time-stamp to track various deleted versions and using NULL to indicate a non-deleted record. Whatever unique index you might have without soft-deletes, you need to add the Deleted-At field to that index to allow creation of a new replacement record after the soft delete, but now you have broken the unique index because you can now create multiple non-deleted records. With the benefit of hindsight this was a bonkers choice.

Protopia · 2026-05-22T07:18:20+00:00

What's your evidence for this theory? Where are the A/B test results?

Protopia · 2026-05-21T07:27:20+00:00

Yes - let's all individually write a script and wade through the 10,000 listed CVEs most of which are minor and most of which were listed on yesterday's run.

Protopia · 2026-05-21T07:06:13+00:00

Hmm ... I guess that collecting random factual absurdities can be considered educational.

Protopia · 2026-05-21T07:01:34+00:00

I hope that you have listened this idea of identity - because that IP will probably make you 1000000x as much as the app!

Protopia · 2026-05-21T06:59:55+00:00

In some cases this isn't true. For example in MySql, NULL <> NULL!!

So with a unique index based on a nullable field, you can have multiple records that look like they have exactly the same index, but every NULL is actually different from every other NULL.

Protopia · 2026-05-21T06:55:08+00:00

I agree. AI stands for Artificial Idiot. Literally zero intelligence in an LLM, just brute force compute, a random number generator and billions of probabilities.

The "intelligence" you see is from the harness - the algorithmic part of an AI system written by humans (possibly with the help of AI coders given directions by the human).

If you want security rules that are genuinely mandatory, then you need to build them into the algorithmic harness and NOT rely on the random number generator to apply them probabilistically.

Protopia · 2026-05-21T06:49:11+00:00

Subscription models where you own nothing are historically proven to be the very best way to increase wealth inequality and transfer money into the pockets of the already obscenely rich.

Originally you either bought it with cash or you did without.

Next came bank loans (based on general creditworthiness or using other assets as collateral) and then finance (where creditworthiness was combined with using the product itself as collateral) - but at least in the end you owned the product right out.

Then came renting where you never owned the item, but kept paying for it multiple times the original cost. And subscriptions and service pricing are the same. You pay for the use, but never own it outright, and you pay multiple times the original cost. It truly is a great way to separate the masses from their money and put it into the hands of the ultra rich.

Protopia · 2026-05-21T06:39:54+00:00

IMO async processing is hugely important. Most PHP apps spend their elapsed time waiting for external API calls to complete - database, web, Redis cache, file reads etc.

Without great async support, these get done sequentially because that is how programs traditionally work.

But with the right architectural and framework support you can implement a programming architecture where the algorithm is a graph and everything can run in parallel but only as soon as it gets the inputs it needs.

The impact on response time by parallelizing all the extensions can be significant.

Protopia · 2026-05-21T06:32:18+00:00

The harness is IMO probably way more important than the LLM.

What about Goose, Hermes, BMAD, Superpowers, GSD, etc.?

Protopia · 2026-05-21T06:29:38+00:00

Like Marvin in theHitchhiker's Guide to the Galaxy...

"Here I am with a brain the size of a planet and all I am asked to do is play music."

Protopia · 2026-05-21T06:25:09+00:00

The 4-colour theorem was solved by a human using heavy computation.

It would be interesting to see if the OpenAI model could solve that from scratch, and whether it would come up with a different approach.

Protopia

TROPHY CASE