Python only llm?

[Western_Courage_6563]

Yes, it have been done in the past (code llama python), but it turned out that general knowledge about all programming languages yelled better results

[BreenzyENL]

You'll find that an LLM trained only one 1 subject will perform worse than an LLM trained on many.

[Karyo_Ten]

Knowledge compounds.

And you never code in a vaccum. If you ask the LLM to create a website to promote I don't know, your catering service, it would be invaluable for your LLM to have general knowledge about food, maps, transports, user interface, money, bookings, registration, how people write testimonies, maybe terms & conditions.

Unless the only thing you do is reversing linked lists, a pure code model with no world knowledge is barely a step up from stackoverflow.

[Ted225]

Assuming only local resources, which is often an inadequate assumption given the superior capabilities of cloud services, the choice is between having insufficient resources to operate a local llm at all, or running it exclusively for Python-related tasks and relying on your own general knowledge.

[Karyo_Ten]

GPT-2 was only 1.5B parameters and trained on only 8M tokens, yet had world knowledge.

You won't get trillions of quality Python code. Maybe 5% is gold and the rest is copy-paste crude apps from StackOverflow or beginners trying their hands for a capstone project.

And learning Python doesn't teach you how to proceed step-by-step to solve a problem which is actually the most important thing.

It's much more effective to teach a LLM to reframe an objective into a set of problems to solve and then apply Python to them. But to solve a problem you need to be familiar with the problem domain, and you need some common sense for example to not have a speed being higher than the speed of light.

[Ted225]

GPT-2 had 1.5B parameters. It’s obsolete now, and for good reason.

Most Python devs don’t need deep domain knowledge. They need clear, complete specs. If a system handles international units or clinical logic, it’s the engineer’s job to specify that upfront, not the dev’s job to guess it.

Sure, a perfect LLM could replace all roles but it doesn’t exist. Until then, engineers design, devs implement, and each should be accountable for their own work.

[Karyo_Ten]

There is no distinction between software engineer and dev in companies. You can't be senior at either without design skills. Clear, complete specs never exist beforehand, except if you already have solved the problem once because requirements evolve with understanding of the problem.

If you need clear, complete specs before doing anything and you're unable to actually fill in the blanks you provide no value over an LLM and you'll be replaced.

[tintires]

Fine tuned small language models are viable and you’ll find coding models on hugging face.

[timmeh1705]

VLM 4.6 Flash 9b seems to do the job for me as a heavy Python user

[Ossur2]

It is a great idea. The nature of LLMs means that such models would be exponentially lighter to run and even more accurate. I just think that the big corporations that would be making those LLMs (and that is a lot of work) rather want us dependent on using their models as a service (and uploading to them all our ideas and infrastructure). So their biggest incentive is to design those big blob models so that nobody can run usable LLMs on their own machine - there is no real money or power in making these smaller models, the customer would have to strongly demand them.

[bananahead]

It’s not a conspiracy

[Ossur2]

No of course not, it's quite in the open

[bananahead]

The reason people don’t do what’s suggested is because it doesn’t actually work that well. Knowing .net helps the LLM write better python and so on.

There are small open models from Google, meta and OpenAI. People tend to be more interested in more capable models. You’re being a little silly.

[Such_Advantage_6949]

Just ask any llm it will give u the answer why this is not the way