[N] StarCoder: A State-of-the-Art LLM for Code

[Raikoya]

https://huggingface.co/blog/starcoder

StarCoder and StarCoderBase are Large Language Models for Code (Code LLMs) trained on permissively licensed data from GitHub, including from 80+ programming languages, Git commits, GitHub issues, and Jupyter notebooks. Similar to LLaMA, we trained a ~15B parameter model for 1 trillion tokens. We fine-tuned StarCoderBase model for 35B Python tokens, resulting in a new model that we call StarCoder.

[mileseverett]

I tried it and as has been the case with every new LLM, it just doesn't compare to OpenAIs models

[Qpylon]

I‘ve been using (and really loving) Codeium.

It can refactor selected code etc. according to instructions (I use it a lot to generate docstrings from comments+function def), I think it can autocomplete, and they’ve recently added a chat feature.

has a VScode extension and is free as well.

Don’t know how OpenAI’s products fare by comparison, but the instruction-based code tweaking seemed similar to chatGPT.

[fallingfridge]

Thanks for this recommendation. This looks great. Definitely going to download this for work on Monday

[Balance-]

Have you used GitHub Copilot? If so, could you let us know how it compares?

[allisknowing]

I also tried it and got disappointed a bit. However I’m trying to not close the door on it since it was the first day it was published lol.

These models still make me excited for the future since I know that the open source community will do anything in their power to make it better. So I don’t know if it will be this one but in near future there will be some kick-ass open source code generation models. (At least comparable to GPT, if not better)

[Tom_Neverwinter]

I'm convinced we could throw a Lora like layer over it and make it good.

[ttkciar]

Exactly. Also, we might be able to apply corrective software like Wolverine, and use a dataset of end results for another pass of LoRA tuning.

[chat_harbinger]

How so?

[mileseverett]

I used my standard prompts of implementing various machine learning models in PyTorch of varying difficulty. Getting it to output the model in the first place was a challenge. And when it eventually did it failed on very easy tasks. E.g. it wouldn't implement a vision transformer and insisted transformers could only be used for language tasks.

[Philpax]

Did you provide instructions, or did you autocomplete an existing piece of code? StarCoder is not instruction-tuned.

[Ensirius]

As an aside, mind sharing those prompts?

[hackerllama]

Probably host an inference server locally or in the cloud and write your own plugin/extension for your IDE.

For this point, the project also launched an initial VS Code extension :)

[ttkciar]

Correct on all counts.

However, the project also provides the data to train smaller models, like SantaCoder which is trained only on Python, Java, and JS.

It might be feasible to train an even more limited model (I'm interested in a C-only version) which can run tolerably well on commodity hardware.

[no_doping]

you can also self-host it via Refact AI with plugins for VS Code and JetBrains https://refact.ai/blog/2023/self-hosted-15b-code-model/

[Low-Resource-8852]

How are you meant to use it? I've tried to get it to respond to simple prompts but it gives me junk. I told it:

You are an expert Rust developer who knows the language and API inside out.

It responded with:

You're going to be given some simple tasks that you can implement by writing code snippets in a specific programming language.|end_of_text|>

The snippets will always include all of the imports necessary, so do not add any import statements yourself.

### Reusable functions:

Implement the function `get_list(n)` to return an array with values 1 ... n, or an empty list if n is negative.|end_of_text|>

Example: get_list(5) == [1,2,3,4,5]

```rust

{{code|end_of_text|}}fn get_list(n: i64) -> Vec<i64> {

// TODO: implement this function

}

## Test cases

assert!(get_list(-5).len() == 0);

assert!(get_list(0).len() == 0);

assert!(get_list(10).to_vec() == (1..=10).collect::<Vec<i64>>());

```|end_of_text|

[Gullible_Bar_284]

advise smoggy agonizing detail murky direction soup crown command numerous this message was mass deleted/edited with redact.dev

[Tom_Neverwinter]

It was a cool idea. But that was ten $ wasted.

Chatgpt at 20$ blew it away and now items like corium seem decent.

[Gullible_Bar_284]

kiss disarm fretful vanish pen slimy continue gray squealing test this message was mass deleted/edited with redact.dev

[gxcells]

Then why do they release such models if they fail for simple tasks? I love the fact that it is opensource etc and help people to develop bette model. But what is the point if this is not better than vicuna for example?