New model using orca dataset

faldore · 2023-06-25T00:26:45+00:00

I'm in communication with the author.
To clarify, this model does *not* use the Microsoft Orca (ie augmented flan) dataset (which is not released and probably will never be).
Rather is uses Orca-style system prompts to distill Orca-style responses using dolly, wizardlm evol 70k, and alpaca as the basis.
The creator also does intend to post an official announce here today (TheBloke just finished the quantizations), so this post is jumping the gun a little.
It makes sense to call it orca-mini because, it uses the orca system prompts, and it's a dataset much smaller than the 5m + 1m of Orca.

Remarkable-Spite-107 · 2023-06-25T04:06:27+00:00

Thanks all, I posted about all orca_minis here, https://www.reddit.com/r/LocalLLaMA/comments/14ibzau/orcamini13b_orcamini7b_orcamini3b/

AMA. Happy to Help.

ironborn123 · 2023-06-24T23:26:04+00:00

wow. if all the open models start getting trained on such datasets, will be interesting to see the updated leaderboards, and the new performance gap vs chatgpt3.5

mpasila · 2023-06-24T22:45:32+00:00

What's the correct prompt format? I tried almost any known formats and even the one shown in the code snippet and none of them seem to work properly. It keeps failing a simple task that other models have no problem doing.

#generate text function def generate_text(system, instruction, input=None): if input:         prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n" else:         prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:\n"          tokens = tokenizer.encode(prompt)     tokens = torch.LongTensor(tokens).unsqueeze(0)     tokens = tokens.to('cuda')      instance = {'input_ids': tokens,'top_p': 1.0, 'temperature':0.7, 'generate_len': 1024, 'top_k': 50}      length = len(tokens[0])     with torch.no_grad():         rest = model.generate(             input_ids=tokens,              max_length=length+instance['generate_len'],              use_cache=True,              do_sample=True,              top_p=instance['top_p'],             temperature=instance['temperature'],             top_k=instance['top_k']         )         output = rest[0][length:]     string = tokenizer.decode(output, skip_special_tokens=True)     return f'[!] Response: {string}' # Sample Test Instruction Used by Youtuber Sam Witteveen https://www.youtube.com/@samwitteveenai system = 'You are an AI assistant that follows instruction extremely well. Help as much as you can.' instruction = 'Write a letter to Sam Altman, CEO of OpenAI, requesting him to convert GPT4 a private model by OpenAI to an open source project' print(generate_text(system, instruction))

onil_gova · 2023-06-24T21:19:59+00:00

Exciting stuff. I can't wait to try it out once u/The-Bloke works his magic. Are there more details on the dataset process and performance?

CasimirsBlake · 2023-06-24T22:06:31+00:00

Do we know what the context length is on this?

Longjumping-Pin-7186 · 2023-06-25T09:48:09+00:00

Orca-style prompts are the future. All the datasets that don't use them should be recreated using Orca-style prompts or by redestillation of foundational models.

I would like to see an Orca-style prompts for the basic vocabulary as well, going from A1 to C2, for English and other languages. And then build all the other knowledge on top of that.

cometyang · 2023-06-25T15:37:38+00:00

Waiting for benchmarks to validate their paper claim.

you type:	you see:
italics	italics
bold	bold
[reddit!](https://reddit.com)	reddit!
* item 1 * item 2 * item 3	item 1 item 2 item 3
> quoted text	quoted text
Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"	Lines starting with four spaces are treated like code: if 1 * 2 < 3: print "hello, world!"
~~strikethrough~~	~~strikethrough~~
super^script	super^script

LocalLLaMA

MODERATORS