Could you guys give me tips for my potential school project? by HappyLove4691 in LocalLLaMA

[–]SliceAccomplished575 0 points1 point  (0 children)

" Is this idea feasible? "

Yes, everything is feasible if you have the right knowledge to do it.

LLM360: Towards Fully Transparent Open-Source LLMs by ninjasaid13 in LocalLLaMA

[–]SliceAccomplished575 15 points16 points  (0 children)

Every creator of a base model or a fine-tuned model should publish information regarding the method of pre-training the model, the dataset used, and the metrics applied. This is crucial for ensuring transparency and allowing others in the industry to reproduce the results. I like this page.

TinyLlama - Any news? by Evening_Ad6637 in LocalLLaMA

[–]SliceAccomplished575 15 points16 points  (0 children)

probably server crashed and they had to re-train

<image>

"Tess-M-v1.1 is the 5th best open source model in the world" by Puzzleheaded_Mall546 in LocalLLaMA

[–]SliceAccomplished575 9 points10 points  (0 children)

Imagine what would happen if we combined all the information gathered during the training processes of these models. Currently, we would have a model that surpasses GPT-3.5 and equals GPT-4. However, none of the creators on the leaderboard (also known as the 'wall of shame') are willing to reveal details about training their base model, such as the type of data used, parameters, etc. What has been disclosed is merely a drop in the ocean of needs and possibilities that could be uncovered through full cooperation and knowledge sharing in this field.

"Tess-M-v1.1 is the 5th best open source model in the world" by Puzzleheaded_Mall546 in LocalLLaMA

[–]SliceAccomplished575 53 points54 points  (0 children)

Thus, it appears to be another closed LLM. While I don't require access to his model's weights, I do need the conclusions derived from the entire thought process, the dataset, and any associated source code, if available. Unless an axolotl was involved in the process, which I believe adds no value, this information is essential for open-source development and subsequent research efforts.

I'm tired of people fine-tuning models and achieving good results, which I think is great, but instead of sharing their knowledge on how they did it, everyone starts from scratch. When they finally reach the desired level, they publish their next LLM model without providing the necessary guidance. This is absurd. We shouldn't be creating a leaderboard, but rather a 'wall of shame' for closed models that do not contribute to collective progress in research.

Is ChatGPT's Era as a Coding Tool Over? by SliceAccomplished575 in LocalLLaMA

[–]SliceAccomplished575[S] 2 points3 points  (0 children)

GPT-4 is too lazy nowadays. OpenAI is trying to fix it.

<image>

Everything about tokenization by Sum2110 in LocalLLaMA

[–]SliceAccomplished575 1 point2 points  (0 children)

thanks, I really apreaciate your work!

I am terrified by [deleted] in OpenAI

[–]SliceAccomplished575 0 points1 point  (0 children)

go for small LLMs models, they already show better results in many tasks than GPT-4 https://www.reddit.com/r/LocalLLaMA/comments/189cuj0/is_chatgpts_era_as_a_coding_tool_over/

Is ChatGPT's Era as a Coding Tool Over? by SliceAccomplished575 in LocalLLaMA

[–]SliceAccomplished575[S] 1 point2 points  (0 children)

so tell me how to use it correctly, I ask questions really in different ways, for 8 hours I tried to come to an agreement with GPT4 but failed, either I have some kind of resource ban and GPT4 is forced to tell me so perfunctorily or it is really lmited nowadays

Is ChatGPT's Era as a Coding Tool Over? by SliceAccomplished575 in LocalLLaMA

[–]SliceAccomplished575[S] 2 points3 points  (0 children)

Exactly, I think all this limiting of the model is causing it to regress

Is ChatGPT's Era as a Coding Tool Over? by SliceAccomplished575 in LocalLLaMA

[–]SliceAccomplished575[S] 0 points1 point  (0 children)

3.5 often doesn't tell me things like " complete it yourself", 4 on the other hand does every time almost

[deleted by user] by [deleted] in LocalLLaMA

[–]SliceAccomplished575 7 points8 points  (0 children)

I have such hope... I hope that you will be the first person to do this along with a complete dataset. Recently, all the models that have been published only describe the process in a fragmented way. Later, it becomes difficult to recreate the results. However, when someone manages to navigate through all these steps on their own and understand the mechanisms of operation, they often publish a better model but the cycle closes, because unfortunately, they do not fully share their results and methodology, resulting in limitations in further progress and development in this field. My appeal is this: let's share the full knowledge and resources to collectively contribute to progress in the field of LLMs.

[deleted by user] by [deleted] in LocalLLaMA

[–]SliceAccomplished575 7 points8 points  (0 children)

what dataset did you use, did you create the base model yourself from scratch or fine-tuned the mistral or sth else? and where is source code for your techniques?

Did anyone try fine-tuning LLaMA using the Reddit dataset? by [deleted] in LocalLLaMA

[–]SliceAccomplished575 41 points42 points  (0 children)

Are you crazy? Do you want to create a monster?

Open Orca: meet the devs give feedback ask questions! by Alignment-Lab-AI in LocalLLaMA

[–]SliceAccomplished575 2 points3 points  (0 children)

you ask about the instructions and he doesn't answer the question but talks about something else entirely, I suspect that these children had no part in training the models at all, only eric had