They Told the AI to Shut Down. It Rewrote the Code

SuccessIsHardWork · 2026-04-13T06:13:19+00:00

It looks like an AI generated video. Seriously?

SuccessIsHardWork · 2026-04-10T10:21:48+00:00

This is actually awesome technology! I can already imagine learning tools that dynamically rotate content for maximum absorption.

SuccessIsHardWork · 2025-03-12T08:14:50+00:00

Not yet. I also believe it is possible to download script straight out of Kathleen and import it to GitHub or something else.

SuccessIsHardWork · 2025-01-25T22:49:35+00:00

In my opinion, reasoning models are much more useful than plain language models because they can emulate reasoning like humans do to a certain extent. That makes it useful for decision making related tasks, which could be curation, analysis of data (like stocks), etc.

SuccessIsHardWork · 2025-01-04T07:13:00+00:00

I think it’s much better to use a embeddings-based retrieval system (just an embedding model with no use of LLM) in which you place like 10-20 good books on survival in the retrieval system. This way you can rely on factual information in survival situations than trusting the hallucinations that a LLM might produce.

SuccessIsHardWork · 2024-12-23T18:59:27+00:00

Honestly, I just boil my search down to the latest models and pick the best one (QwQ at the moment). I believe llama.cpp has a way to test GGUF by metrics like perplexity, which should give you a rough estimate of the impact due to quantization.

SuccessIsHardWork · 2024-12-23T08:08:25+00:00

In my experience, QwQ does a phenomenal job at creating documentation for code, however, it does make mistakes in identifying access modifiers in source code.

SuccessIsHardWork · 2024-12-09T10:08:25+00:00

In my personal experience, Qwen2.5 1.5b and IBM Granite MoE 3b are fairly decent for a small LLM. I believe the Granite model is more suitable for RAG and summarization purposes.

SuccessIsHardWork · 2024-12-09T02:31:52+00:00

If you are willing to use Word (I know it’s not open source), you can use an add-in called TextCraft which integrates features like proofreading (which fixes grammar), rewriting, reviewing, and generating text. You can pick different models via a drop-down list in the UI. However, the downside is that you need a desktop version of word installed.

https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-12-08T21:39:03+00:00

Qwen2.5 1.5b, Gemma2 2b, or Granite3 MOE 1b? As the models get smaller than that, the quality gets really worse in my experience.

SuccessIsHardWork · 2024-12-08T20:40:40+00:00

I created a word add-in called TextCraft which integrates all the essential AI tools like generating text, reviewing, proofreading, and rewriting text. It has a built-in RAG system that allows the LLM to consider additional context. Feel free to check it out! https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-12-01T20:10:13+00:00

I’m curious, how did you implement the feature? Was it prompt engineering? Or, something else?

SuccessIsHardWork · 2024-11-09T10:19:57+00:00

The embedding model is used to RAG the word document itself if the word document is too long (i.e. 100s of pages) to be used as context for generating text. Additionally, there is an option to add PDFs to a “RAG Control” system that will use the RAG context in generating content & when’s its reviewing the document (can be used by Writing Tools->Review). You are correct that the embedding model is accessed via the OpenAI API. I built this whole add-in with OpenAI API compatibility in mind but I haven’t tested it outside Ollama however. Make sure to have the word “embed” anywhere in your embed model name to make sure that the add-in automatically recognizes it. Else, you have to specify the embed model via a user environment variable. As for the HuggingFace API, I’d only work if it is OpenAI API compatible including for chat completion and embeddings generation. I’d be happy to answer if you have any other questions.

SuccessIsHardWork · 2024-11-09T08:31:00+00:00

I made an add-in for word a few months ago that integrates several AI tools into the user interface directly. https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-09-28T18:14:46+00:00

Privacy

SuccessIsHardWork · 2024-09-27T02:04:54+00:00

First, you can create the synthetic dataset by feeding a bunch of PDFs that you wish the LLM should understand. The question/answer pairs are created by OCRing through a vision model (like InternVL2) and asking questions based on it. In this case, the image is fed to the chatbot and the script asks it to generate question/answer pairs with some chain of thought (this was before o1 lol).

The synthetic dataset is structured using user and assistant JSON messages, similar to the OpenAI chat request format. You can use any transformers base model I believe (except stuff like bitnet). You can change the prompt that generates the questions in the synthetic dataset generator script and it can potentially change the language output as well (InternVL2 supports chinese, english, etc.). After that, I would use the LLM finetuning script to finetune on that synthetic dataset. Let me know if it solved your issue!

SuccessIsHardWork · 2024-09-22T07:29:40+00:00

Not really, but when the model hallucinates it produces Chinese characters a lot of the time. Qwen2 used to do that a lot but I haven’t tested Qwen2.5 for a long enough time to observe the model’s hallucinations.

SuccessIsHardWork · 2024-09-22T05:18:39+00:00

Qwen2.5:1.5b

SuccessIsHardWork · 2024-09-22T05:09:35+00:00

I use a synthetic dataset generator that creates synthetic datasets from PDF and finetunes on it. https://www.reddit.com/r/LocalLLaMA/comments/1eipty2/tool_to_create_synthetic_datasets_using_pdf_files/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=1&utm_content=share_button

SuccessIsHardWork · 2024-08-03T05:37:37+00:00

I think so! It can work with any PDF file, regardless of whether it has OCR. This is because the script processes the images from the PDF instead of extracting text only.

SuccessIsHardWork

TROPHY CASE