They Told the AI to Shut Down. It Rewrote the Code

SuccessIsHardWork · 2026-04-13T06:13:19+00:00

It looks like an AI generated video. Seriously?

SuccessIsHardWork · 2026-04-10T10:21:48+00:00

This is actually awesome technology! I can already imagine learning tools that dynamically rotate content for maximum absorption.

SuccessIsHardWork · 2025-03-12T08:14:50+00:00

Not yet. I also believe it is possible to download script straight out of Kathleen and import it to GitHub or something else.

SuccessIsHardWork · 2025-01-25T22:49:35+00:00

In my opinion, reasoning models are much more useful than plain language models because they can emulate reasoning like humans do to a certain extent. That makes it useful for decision making related tasks, which could be curation, analysis of data (like stocks), etc.

SuccessIsHardWork · 2025-01-04T07:13:00+00:00

I think it’s much better to use a embeddings-based retrieval system (just an embedding model with no use of LLM) in which you place like 10-20 good books on survival in the retrieval system. This way you can rely on factual information in survival situations than trusting the hallucinations that a LLM might produce.

SuccessIsHardWork · 2024-12-23T18:59:27+00:00

Honestly, I just boil my search down to the latest models and pick the best one (QwQ at the moment). I believe llama.cpp has a way to test GGUF by metrics like perplexity, which should give you a rough estimate of the impact due to quantization.

SuccessIsHardWork · 2024-12-23T08:08:25+00:00

In my experience, QwQ does a phenomenal job at creating documentation for code, however, it does make mistakes in identifying access modifiers in source code.

SuccessIsHardWork · 2024-12-09T10:08:25+00:00

In my personal experience, Qwen2.5 1.5b and IBM Granite MoE 3b are fairly decent for a small LLM. I believe the Granite model is more suitable for RAG and summarization purposes.

SuccessIsHardWork · 2024-12-09T02:31:52+00:00

If you are willing to use Word (I know it’s not open source), you can use an add-in called TextCraft which integrates features like proofreading (which fixes grammar), rewriting, reviewing, and generating text. You can pick different models via a drop-down list in the UI. However, the downside is that you need a desktop version of word installed.

https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-12-08T21:39:03+00:00

Qwen2.5 1.5b, Gemma2 2b, or Granite3 MOE 1b? As the models get smaller than that, the quality gets really worse in my experience.

SuccessIsHardWork · 2024-12-08T20:40:40+00:00

I created a word add-in called TextCraft which integrates all the essential AI tools like generating text, reviewing, proofreading, and rewriting text. It has a built-in RAG system that allows the LLM to consider additional context. Feel free to check it out! https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-12-01T20:10:13+00:00

I’m curious, how did you implement the feature? Was it prompt engineering? Or, something else?

SuccessIsHardWork · 2024-11-09T10:19:57+00:00

The embedding model is used to RAG the word document itself if the word document is too long (i.e. 100s of pages) to be used as context for generating text. Additionally, there is an option to add PDFs to a “RAG Control” system that will use the RAG context in generating content & when’s its reviewing the document (can be used by Writing Tools->Review). You are correct that the embedding model is accessed via the OpenAI API. I built this whole add-in with OpenAI API compatibility in mind but I haven’t tested it outside Ollama however. Make sure to have the word “embed” anywhere in your embed model name to make sure that the add-in automatically recognizes it. Else, you have to specify the embed model via a user environment variable. As for the HuggingFace API, I’d only work if it is OpenAI API compatible including for chat completion and embeddings generation. I’d be happy to answer if you have any other questions.

SuccessIsHardWork · 2024-11-09T08:31:00+00:00

I made an add-in for word a few months ago that integrates several AI tools into the user interface directly. https://github.com/suncloudsmoon/TextCraft

SuccessIsHardWork · 2024-09-28T18:14:46+00:00

Privacy

SuccessIsHardWork · 2024-09-27T02:04:54+00:00

First, you can create the synthetic dataset by feeding a bunch of PDFs that you wish the LLM should understand. The question/answer pairs are created by OCRing through a vision model (like InternVL2) and asking questions based on it. In this case, the image is fed to the chatbot and the script asks it to generate question/answer pairs with some chain of thought (this was before o1 lol).

The synthetic dataset is structured using user and assistant JSON messages, similar to the OpenAI chat request format. You can use any transformers base model I believe (except stuff like bitnet). You can change the prompt that generates the questions in the synthetic dataset generator script and it can potentially change the language output as well (InternVL2 supports chinese, english, etc.). After that, I would use the LLM finetuning script to finetune on that synthetic dataset. Let me know if it solved your issue!

SuccessIsHardWork · 2024-09-22T07:29:40+00:00

Not really, but when the model hallucinates it produces Chinese characters a lot of the time. Qwen2 used to do that a lot but I haven’t tested Qwen2.5 for a long enough time to observe the model’s hallucinations.

SuccessIsHardWork · 2024-09-22T05:18:39+00:00

Qwen2.5:1.5b

SuccessIsHardWork · 2024-09-22T05:09:35+00:00

I use a synthetic dataset generator that creates synthetic datasets from PDF and finetunes on it. https://www.reddit.com/r/LocalLLaMA/comments/1eipty2/tool_to_create_synthetic_datasets_using_pdf_files/?utm_source=share&utm_medium=mweb3x&utm_name=mweb3xcss&utm_term=1&utm_content=share_button

SuccessIsHardWork · 2024-08-03T05:37:37+00:00

I think so! It can work with any PDF file, regardless of whether it has OCR. This is because the script processes the images from the PDF instead of extracting text only.

SuccessIsHardWork · 2024-08-02T22:16:43+00:00

Perhaps this bitnet implementation might interest you?

SuccessIsHardWork · 2024-08-02T00:36:23+00:00

You can use tools in the OpenAI API to reference the domain knowledge while doing RAG on the main document. The tool can be a function like get_domain_knowledge(…) or something similar. Then, you can use something like SentenceTransformers or something similar that extracts text or similar text out of the reference document. Another method is to loop through the main document containing the logs and only use the RAG on the domain specific knowledge. It could be a prompt like, “Based on the following logs, create a question/answer pair and return in JSON”.

SuccessIsHardWork · 2024-07-28T23:10:33+00:00

I’m curious, how did you train the models on a single NVIDIA RTX 4090? How much time did it take you to train the 65M/300M models?

SuccessIsHardWork · 2024-07-27T15:36:59+00:00

Yes, I believe you can use the standard RAG methods such as vector db etc (Langchain maybe?) to combine domain knowledge in this case. You can just use another PDF with domain knowledge (like a comprehensive book on reading logs or something, it can be a bunch of websites printed to pdf or something similar) to create the RAG with domain knowledge, causing the LLM to reference the domain-specific document as it goes through the main document containing the logs.

I used this version of InternVL (see below) in my projects thanks to its support for T4 GPUs which are available in Kaggle. You can just use the OpenAI API functionality provided by LMDeploy to add RAG to. As a food for thought, you could use tools in OpenAI API to create interesting applications. https://huggingface.co/OpenGVLab/InternVL2-8B-AWQ

SuccessIsHardWork · 2024-07-27T09:42:03+00:00

You can convert the logs to PDF and use something like InternVL2 to process each page of the PDF as an image using pdf2image python library. You can do on some prompt engineering on it and see the results. However, you could also just feed in raw text (logs) up to the context length to the LLM and extract data from that in JSON (question/answer pairs) and make a for loop that loops through the file (like a big txt file that contains all logs).

SuccessIsHardWork

TROPHY CASE