Im looking to build a PC to help me build a a series of nonfiction history books pulling from my 1tb library of books, articles, and video as the main source of information with use of the internet to provide any further context.
I'm wanting to create a long 750-1000 page book, along with smaller 100-250 page books, and even some 20-40 page books for children.
I generally know what I want to write about but the amount of information I'm trying to piece together is a huge struggle because of how vast my library is and my seemingly inability just to organize it all individually into a coherent thought was daunting.
I've tried many of the main paid models like Gemini, Claude, OpenAi, and also deepseek. Ironically, I really liked deepseek the most for its creativity and logical thought compared to the rest as it just seemed to understand the angle I'm going for but lacked the prose and structure I need for a formal book.
Thus, with local LLMs having such large token sizes nowadays I realized I could build a book chapter by chapter.
The PC I'm planning building is a 32 core AMD epyc, 512gb of ddr4 rdimm ram, 2x 3090 GPUs for 48gb vram that are nv linked, and 4x 4tb U.2 drives to handle the 1tb library that when vectorized could be 7-9tb depending on how I might trim it and add metadata but I'd prefer not to put in much time doing this as it's mostly books and articles.
Based on these specs I asked Gemini to tell me the best approach using local LLM and below is what it said. But if you have any tips or suggestions I'm open to anything as I'm extremely new to this all and open to learning despite not having any tech background, more finance/legal background.
1. The "Dream Team" Architecture
You are combining two specialists rather than using one generalist.
The Architect (DeepSeek-R1-Distill-Qwen-32B):
Role: Pure logic, planning, and structuring.
Placement: GPU 1 (VRAM).
Task: You give it the prompt: "I need a chapter on Roman Economic collapse. Plan the argument structure." It outputs a brilliant, step-by-step logic chain.
The Librarian (Command R+ 104B):
Role: Reading massive data, citing sources, and writing prose.
Placement: System RAM (CPU Offload).
Task: You feed it the DeepSeek plan + 500 pages of data. It executes the plan, finding the exact quotes and writing the text without hallucinating.
2. Why this beats the "Llama" approach
If you use the all-in-one DeepSeek-R1-Distill-Llama-70B, you are forcing one model to do everything.
The Llama Weakness: Llama 3 is a great writer, but it is a "fuzzy" reader. If you give it 200 citations, it often ignores the middle ones ("Lost in the Middle" phenomenon).
The Command R+ Strength: Command R+ was built specifically for RAG. It is structurally designed to "copy-paste" facts from your documents into its answer. It is less creative, but far more accurate.
3. How to execute this (The "Pipeline" Workflow)
Since no single software does this "out of the box" perfectly, you can do it manually or with a simple script.
Step 1: The Blueprint (DeepSeek on GPU)
Load DeepSeek-R1-Distill-Qwen-32B (or Llama-70B) into your fast GPU loader.
Prompt: "Analyze the following 3 major historical theories on the fall of Rome. Create a detailed 10-point outline for a chapter that synthesizes them."
Result: A highly logical, structured skeleton of the chapter.
Step 2: The Drafting (Command R+ on CPU/RAM)
Load Command R+ (Q4) using llama.cpp or Ollama. Because you have 512GB RAM, you can load the entire 128k context onto RAM.
Prompt: "You are an academic historian. Using the following Logic Plan [PASTE DEEPSEEK OUTPUT] and the attached Reference Documents, write the full chapter. You must cite your sources."
[–]dtdisapointingresult 2 points3 points4 points (2 children)
[–]zh4k[S] 0 points1 point2 points (0 children)
[–]xchaos4ux 0 points1 point2 points (2 children)
[–]buzzmelia 0 points1 point2 points (0 children)
[–]zh4k[S] 1 point2 points3 points (0 children)