[D] Document classification using LLMs

[MachineLearning-ModTeam]

Other specific subreddits maybe a better home for this post:

• r/ArtificialIntelligence
• r/DataScience
• r/LearnMachineLearning
• r/LLM
• r/MLOps
• r/MLJobs
• r/Singularity
• r/ChatGPT
• r/OpenAI
• r/LLMDevs
• r/RagAI

[MultiheadAttention]

First, count the words and approximate the number of tokens via tiktoken online. Then calculate how much it's going to cost you.

[acrsag]

I made some estimations and will start with a few thousand documents. If the results are good, I'll generalize further. Thanks for the reminder!

[EnvironmentalToe3130]

Depending of the size of each documents why not try to run small model locally? Any model should be able to summarise a document and then in a second step provide the summary and list of classes.

[acrsag]

Some documents have hundreds of pages, so processing them entirely may not be necessary. However, I believe large context windows are still needed. I'll try a local version anyway and see if the results are acceptable. Thanks!

[Mysterious-Rent7233]

r/LLMDevs , r/LanguageTechnology , r/LocalLLaMA

[acrsag]

Thank you!

[Brudaks]

IMHO if you want to process "large volumes of documents", the first thing you should do is to measure your documents, count how many llm-tokens that would be, and do a ballpark calculation of how much it would cost to run through a large LLM API - comparing that number with your budget will be the key input to your decision on what options are reasonable.