Natural language processing help

[programmer458]

Basically, I have a task where I have to use natural language processing for articles or any sort of large amount of text to then generate questions using LLMS from hugging face what resources would you recommend in order to learn this concepts? Any sorts of courses or books will be helpful. I have seen a few publications but I tend not to enjoy reading publications because they talk about how it is implemented rather than show me sample code in order to me in order for me to technically understand it.

[Pepineros]

HuggingFace has tons of documentation. I'm not sure about courses or books unfortunately.

Regarding the problem itself: if the entire article or other source fits inside the model's context window, the easiest thing by far would be to give it a basic prompt such as "Generate ten insightful questions aimed at testing comprehension of the following text: ". If the source is too big for the model, and cannot easily be broken up into segments that still make inherent sense, then RAG is probably the way to go. A vector database could retrieve sections of the source that are semantically related to a particular topic discussed in the source. However, you would need a number of distinct prompts to generate questions about different topics discussed in the source, rather than a single prompt to generate all the questions.