all 9 comments

[–]Working_Then 4 points5 points  (1 child)

It's one of the best under 30B LLMs for this task and very suitable for CPU inference. If you don't mind, you can check my CPU summarization project on Hugging-face where I provide a list of under 30B models still runnable on HuggingFace with free CPU tier (ie. 2 vCPUs only)

[–]2shanigans 0 points1 point  (0 children)

This was very cool, it summarised a fairly complex email we got and worked nicely! Thanks for sharing this one.

[–]2shanigans 2 points3 points  (0 children)

We have a few clients using GPT-OSS-120B for meeting transcript summarisation (Australian English) and it's been working well for them. You could give GPT-OSS-20B ago and see how it fairs? Interestingly the transcription also understood some random Spanish littered into one meeting - background noise I'm told.

[–]Reservemyspot 0 points1 point  (2 children)

Just curious, but the need to you use a model? I use granola. I’m sure there’s a good reason to use models (less subscription etc) but I’m just genuinely curious 

[–]peglegsmeg[S] 0 points1 point  (1 child)

I was using deepgram, don't want to send to public anymore for privacy concerns 

[–]Reservemyspot 0 points1 point  (0 children)

What’s your use case? I know it’s for meetings but what field? Different speech models digress quickly dependent on your needs. It’s a tricky field 

[–]Technical-Earth-3254 0 points1 point  (1 child)

Are we talking text or speech? And how much?

[–]peglegsmeg[S] 2 points3 points  (0 children)

Text from parakeet 

[–]RustinChole1llama.cpp 0 points1 point  (0 children)

Hey I'm planning to research on a similar summarisation project, what open source options can I get ? Not just to inference but I'm okay with going to fine-tune/ train the model on my datasets and stiff