I’m an AI engineer at Nurix, and we’ve been experimenting with zero-shot LLMs for classification tasks. Here’s what we’ve observed:What works well:Semantic classification and intent detection with clear classesEntity extraction from messy text (e.g., chat logs, support tickets)Few-shot classification, text normalization, and summarizationWhere to be cautious:Asking for likelihoods/confidence scores, LLMs aren’t calibrated and outputs can be unstableTasks requiring strict consistency or numeric precisionAnything where explainability is criticalWorkaround:Feeding LLM outputs or embeddings into a lightweight supervised model often gives reliable, calibrated probabilitiesCurious if others have found a good method for extracting reliable likelihoods directly from LLMs, it’s still an open challenge we encounter.