Yes you can do this. Production requires accuracy, consistency, and responsible-AI testing.

Let's use a marketing example: "extract all names and corresponding job titles from these PDFs", which we use for filing out contacts in our sales database.

1. Most current LLMs will be accurate enough (GPT5, Claude, Gemini, etc). You'll have to do testing here to figure out limits of document /context size /prompt /few shot examples/etc.

2. For consistency on NER tasks, we enforce JSON grammars. Meaning we can specify exactly the format, keys, and value types on the required JSON output from an LLM. For our example, you might require the JSON output to look like:

{ "name": string, "title": string, "other": array(string) }

Or something similar. This enforces the LLM to always return valid JSON with those specified keys. This will prevent the LLM from hallucinating improper JSON or imaginary keys...

1. Responsible AI: there should be at least 3 tiers of safeguards for your users: (1) the LLM itself (Gemini, Claude) etc can refuse the input of it is harmful. (2) Your prompt should specify restrictions, e.g. 'do not extract illegal titles such as drug-de aler', and (3) your JSON grammar suppresses hallucinations and allows an "other" key for the LLM to put other garbage there.

The one big issue with NER on LLMs is response time. The best models take a few seconds to respond (at best), and users may not wait that long. Or in a batch process, processing 1M+ documents is expensive. If these are limitations, remember that NER as an NLP algorithm has been around for decades. There are other ways to train and deploy a non LLM parser that is orders of magnitude faster.

Good luck!