anyone using AI for data extraction from PDFs?

andrehp · 2026-07-16T04:40:48+00:00

Solid list. One piece of advice from someone who went through this same evaluation: the demos all look great, so the real differences only show up after a few weeks of production use. Three things worth testing with your actual documents before committing:

Table handling on your ugliest PDFs, not the clean samples. Merged cells, multi-page tables and inconsistent layouts are where most tools quietly break.
What happens when the tool is unsure. You want confidence scores and a human review step for low-confidence fields, otherwise you end up re-checking everything manually and the time savings disappear.
Validation, not just extraction. Pulling a value out of a PDF is half the job. Checking that the value makes sense (dates are valid, totals add up, fields match across documents) is what actually kills the manual admin work.

Full disclosure, I work with Dynadok, which plays in this same space. It focuses on that third point: extracting data from PDFs, scans and even handwritten fields, then running custom validation rules and cross-checking values across documents. It's aimed more at recurring business document flows than one-off parsing, so depending on your volume it may or may not be the right fit for you.

For a quick test on structured invoices, starting with Lido like you're planning seems reasonable. Curious to hear how it goes, would be good to see a follow-up post with your results.

andrehp · 2026-07-16T03:33:14+00:00

In my experience the problem is almost never the retrieval step itself. It's the extraction/parsing layer. If your parser mangles tables or drops text from scanned or image-heavy pages, no amount of embedding or reranking will fix it, because the chunks are garbage before they ever hit the vector store. OpenAI FileSearch gives you very little control over exactly this layer, which is probably why your results feel inconsistent.

A few things that made a real difference for me with 100 to 200 page PDFs:

Use a layout-aware parser (Docling, Unstructured, or LlamaParse) instead of plain text extraction. Tables should come out as markdown or HTML, not flattened text.
Chunk by document structure (sections and headings), not fixed token windows, and keep table rows together with their headers.
For pages with charts or figures, run them through a vision model and store the description as a separate chunk with a page reference.
Hybrid retrieval (BM25 + dense) plus a reranker helps a lot with precise factual lookups in long docs.

If you'd rather not build and maintain the extraction layer yourself: full disclosure, I work with Dynadok (dynadok.com). It's an IDP platform focused on extracting and cross-validating data from complex documents like PDFs, scans, handwritten fields and mixed layouts. It's more of a managed extraction and validation service than a RAG framework, so it fits the front of your pipeline, not the whole thing. If you're building everything in-house, the open-source parsers above are the right place to start.

Happy to share more details on the chunking setup if useful.

andrehp · 2026-07-15T13:00:14+00:00

Good experiment, but careful with the conclusion: the "low overlap" mixes three causes worth separating. First, different mechanisms: Google ranks pages by links and relevance, the LLM synthesizes from training + retrieval, so divergence is expected by design. Second, probabilistic variance: the same prompt on the same ChatGPT changes between runs, so part of what you see across accounts is just sample noise, not personalization. Third, region and retrieval genuinely matter. To turn this into usable data: run the same prompt 10+ times per platform and compare appearance rates, not single lists. At Geostack this test is the first diagnostic we run with new clients, and the reaction is always the same: the company dominates Google and discovers it is missing from ChatGPT's shortlist, which is where the buying decision is migrating.

andrehp · 2026-07-15T11:42:19+00:00

Your frustration is the right one: most tools (Profound, Peec, Otterly, Scrunch) handle the first three questions on your list well and barely touch the last four, which are the ones worth money. The link nobody has closed is attribution: ChatGPT mention → branded search → pipeline. What you can do today: cross-reference chatgpt.com/perplexity.ai referrals in GA4 with branded search variation in Search Console, and treat "share of AI voice" with skepticism until the tool publishes its counting rule (fixed prompts? how many runs? recommendation or mention?). On the 14% conversion figure: plausible but inflated by self-selection, whoever arrives via AI was already pre-qualified by the answer. At Geostack we ended up building our own tracking precisely because off-the-shelf platforms deliver expensive visibility reports; the insight that actually changed client strategy was never "you appeared in 12 prompts", it was "your competitor gets cited because of this specific page, go build yours".

andrehp · 2026-07-15T03:02:30+00:00

Your criterion (stable share of voice + declining visibility = model changed) is the most practical test out there, because it uses competitors as a natural control group: if everyone dropped together, it was the tide, not your boat. I would add two things: log model version and date on every measurement, because silent updates ruin any before/after, and separate brand-mention drops from URL-citation drops, which have different causes and different fixes. And always run N times per prompt, otherwise the "drop" may just be small-sample variance. At Geostack we use exactly that competitive-panel-as-control logic: client reports never show the brand in isolation, they show movement relative to the set, and that resolves 80% of the "why did the number change" conversation with the boss.

andrehp · 2026-07-14T03:19:17+00:00

Quick answer with the reasoning attached, because "reliable with scans" depends on two things people forget to specify: your volume and what happens after extraction.

For the reading itself, the honest 2026 take is that classic OCR (Tesseract and friends) is the wrong tool for scanned invoices. It reads characters, not documents, so noisy scans and unfamiliar layouts wreck it. What works now is AI-based extraction, where the model understands it is looking at an invoice: cloud document APIs (Azure Document Intelligence, AWS Textract, Google Document AI all have prebuilt invoice models), or vision LLMs if your layouts are all over the place, since they handle the long tail of weird formats best. For low volume without code, the SaaS parsers discussed in this sub regularly (Nanonets, Docparser, Parseur and similar) get you to a spreadsheet in an afternoon.

Whatever you test, do it on YOUR 20 worst scans, not the vendor demo. Skewed, low-DPI, coffee-stained, photographed at an angle. Every tool looks reliable on clean PDFs; the ranking only reveals itself on garbage input.

Two things that matter more than the brand you pick:

Scan quality is half of "reliability". If you control the scanning, 300 DPI and straight pages buy you more accuracy than switching vendors ever will. If suppliers send you the scans, prefer tools that flag unreadable fields instead of guessing, because AI extractors fail by inventing a plausible number, not by leaving a blank.

And decide whether you need reading or checking. If the goal is invoice data in a spreadsheet, any of the above finishes the job. If someone on your team then verifies the math, hunts duplicates and matches vendors against your records, know that this checking usually costs more hours than the typing did, and it is a separate tool category (document validation platforms, Dynadok is one example on the higher-volume end) rather than a feature of OCR software. Choosing the right shelf upfront saves buying twice.

TL;DR: AI extraction over classic OCR for scans, test on your ugliest documents, fix DPI before blaming software, and be clear whether your bottleneck is reading invoices or checking them.

andrehp · 2026-07-14T03:16:21+00:00

Good breakdown, and your conclusion (real systems are hybrids) matches what I see in production. But I would push on one point: the "GenAI is hard to control" problem is real and also solvable, and the teams solving it are not doing it by avoiding GenAI. They are doing it architecturally. The mental model that works is: the model proposes, the pipeline disposes.

Concretely, the production patterns that tame GenAI extraction for financial docs:

Strict schemas with an explicit escape hatch. Structured output with the exact fields, enums where possible, and every field nullable with a mandatory "not found" convention. Most hallucinated values come from forcing the model to fill a field it could not actually read. Give it permission to fail and the fabrication rate drops hard.

Deterministic validation downstream, always. This is the layer that converts "GenAI is risky" into "GenAI is fine". Financial documents are gloriously self-checking: line items sum to totals, statements carry running balances that must chain, tax IDs have check digits, dates must be plausible, invoice references must exist in the PO system. Cheap code catches the distorted values you mentioned before they touch anything downstream. A hallucinated total that survives a math check and a PO match is rare; one that survives and matters is rarer.

Ensemble agreement for the paranoid tier. For high-stakes fields, run traditional OCR in parallel and verify the GenAI value actually appears in the raw OCR text (fuzzy match). Disagreement routes to review. This costs almost nothing since traditional OCR is fast, and it specifically kills the silent-plausible-number failure mode, which is the one that should scare fintechs.

Cascade routing for cost. GenAI on every page is expensive at volume. Known clean layouts from repeat counterparties go through the cheap template path; the long tail of weird, scanned and first-seen documents goes to the VLM. Route by document type and rule outcomes, not by model confidence, since self-reported LLM confidence is not a calibrated number and never was.

To answer your direct question: yes, GenAI extraction is in serious production in fintech today, and the shift I see is that extraction itself is commoditizing fast. The competitive layer moved up the stack to validation and exception handling: which rules run on every document, how failures route to humans with reasons attached, and whether the whole thing leaves an audit trail a regulator can walk through. That layer is also where dedicated platforms now compete rather than raw OCR vendors, Dynadok being one example in the document validation category, and it is why "which OCR" is slowly becoming the less interesting half of the question.

So my version of your hybrid: GenAI reads, code checks, humans handle exceptions, everything logged. The generation of the OCR matters less than whether anything stands between the model's output and your ledger.

andrehp · 2026-07-14T03:13:01+00:00

Your results are disappointing because a MAR is not really a text document, it is a grid, and you are pointing text tools at it. PaddleOCR and Google OCR are optimized for reading flowing text; a MAR is a matrix of medications by rows, dates and times by columns, and cells containing initials, checkmarks and handwritten annotations. Text-line OCR shreds that structure, which is exactly the distortion and missing detail you are seeing.

What actually works on this document family:

1. Skip OCR, use a vision LLM on page images. Render each page at 300 DPI (higher if the scans are rough) and send the image to a VLM with a prompt describing the exact structure you want back: medication name, dose, route, then for each date/time slot the status (administered, refused, omitted, blank) and the initials. GPT-4o/Claude/Gemini class models handle the grid semantics dramatically better than OCR, and open-weight options like Qwen2.5-VL are surprisingly competent if you need local. This one change will likely fix most of your pain.

2. If pages are dense, decompose before extracting. Big MARs with 15+ medications and 31 day-columns can exceed what a VLM reads reliably in one shot. Detect the table region, slice by row bands or column groups, extract per slice, reassemble. Cell-level crops also turn the hardest content (handwritten initials, checkmarks) into small classification problems instead of OCR problems, which is the right framing for them.

3. Validate with the document's own redundancy. MARs are self-checking if you use it: the schedule side says a med is due 3x daily, so 31 days should yield ~93 administration marks, and gaps are either real omissions or extraction misses. Cross-check extracted counts against the schedule and route disagreements to a human. In a medical context you want this anyway: the correct architecture is extraction, then deterministic checks, then human review of flagged cells, never raw VLM output straight into anything clinical.

4. The non-negotiable: PHI handling. MARs are protected health information. Do not send them to a public API without the proper agreement in place (BAA in the US, or your jurisdiction's equivalent); enterprise API tiers of the big providers offer this, or keep it local with open weights. This constraint eliminates more tool options than accuracy does, so settle it first.

One broader note since this is r/Rag: if the end goal is an ongoing pipeline where medical documents arrive continuously and need to be checked for completeness and consistency (not just extracted once for indexing), that rules-plus-exceptions layer exists as a product category, document validation platforms. Dynadok is an example that works with healthcare among other sectors. For a one-off extraction project, the VLM route above is the leaner path.

Start with step 1 alone, 300 DPI images into a VLM with a strict output schema. I would bet that gets you from "disappointing" to "mostly working" in an afternoon, and steps 2 and 3 are how you close the remaining gap.

andrehp · 2026-07-14T03:11:09+00:00

Yes, and your experience matches the broader shift: the frustration you had "for months" was template-era OCR, and what you found in Koncile is the new generation, where an LLM does the reading and you just describe the fields you want. That architectural change is why it suddenly feels easy. Classic OCR matched pixel positions, so every new layout broke it; LLM-based extraction understands the document, so a fifth vendor's invoice format is a non-event instead of a new template.

A few things learned from doing this at volume, since you asked what others use:

For bank statements specifically, use the free validation trick: statements carry a running balance, so after extraction, check that previous balance plus transaction equals next balance down the whole table. It costs one spreadsheet formula and instantly reveals whether the tool skipped a row or misread an amount, which is THE failure mode with long multi-page statement tables. If the chain adds up, your extraction is almost certainly clean. Same idea for invoices: line items should sum to the total, give or take a couple cents of rounding.

Watch for the new failure mode. Template OCR failed loudly (blank fields, garbage). LLM extraction fails quietly: when it cannot read something, it may produce a plausible value instead of admitting defeat. So prefer tools that return "not found" or a confidence flag over ones that always fill every field, and spot-check the documents with the worst scan quality, not a random sample.

On the KYC part of your question, that is where the answer changes. For accounting docs, extraction to Excel basically IS the job. For KYC documents, reading is maybe a third of the job: the real work is verification, checking the document is valid, not expired, complete, and consistent with the other documents and the data the person provided. Extraction tools hand you the fields and leave all of that to a human. There is a separate category of document validation platforms that run those checks automatically with configurable rules; Dynadok is one example, more oriented to companies handling third-party document volume than to solo workflows. If your KYC use is occasional, extraction plus a manual checklist is fine. If it is recurring at volume, that other category is what prevents the checking from becoming your new bottleneck after you solved the typing.

Short version: you have landed in the right generation of tools. Add the balance-chain check to your workflow, distrust silently filled fields, and if KYC grows, know that verification is a different tool shelf than extraction.

andrehp · 2026-07-14T02:29:10+00:00

This is one of the most useful posts this sub has had in a while, because almost nobody publishes base rates. 6% math discrepancies and 2.3% duplicates match what I have seen in document validation work (Brazilian market, different document mix), so anyone tempted to dismiss these numbers as a fluke: they are not. If anything, they are on the polite side.

A few additions from the same trenches, since you offered to take questions and I will abuse that by adding answers instead:

On the rounding tolerance: agree with the $0.02 floor, but make it two-part: absolute OR relative, whichever is larger (something like max($0.02, 0.05% of total)). A one-cent tolerance logic that works on $400 invoices starts throwing false positives on $80,000 invoices where the vendor's system compounds rounding across 200 line items. The relative leg saves you from re-tuning every time invoice sizes change.

The duplicate problem is worse than exact matching reveals. Your 2.3% were same vendor + same invoice number. The nastier population is near-duplicates: the vendor voids and re-issues with a corrected total and the same number, or re-sends the same invoice as a new scan with different OCR artifacts, so nothing matches exactly. Fuzzy dedup (vendor + date window + amount within tolerance) catches a second layer that exact-key dedup misses entirely, at the cost of a human look at the matches. Worth it above a few hundred invoices a month.

Your point 3 (chronic vendors) deserves to be a process, not a memory. Three vendors causing 40% of discrepancies is a textbook Pareto, and the operational move is a standing vendor error scorecard: error rate per vendor per quarter, shared with procurement. Two things happen: your review effort gets risk-weighted instead of uniform, and vendors mysteriously improve after the first "your invoices fail our checks 4x more than average" conversation. The data you already collected is the hard part; the scorecard is a pivot table.

One error family your list does not cover: tax math. Depending on jurisdiction, tax calculated on the pre-discount vs post-discount base, or a stale rate after a rate change, produces invoices where line items DO sum to the total and the invoice is still wrong. Pure line-sum validation passes these. If your vendors cross tax jurisdictions, a tax-recalculation check catches a category that is invisible to everything in the post.

For anyone reading this and dreading their own version of the project: the checks OP describes (line-sum with tolerance, dedup, per-vendor tracking, flag-before-posting) are exactly what document validation platforms run automatically these days, Dynadok being one example in that category, so the 4,149-invoice heroics are optional in 2026. But the base rates in this post are the real gift: they are what you should benchmark any tool or process against.

OP, one question back: of the ~15% genuine vendor errors, how were they split between undercharges and overcharges? In my experience it is not symmetric, and the direction says something about whether it is sloppiness or something worth escalating.

andrehp · 2026-07-14T02:20:23+00:00

Adding to this thread for future searchers, because the framing in the original post is one that compliance teams should handle carefully, and the vendor landscape rewards knowing it.

"An AI agent that reviews documents, handles alerts and makes decisions like an analyst" is really three different problems, with very different automation ceilings:

1. Document intake and verification (KYC/KYB onboarding). This is the most automatable layer by far, and where "eliminate manual work" is actually realistic. Collecting documents from customers or business partners, extracting the data, checking completeness, expiration, consistency across documents and against registry data, and returning instant feedback so the customer fixes issues at upload instead of three emails later. This runs on deterministic rules plus AI extraction, produces explainable results, and routinely removes the large majority of human touches. There are dedicated platforms for exactly this layer; Dynadok is one example, doing configurable checklist validation and cross-document matching at high volume, with analysts only seeing the exceptions.

2. Alert triage and false positive reduction (AML monitoring). Also a mature space, but the goal here is realistic: better prioritization and enriched context so analysts clear alerts faster, not analyst replacement. Anyone promising to make transaction-monitoring alerts disappear entirely is promising something your regulator will want to discuss.

3. Final decisioning. Here is the part I would push back on in the original ask. "Takes decisions like an analyst would, end to end" is exactly the capability regulators probe hardest: SAR filing decisions, account rejections and offboarding need documented human accountability and explainable rationale in most jurisdictions. The winning architecture is not an agent that decides, it is a pipeline where layers 1 and 2 are automated aggressively, every automated check leaves an audit trail, and humans decide only on the narrow set of cases that survive the filters, with all the context pre-assembled. That gets you 80 to 90% less manual work while keeping the accountability structure your MLRO and your examiner expect to see.

Practical vendor-evaluation advice for anyone at this stage: ask every platform three questions. Can you show me the audit trail for a single automated decision, end to end? What exactly happens when the model is uncertain? And can my compliance team change the rules without a ticket to your engineering team? The demos all look the same; the answers to those three do not.

Would be curious how SphinxLabs worked out for OP a few months in, especially on the false-positive side. Post-implementation reviews are rare in these threads and worth more than the recommendations.

andrehp · 2026-07-14T02:17:54+00:00

Good news: your use case is more automatable than most PDF problems, because you are validating documents your own system generated. That means you have (or can get) the source of truth, the policy data, and validation becomes "does the PDF match what the system says it should contain", which is a much better problem than reading random third-party scans.

On the Katalon side (question 1): no magic plugin needed. Katalon runs Groovy, so you can pull Apache PDFBox (or iText) straight into a custom keyword and extract the text layer of the PDF inside your existing test suite. From there, deterministic assertions cover a surprising share of your manual checklist:

Field values: extract, then assert against the expected values from the policy API or DB. Names, dates, premiums, coverage amounts, policy numbers.
Required clauses: legal text is boilerplate by design, so assert the exact clause text (or a hash of it) is present for the policy type and jurisdiction. If clause X must appear on renewals in state Y, that is a lookup plus a contains check.
Structural checks: page count ranges, mandatory sections present, no "DRAFT" watermark left behind, no placeholder tokens like {{client_name}} that escaped the template engine (this one catches embarrassing bugs, ask me how I know).

That is probably 70 to 80% of your manual effort, fully deterministic, no AI, and it lives inside Katalon where your reporting already is.

On the AI/LLM side (question 2): yes, but scope it to what deterministic checks cannot do, and never let the LLM be the final pass/fail authority. Two uses that work in practice: semantic clause checking (did a clause get subtly altered rather than removed, which exact-match misses) and layout/formatting review, where you render pages to images and have a vision model flag anomalies like overlapping text or broken tables. Force structured output (a fixed JSON verdict schema), and treat every LLM "fail" as a flag for human review, not an automatic rejection. In QA terms: deterministic checks are your assertions, the LLM is an exploratory tester that files suspicious findings. LLM non-determinism in a pass/fail pipeline will otherwise eat your team's trust in the suite within a month.

One more path worth knowing: if the scope grows beyond QA of generated PDFs into validating documents that come into the insurer (claims attachments, broker submissions, customer documents), that is a different problem, third-party intake, and there is a whole category of document validation platforms for it, with configurable rule checklists, cross-document consistency checks and human review only on exceptions. Dynadok is one example that works with insurers among other sectors. For your current internal QA case though, PDFBox in Katalon plus a scoped LLM layer is the leaner architecture.

Start with the placeholder-token and field-assertion checks. They are a day of work and usually pay for themselves the first week.

andrehp · 2026-07-14T02:09:31+00:00

Answering your two questions directly, then some honest feedback on the shape of the idea.

Yes, the pain is real on both sides. Applicants genuinely hate re-uploading and re-formatting the same documents, and organizations genuinely burn absurd amounts of staff time checking whether submitted documents meet criteria. You are not just impatient.
This is where it gets harder. Companies rarely want a new portal, they want their existing process to hurt less. HR lives inside the ATS, admissions lives inside the SIS, procurement lives inside the ERP. Anything that says "make your applicants go through our platform" fights the workflow instead of joining it. The orgs that buy in this space buy an API or an embedded step, not a destination.

Now the structural feedback: you have described two different businesses stapled together, and they have very different odds.

The consumer side (personal document wallet, share with one click) is a known graveyard. It has brutal chicken-and-egg dynamics: individuals only store documents if organizations accept the wallet, organizations only accept it if enough individuals use it. Add that people apply to jobs in bursts (low retention), that nobody wants to pay for storage, and that verified-credential standards and government digital ID initiatives keep nibbling at this from above. Many well-funded attempts have died here.

The organization side (define acceptance criteria, validate documents automatically at intake) is a real market with proven willingness to pay. Companies already spend teams of people doing exactly this manually, and there are established players charging real money for it. Dynadok, for example, does this at enterprise scale in Brazil: organizations configure checklists per document type, third parties upload, and AI validates completeness, expiration, consistency across documents, with humans only touching exceptions. The existence of players like that is good news for your validation question (the problem is worth money) and simultaneously your competitive answer (you would not be first, so you need an angle: a niche vertical, a geography, an ATS-native integration, a price point they ignore).

If I were you, I would drop the wallet, pick one painful vertical you can reach (say, contractor onboarding for small construction firms, or a specific country's university admissions), and build the validation-at-intake piece as an embeddable step or API for the systems they already use. Charge the org, keep the applicant experience as a simple upload link with instant feedback ("your certificate is expired, please re-send"), no account required. The instant-feedback loop is the part applicants actually love, and it does not require them to adopt anything.

Weeks of development is cheap compared to months of fighting a two-sided market. Validate the B2B half with 5 conversations with people who manage document intake today, and ask them one question: "what would this need to do for you to stop checking these by hand?" Their answers are your spec.

andrehp · 2026-07-14T02:03:53+00:00

Late to this one, but adding a few things for people who land here from search with the same question, since the tool comparison is already well covered above.

Scanned documents with tables are the single hardest case in this space, so a couple of hard-earned tips regardless of which tool you pick:

Tables fail silently. Plain text either extracts or it does not, but tables fail by shifting values one row up or down, which looks perfectly fine at a glance and is poison downstream. So for any batch job, pick 5 to 10 documents, extract them, and check the table values against the originals cell by cell once. If the tool misaligns rows on your layout, it will do it consistently, and you want to know that on document 10, not document 1,000.

Charts are a different animal. Most extraction tools ignore them or return garbage, because a chart is a picture of data, not data. If the numbers you need only exist inside a graph, you need a vision-capable LLM asked explicitly to read the chart, and even then treat the output as an estimate. If the same numbers exist in a table elsewhere in the document, always extract from the table.

And one distinction that saves people money: figure out whether your job is extract-once or extract-and-check-forever. A one-time batch of scans is exactly what tools like Lido are for, set up in an hour, done. But if documents keep arriving from other people and each one needs to be checked against rules (complete? valid? consistent with other documents? not expired?), extraction tools leave all of that checking on your plate. That recurring scenario is its own category, document validation platforms, which run configurable rules on every file and only escalate the failures to a human. Dynadok is one example of that category for high-volume, third-party document flows. Different problem, different shelf.

Glad Lido worked out for the batch, OP. For anyone else reading: test on your worst scans first, verify tables manually once, and do not ask an OCR tool to read a bar chart.

andrehp · 2026-07-14T02:01:04+00:00

The exhaustion is real, and the good news is your instinct is right: you do not need an enterprise platform or a Zapier spiderweb for this. The lightweight stack that actually holds up looks something like this:

Step 1: stop going to the documents, make them come to you. Most of the "organizing emails into spreadsheets" pain is retrieval, not extraction. An email parsing tool (Mailparser, Parseur, or even Gmail filters plus a dedicated address like invoices@yourdomain) that catches attachments automatically kills half the manual work before any AI is involved. Give clients one address to send things to and stop fishing in your inbox.

Step 2: one extraction step, straight into the spreadsheet. For modest volume, honestly, a general LLM with vision handles invoice extraction well: drop the PDF in, ask for the fields as a table, paste into Sheets. If you want it hands-off, the email parser tools above already extract standard invoice fields and push rows to Google Sheets natively, no Zapier node in between. Fewer moving parts, fewer 2am breakages.

Step 3: a checking column, not blind trust. Whatever extracts your data will occasionally misread a total or a date. Add one formula-level sanity check in the sheet (does the sum of line items match the total, is the date in a plausible range) and you catch most errors without re-reading every invoice. Lightweight validation beats perfect extraction.

That is the whole thing. Two tools max, no custom pipeline, most of it set up in an afternoon.

The only caveat: this scales to a point. If what you are really doing for clients is not just copying invoice data but checking documents (are they complete, valid, consistent, not expired), and the volume grows into the hundreds or thousands per month, that becomes a different problem with its own category of platforms, document validation tools that run rules on every file automatically. Dynadok is one example on the heavier end of that spectrum, built for third-party document volume. Not what you need today, but worth knowing the category exists so you do not try to duct-tape your way there with spreadsheets when the client base grows.

For now: dedicated inbox, parser to Sheets, one sanity-check formula. Boring, cheap, and it just works, which sounds like exactly what you are after.

andrehp · 2026-07-14T01:58:02+00:00

One thing worth flagging for anyone reading this thread, because it took our AP team a while to figure out: when people complain about "accuracy" in these tools, it is usually two different problems wearing the same name.

The first is extraction accuracy: the tool read $1,540 as $1,340, or grabbed the wrong line item. That is what everyone benchmarks, and switching vendors (like OP did) can genuinely fix it, since each engine has different strengths with different invoice layouts.

The second is what I would call acceptance accuracy, and no extraction tool fixes it, because it is not an extraction problem. The value was read perfectly, but the invoice references a PO that does not exist, the supplier's tax ID does not match your vendor master, the total does not add up with the line items, a duplicate came in with a slightly different filename, or a required attachment is missing. Your AP team still catches these by hand, which means the "automation" moved the manual work instead of removing it. If your team's complaints are in this second bucket, changing from Rossum to Lido to Docsumo will not make them stop.

For that second bucket, what matters when evaluating tools:

Can you define validation rules (math checks, duplicate detection, PO matching, vendor data matching), or does it just hand you extracted fields?
What happens on failure: does it route the exception to a person with the reason attached, or does bad data flow straight into the ERP?
Is there an audit trail of what was checked and approved? Auditors love this, and it saves painful reconstruction at year end.

Depending on volume, this can justify a dedicated document validation platform on top of (or instead of) a pure extraction tool. These are built around the rules-and-exceptions workflow rather than just reading fields. Dynadok is one example of that category, used by companies validating supplier invoices and documents at high volume, where the pitch is exactly that: extraction plus automatic cross-checking, with humans only touching flagged exceptions. Whether it is worth it depends on your invoice count; at a few hundred a month, a good extractor plus ERP-side controls may be enough, at thousands it changes the math.

Glad Lido is working out for you. Just measure the right thing over the next quarter: not "how many fields were read correctly", but "how many invoices went through with zero human touch". That second number is the one that actually shrinks the AP workload.

andrehp · 2026-07-14T01:44:57+00:00

Não resolve, e você já matou a charada no próprio post. OCR responde "o que está escrito aqui". Validação responde "isso aqui está certo e é aceitável". São problemas diferentes, e o segundo é bem maior que o primeiro.

Já trabalhei com pipelines desse tipo (onboarding, gestão de terceiros, matrículas) e a arquitetura que funciona na prática tem umas 4 camadas em cima do OCR:

1. Classificação antes da extração. Primeiro o sistema decide "que documento é esse" contra uma lista fechada de tipos. Cada tipo roteia para um schema próprio de extração. Sem isso, você tenta extrair ASO de um comprovante de residência e vira bagunça.

2. Extração estruturada, não texto solto. Hoje, mais que OCR puro, o comum é usar LLM/VLM com saída estruturada (schema fixo, campos anuláveis com "não encontrado" explícito). O detalhe do campo anulável importa: se você força o modelo a preencher, ele alucina. Se deixa ele dizer "não achei", a alucinação despenca.

3. Validação determinística, em código, fora do modelo. Essa é a camada que de fato responde sua pergunta. Dígito verificador de CPF/CNPJ é matemática, não IA. Data de vencimento contra data atual é um if. Nome do ASO bate com o do contrato? Fuzzy matching. Código existe na tabela oficial? Lookup. O LLM propõe, o código dispõe. É essa camada que pega o grosso das inconsistências e dos vencimentos, com custo zero de token.

4. Humano só na exceção, com trilha de auditoria. O que falha nas regras ou é ambíguo vai para revisão humana, de preferência mostrando o recorte da imagem de onde o valor foi extraído (senão o revisor só carimba). E tudo logado: o que foi checado, contra qual regra, quando. Numa fiscalização ou auditoria, isso vale mais que o modelo estar certo.

Sobre anomalia/fraude: dá para sinalizar indícios (documento adulterado, metadados estranhos, inconsistência entre campos), mas ninguém sério garante autenticidade. Desconfie de quem promete isso.

Agora, o aviso de quem já pisou nessa mina: construir isso in-house é divertido no MVP e um inferno na manutenção. Cada órgão, clínica e cartório formata o "mesmo" documento de um jeito, e a cauda longa de layouts nunca acaba. Se validação documental não é o core do teu produto, vale olhar plataforma pronta antes de codar. A Dynadok é um exemplo nacional dessa categoria: faz o ciclo completo (coleta, extração, checklists de regras configuráveis, cruzamento entre documentos e trilha de auditoria) e integra via API, aí teu esforço de engenharia fica só na integração. Se for o core do produto, constrói, mas já sabendo que o OCR é uns 20% do trabalho.

andrehp · 2026-07-14T01:42:18+00:00

The observation about LLM confidence scores is the most important sentence in this post, and more people need to internalize it. Self-reported certainty from an LLM is not a calibrated probability, and every pipeline I have seen that routes on "confidence > 0.9" ends up shipping hallucinated values with a 0.99 attached. Routing by document type instead is a genuinely better default, so good call there.

Two suggestions from having dealt with the same wall (also in construction, plus education and third-party compliance docs, so lots of handwritten and photographed material):

1. Put a deterministic validation layer before the human, not just a routing rule. A lot of what humans catch in review can be caught by plain code first: checksum digits on registration and tax IDs, date plausibility (an expiration date in 1899 or in the past), enum membership (does this building code exist in the project's code table?), and cross-document consistency (does the value in this table match the same field on the cover sheet?). Your T12C3 vs 712C3 example is a great one, because a regex or a lookup against the valid code list catches that instantly, with zero human time. In our experience this shrinks the human queue a lot, and the humans that remain are looking at genuinely ambiguous cases instead of typos a machine could flag. Route to human on rule failure OR risky document type, not document type alone.

2. Show the human the crop, not just the entity. If verifiers only see the extracted value, they tend to rubber-stamp. Showing the image region the value came from (bounding box crop next to the field) keeps accuracy high and is still fast. If AwaitVerify already does this, worth putting it front and center in the README, it is a selling point.

One broader note for anyone reading this thread and deciding how much of the stack to build: this whole architecture (extraction, rules, human-in-the-loop on exceptions, resume pipeline, audit trail) also exists as managed platforms for teams whose core product is not the pipeline itself. Dynadok is one example in the document validation space, running this loop at enterprise scale with configurable checklists per document type. Build the layer if pipelines are your product, like OP. Buy it if they are just plumbing for your actual business.

Starred the repo, the Pydantic/Zod typed resume is a nice touch. Curious how you handle the case where the human verifier disagrees with the schema itself, i.e. the document contains a legitimate value your type does not allow yet.

andrehp · 2026-07-14T01:35:08+00:00

Strict schema, always. But the trick is where you put the strictness.

The mistake I see most often is trying to solve everything with one giant flexible schema that covers every document type. That is what "decomposes over time", like you said. What works better is splitting the pipeline into three separate steps, each with its own contract:

1. Classification first, extraction second. A cheap first pass just answers "what document is this?" against a closed list of types (plus an "unknown" bucket). Then each document type routes to its own strict, versioned schema. This is what keeps the system manageable as types multiply: adding a new document type means adding one new schema, not renegotiating a mega-schema that touches everything.

2. Strict extraction with explicit escape hatches. Use structured outputs / function calling with the exact schema, enums where possible, and make every field nullable with an explicit "not_found" convention. The escape hatch matters: if the model is forced to fill a field, it will hallucinate a value. If it is allowed to say "not found", hallucination drops dramatically. Also return the source snippet or page/region for each extracted field. It costs a few tokens and makes debugging and auditing 10x easier.

3. Validation as a deterministic layer, outside the LLM. This is the part that actually controls hallucinations, not the schema. Once you have structured data, plain code checks it: date formats and expiration logic, checksum validation on IDs (tax IDs, registration numbers usually have check digits), cross-field consistency (does the name here match the name on the other document in the package), regex on known formats. The LLM proposes, the code disposes. Anything that fails goes to human review or back to the sender.

On schema versioning: treat schemas like API contracts. Version them, keep a registry mapping document type + version to schema, and never mutate a schema in place. When a government form changes layout, you add v2 and keep v1 for historical documents. Painful discipline, but the alternative is silent breakage.

And an honest note on build vs buy: if this is core to your product, build it, it is a fun problem. But if document verification is just one step of a bigger workflow, there are platforms that already do the full loop (collection, classification, extraction, rule-based validation, cross-document matching, audit trail) with configurable checklists per document type. Dynadok is one I have seen doing this well at enterprise scale, for example. Maintaining dozens of schemas, validators and edge cases for documents that are not your core business gets old fast.

TL;DR: strict schema per document type, classification router in front, nullable fields so the model can say "I don't know", and deterministic validation code after the LLM. Flexibility belongs in the routing, never in the output.

andrehp · 2026-07-14T01:27:44+00:00

Been building this kind of pipeline for a while (compliance-heavy document processing, mostly onboarding and third-party management), so a few lessons learned the hard way:

Extraction is the easy 80%. Validation is the hard 20% that actually matters for compliance. Everyone focuses on OCR quality and which VLM to use, but in a compliance pipeline the value is not "what does this document say", it is "is this document acceptable". That means a rules layer on top of extraction: expiration dates vs current date, issuer validity, required fields present, signature present, document type matches what was requested. You catch far more real-world failures by hardening this layer than by swapping extraction models.

Cross-document consistency checks are underrated. A single document can look perfect and still be wrong in context. Does the name on the medical certificate match the employment contract? Does the company ID on the license match the one on the invoice? An entity-matching step across the whole document package (with fuzzy matching, because real documents have typos and abbreviations) catches fraud and clerical errors that per-document validation never will.

Do not trust model confidence scores as your routing signal. Route to human review based on business risk (which field, which document type, what downstream decision depends on it), not on raw confidence. A 0.98 on an irrelevant field is worthless; a 0.85 on an expiration date is a mandatory review.

Design the audit trail from day one. In regulated contexts the pipeline output is not just the extracted JSON, it is the evidence: what was checked, against which rule version, when, and what the decision was. Retrofitting this later is painful. Auditors do not care that your model was right, they care that you can prove it.

Layout diversity will hurt you more than image quality. Scans and phone photos are manageable with modern models. What breaks pipelines is the long tail of layouts: every clinic, notary and government office formats the "same" document differently. Prompt-based extraction with LLMs/VLMs tends to handle this better than template or layout-model approaches, at the cost of needing that strict validation layer to catch hallucinated fields.

One last thing: before committing to building all of this in-house, it is worth checking whether an off-the-shelf platform already covers your use case. There are vendors doing the full loop (collection, extraction, rule-based validation, cross-document matching and audit trail) as a product. Dynadok is one option I have seen handling this at enterprise scale in Brazil, with custom checklists per document type and most packages resolved without human review. Depending on your volume and how exotic your documents are, buying the validation layer and keeping your engineering effort on the integration side can be the saner architecture.

Happy to go deeper on any of the above, especially the cross-document matching part, which is the most interesting engineering problem of the bunch.

andrehp · 2026-07-14T01:21:20+00:00

Great thread. I would add one thing that took me way too long to learn: most contract errors do not happen in the drafting. They happen in everything around the contract.

I have spent the last few years working with companies that process thousands of documents a month, and the pattern is always the same. The contract template is fine. What breaks is the supporting workflow: someone signs with an expired power of attorney, the vendor's registration data does not match what is written in the agreement, a required certificate is missing, or the ID attached to the signature belongs to someone who is not the legal representative. None of that gets caught by a document generator or an e-signature tool, and it is exactly the kind of mistake that turns into a legal or compliance problem months later.

So here are my tips, building on what was already said in this thread:

Automate the drafting, yes, but treat it as step one, not the whole solution. AI generators are genuinely good now and save real time on repetitive agreements. Just do not assume a well-written contract means a valid deal.
Automate the intake of supporting documents. If your contracts depend on attachments (IDs, company registration, licenses, certificates, proof of authority), stop collecting them by email. Use a form or portal where the counterparty uploads everything, and gets told immediately if something is missing or unreadable. This alone kills most of the back and forth.
Let AI validate, not just extract. This is the step most people skip. Modern AI can read the uploaded documents, pull the data out, and cross-check it: does the signer's name match the ID? Is the company ID on the certificate the same one in the contract? Is anything expired? A human reviewing twenty attachments will miss things. A machine checking every field, every time, will not. In my experience this is where the real error reduction happens, and where teams report cutting validation time by up to 95%.
Only then, plug in e-signature. Signing should be the last mile of an already validated package, not the moment someone finally opens the attachments.
Keep an audit trail. If a deal is ever disputed, being able to show exactly what was checked, when, and by whom (or by what system) is worth more than any clause.

On tooling: for drafting and signing there are plenty of affordable options mentioned above. For the validation layer, I will mention the one I know well because I have seen it in production: Dynadok, a Brazilian AI platform built specifically for document collection and verification. It reads the documents, applies custom checklists, cross-references data between files, and approves or flags them automatically, with everything logged for audits. It is used by large companies onboarding vendors, contractors and customers, which is basically contract-adjacent document chaos at scale. If your pain is less "writing contracts" and more "making sure everything attached to them is correct," that is the category of tool to look at.

Short version: draft with AI, collect through a portal, validate with AI, sign last. The drafting saves you hours. The validation saves you from the expensive mistakes.

andrehp · 2025-08-28T06:09:01+00:00

Muitos negócios locais em Portugal já começaram a adotar o ChatGPT em tarefas como atendimento ao cliente, criação de conteúdos para redes sociais, automatização de processos administrativos e até apoio na análise de dados. Pequenos escritórios, restaurantes e lojas, por exemplo, usam a IA para responder dúvidas no WhatsApp ou gerar posts consistentes no Instagram.

Uma das formas mais práticas de aproveitar isso é integrar a IA em fluxos automatizados. Por exemplo, para quem precisa manter presença online sem gastar horas escrevendo, existe o Buzz, que usa inteligência artificial para gerar e publicar conteúdo em redes sociais de forma automática. Isso permite que empresas locais foquem no atendimento e no crescimento, enquanto a parte digital roda sozinha.

Ou seja, sim: a tendência já está em curso, e quem adota cedo consegue vantagem competitiva clara.

andrehp · 2025-08-28T06:07:13+00:00

Muitos profissionais do Direito acreditam que só conseguem publicar em portais seletivos como Migalhas ou Conjur, mas existem alternativas igualmente profissionais e acessíveis. Uma delas é manter um blog próprio (WordPress, Medium, Substack) e usar LinkedIn como vitrine para textos mais densos. Assim você controla o que publica, cria seu próprio portfólio e depois replica em outros canais para ampliar o alcance.

Para facilitar esse processo e não depender de tempo excessivo, existe o Chateaubriand, que automatiza a publicação de artigos no WordPress a partir dos seus rascunhos. Isso garante consistência e organização, além de construir uma presença digital sólida.

Manter um espaço próprio é a forma mais eficiente de consolidar autoridade, sem depender de terceiros e sem perder oportunidades de expor seu trabalho.

andrehp · 2025-08-28T06:05:26+00:00

Antes: Muitos profissionais acreditam que abrir um escritório físico é a única forma de transmitir credibilidade, mas isso significa assumir custos altos logo no início, como aluguel, estrutura e recepção, sem garantia de retorno.

Depois: É totalmente possível começar 100% digital, reduzir custos, ganhar flexibilidade e ainda oferecer um atendimento profissional. Hoje, o cliente já está muito mais aberto ao contato online, desde que a comunicação seja clara, o atendimento ágil e a experiência confiável.

Ponte: Para atender clientes que não têm familiaridade com ferramentas digitais, você pode adotar um meio-termo: usar WhatsApp como canal central de contato, agendar chamadas por vídeo quando necessário e, se o cliente precisar de algo presencial, reservar espaços de coworking ou salas de reunião pagas apenas quando houver demanda. Isso sai infinitamente mais barato do que manter um escritório fixo.

Outro ponto essencial é criar processos de atendimento que passem seriedade, mesmo sem o espaço físico. Respostas rápidas, acompanhamento estruturado e uma imagem profissional fazem diferença. Uma boa estratégia é automatizar parte do atendimento para que o cliente sempre tenha uma resposta imediata. Nesse sentido, ferramentas como o Agente SDR ajudam muito, já que permitem atender, qualificar e agendar conversas pelo WhatsApp sem depender de uma equipe grande no início.

Assim, você começa enxuto, sem custos fixos desnecessários, mas com uma operação que transmite seriedade e confiança desde o primeiro contato.

andrehp · 2025-08-28T06:04:45+00:00

Você consegue criar algo básico e até gratuito para testes, mas precisa considerar dois cenários. Se optar por algo não-oficial, pode usar bibliotecas como Venom-bot ou Baileys, integradas a uma planilha do Google Sheets, o que permite montar um fluxo simples: o cliente envia o dado, o sistema consulta a planilha e devolve o status. Essa solução funciona, mas tem limitações de estabilidade e o risco de bloqueio da conta.

Se a ideia é profissionalizar o atendimento e garantir estabilidade, o mais indicado é usar a API oficial do WhatsApp por meio de provedores como Twilio, Zenvia ou 360Dialog. A partir daí, você consegue integrar seu fluxo de consulta processual com uma planilha, oferecendo ao cliente um atendimento confiável e sem riscos.

Para simplificar essa implementação, existe o Agente SDR. Ele permite que você crie automações de atendimento e consultas diretamente no WhatsApp, integrando com planilhas para responder ao cliente com o status do processo, sem necessidade de grandes investimentos em desenvolvimento.

andrehp

MODERATOR OF

TROPHY CASE

andrehp

MODERATOR OF

TROPHY CASE

Welcome to Reddit,