Building an IDP tech stack

TL;DR

A fintech startup chains together three open-source tools and a cloud OCR API. Works in demo. In production with 10,000 documents a day and forty different formats, the pre-processing step silently drops 8% of documents. The validation layer flags nothing. Nobody notices for six weeks. That's what shipping an IDP stack without understanding the full seven layers costs you.

An IDP (Intelligent Document Processing) stack is not one tool. It's seven interdependent layers: ingestion, pre-processing, OCR, classification, extraction, validation, and integration. Most teams focus obsessively on layer three (OCR accuracy). The real work is everywhere else.

What is an IDP stack

IDP is the automation of end-to-end document workflows. It takes unstructured, paper-like inputs and outputs structured, actionable data. That sounds simple. The seven layers that make it work are anything but.

Think of IDP as a pipeline. At one end: email attachments, scanned images, PDFs, handwritten forms. At the other: validated records in your ERP, claims approved without human touch, invoices booked in under two minutes.

The seven layers sit between those two points

1. Ingestion: Where documents enter the system.
2. Pre-processing: Where images are cleaned.
3. OCR: Where text is extracted.
4. Classification: Where document type is determined.
5. Extraction: Where specific data is pulled out.
6. Validation: Where accuracy is enforced.
7. Integration: Where data moves downstream.

Each layer has different failure modes. Each needs a different technology. Most teams under-invest in layers 1, 2, 6, and 7 because they're not "sexy." Those layers cause 70% of production problems.

Why building an IDP stack is harder than it looks

You find five open-source tools. A cloud OCR API. A rule engine from GitHub. You stitch them together with webhooks and a message queue. In the lab, it works. The moment you move to production, you hit three invisible walls.

1. Format variation breaks assumptions: Your test set was invoices from one vendor. Production has invoices from four hundred. Invoice A is one page, B is eleven. A uses tables, B uses freeform text. A has the amount in the top-right, B in the bottom-left. Your OCR accuracy drops from 98% in test to 84% in production because the training data never saw vendor format C and D.

2. Open-source tools don't integrate: Tool A outputs XML. Tool B expects JSON. Tool C has no error handling. When tool B fails, there's no retry logic. Documents vanish silently into a dead letter queue. You discover the problem weeks later when a customer asks about missing data.

3. Validation is invisible until it breaks: You build a rule that checks if invoice amount matches purchase order. It works for ninety days, then a legitimate invoice arrives with a 3% price variance. Your rule flags it as fraudulent. The approval queue swallows it. Nobody approves it because they assume it failed. Three weeks pass. The vendor calls.

According to research from Tungsten Automation, approximately 30 to 40% of documents processed with OCR-only solutions still require human review because validation gaps and business rule exceptions aren't handled. Extraction is only one of nine steps in a full workflow, yet it gets 90% of the attention.

Integration complexity kills most builds. Teams assume they'll have two weeks to wire things together. It takes four. By then, you're out of budget and out of time.

The seven layers of a production IDP stack

Layer 1: Document ingestion and intake

This layer is about getting documents into the system. Email. Cloud storage (Dropbox, Google Drive, SharePoint). Batch uploads. SFTP. APIs. Fax gateways. Scanner integration.

Each source has metadata. The email has a sender. The scanner has a timestamp. The cloud folder has a filename. The API has a customer ID. Capturing that metadata now, at the ingestion point, saves you hours later.

Ingestion also includes queuing and deduplication. If the same invoice hits your email inbox three times, your system should catch it. It should queue documents in order. It should handle backpressure if the downstream layers get slow.

Most people skip this layer. They push documents directly to OCR. Then they're shocked when their system crashes under load or processes the same invoice twice.

Layer 2: Pre-processing and image enhancement

A scanned document is an image. That image is noisy. It's tilted at an angle. It might be dark. The text might be faint. OCR works best on clean images, but images are never clean.

Pre-processing is the art of making images cleaner without losing information. Deskewing rotates the image to vertical. Noise removal smooths the background. Contrast adjustment darkens the text. Rotation detection checks if the page is upside down.

This layer matters more than most people realize. A badly pre-processed image can tank OCR accuracy by 15 to 20 percentage points. Teams often underestimate how much work lives here.

Docsumo's OCR guide covers the technical details: adaptive thresholding, deskewing, noise reduction. The key insight is that these are image problems, not OCR problems. Your OCR engine should handle them, but they slow it down. Better to pre-process offline once than make the OCR engine work harder on every document.

Layer 3: OCR and digitisation

OCR extracts text from images. It reads the pixels and outputs characters. Modern OCR is good. Cloud APIs like AWS Textract or custom engines like Docsumo's proprietary OCR reach 95%+ accuracy on clean documents.

The trick is spatial awareness. Don't just extract text. Preserve layout. Know where the text was on the page. That metadata lets downstream layers know that the amount is in a table, or the signature is at the bottom, or the date is in the top-left corner.

Handwriting is harder than printed text. Handwritten amounts, signatures, and notes drop accuracy to 80 to 90% even on good models. Plan for that.

Docsumo's OCR benchmarking report shows how different engines perform on different document types. The key metric is not OCR accuracy in isolation. It's how accurate the extracted text is after it feeds into layer 4 (classification) and layer 5 (extraction).

Layer 4: Document classification

You have an invoice. Is it a purchase order invoice? A credit memo? An advance payment request? Each type has different fields. Your extraction model needs to know which template to use.

Classification is the gatekeeper. It routes documents to the right extraction schema. If classification is wrong, extraction fails downstream even if the OCR was perfect.

Classification can be rules-based (if the document says "CREDIT MEMO", flag it as credit-memo), ML-based (train a classifier on 1,000 labeled examples), or hybrid (start with rules, fall back to ML).

Rules work for stable, well-defined document types. Hybrid works for everything else. Pure ML without rules can work but requires lots of labeled training data and retraining when document formats drift.

Docsumo's guide on document classification walks through the technical approach: feature extraction, model selection, handling new document types.

Layer 5: Data extraction

Now you know it's an invoice. Layer 5 pulls the specific data out: invoice number, date, amount, line items, tax, vendor ID, account code.

Extraction can be template-based (this vendor always puts the amount in cell B15) or learned (train a model to find the amount anywhere). Template-based is fast but brittle. Learned extraction is flexible but needs training data.

Good extraction engines use natural language processing (NLP) to understand context. They know that "Total: 1,234.56" is an amount, not a date or a quantity. They handle currency symbols, decimals, and typos.

Schema-based extraction is the standard now. You define a schema: an invoice has a number (text), a date (YYYY-MM-DD), an amount (decimal), and line items (array of rows with quantity and unit price). The extraction engine fills the schema. If it can't find a field, it leaves it blank and flags it for review.

Docsumo's data extraction capabilities cover schemas, field types, handling tables, and Q&A extraction for unstructured fields.

Layer 6: Validation and business rules

Here's where extraction accuracy matters. But here's the secret: it's not about OCR accuracy. It's about business rule accuracy.

Your OCR engine extracted "1,234.56" perfectly. But layer 6 checks: does this amount match the purchase order? Is it within the contract terms? Does the vendor ID exist in your database? If all three checks fail, the document goes to manual review.

Validation layers have three types of rules:

1. Field-level rules: Amount must be a positive number. Date must be in the last 90 days. Vendor ID must be in the vendor master.
2. Cross-field rules: If invoice is marked as "final", there must be no open line items. If tax is zero, the document must be from a no-tax jurisdiction.
3. Cross-document rules: Invoice amount must match PO amount within 3%. If this vendor already has two pending invoices, flag for approval.

Most teams write rule one and stop. That's why they end up with 30 to 40% fallback rate in human review. Validation needs hundreds of rules, tested against real documents, updated as business logic changes.

The best validation layers also include anomaly detection. Machine learning finds documents that look unusual even though they pass all explicit rules. A 1 million dollar invoice to a new vendor during a holiday weekend gets flagged not because of a rule, but because the pattern is unusual.

Layer 7: Integration and output routing

Validated data goes downstream. To an ERP system via API. To an RPA bot that processes the approval. To a data lake for analytics. To an Electronic Document Management System (EDMS) for archival.

Integration is the hardest layer. Systems expect different formats. One system wants XML, another wants JSON. One wants the data pushed, another wants to pull it. One needs a callback when processing completes, another doesn't care.

Workflow state tracking matters here. You need to know: document arrived at 2pm, pre-processed at 2:03pm, OCR'd at 2:04pm, extracted at 2:05pm, validated at 2:06pm, approved at 3:15pm (manual review). When something breaks, you need an audit trail that shows where.

RPA integration is a specific case. An IDP system prepares data. An RPA bot consumes it. The IDP system might say "invoice is ready for approval" and pass it to the RPA bot, which logs into the ERP and books it. The RPA bot might say "approval succeeded" and pass that back to the IDP system. Both systems need to speak the same language.

Docsumo's RPA integration guide covers the patterns and integration points.

Build vs. buy decisions at each layer

You could build your entire IDP stack. You absolutely should not.

Here's the decision table:

‍

Manual processing: 4 to 6 hours per mortgage file. IDP processing: under 2 hours. That's 70% faster. The buy decision paid for itself in the first month.

Extraction is the visible layer. It's where 90% of conversation happens. But a build-it-yourself extraction layer without proper validation, integration, and error handling is a liability, not an asset.

Common mistakes when assembling an IDP stack

Teams make five repeating mistakes. Avoid them.

Mistake One: Under-investing in pre-processing

You decide OCR accuracy is the bottleneck, so you license the most expensive OCR engine. Your input images are still scanned at 150 DPI with harsh shadows. Pre-processing can't fix that. You needed to address document quality upstream. Spend 20% of your budget on pre-processing, not 5%.

Mistake Two: Treating validation as optional

You build extraction to 95% accuracy and ship it. The 5% of failures will be caught by your validation rules. Except you only built ten rules. Production needs five hundred. Most failures slip through and hit your ERP. Build validation first. Extract second.

Mistake Three: Ignoring exception handling

A document fails classification. What happens? If the answer is "it sits in a queue forever", you've built a time bomb. Every system needs a dead letter queue, a timeout, and a way to route exceptions to humans. Design for failure, not perfection.

Mistake Four: No audit trail

A month after go-live, a customer says their invoice wasn't processed. You have no way to debug it. Did it arrive? Was it classified? Where did it fail? An audit trail showing timestamp, layer, status, and error message is not a luxury. It's table stakes.

Mistake Five: Testing only on clean samples

You test on ten invoices from your biggest customer. You go to production. The second day brings invoices from five new vendors, three of them in languages you didn't expect. Image quality varies wildly. Your extraction accuracy drops from 95% to 73%. You built a system that works on data you've already seen, not on production data.

How to evaluate IDP stack components before committing

Before you buy or build any component, run a pilot.

1. Assemble real documents 

Collect real documents from the next 30 days of production. Aim for 1,000 documents that represent the full distribution: difficult ones, odd ones, ones in languages you underestimated.

2. Measure end-to-end accuracy, not component accuracy

Your OCR engine might be 98% accurate on a sample test set. That doesn't mean extracted data is 98% accurate. Measure the full pipeline: document ingestion through final output. That number matters.

3. Define your STP (straight-through processing) threshold

STP is the percentage of documents that go through without human review. For invoicing, 90%+ STP is achievable with good validation. For complex claims, 70 to 80% might be realistic. Know your threshold before you start.

4. Prototype integration paths

Can the tool output JSON? XML? Will it integrate with your message queue? Your ERP? Test these paths before you commit budget.

5. Run a real pilot

Process 1,000 documents. Count the ones that fail at each layer. Count the ones that pass but are wrong (silently). Count the ones that pass correctly. This tells you if the tool is production-ready for your use case.

How Docsumo covers the full IDP stack

Docsumo is built to cover all seven layers in a single platform. You don't integrate seven tools. You configure one system.

Layer 1 - Ingestion: Docsumo ingests from email, cloud storage (Dropbox, Google Drive, SharePoint), APIs, and SFTP. Metadata is captured automatically. Learn more about automated document processing.

Layer 2 - Pre-processing: Docsumo applies deskewing, noise removal, contrast enhancement, and rotation detection out of the box. No separate tool needed.

Layer 3 - OCR: Docsumo uses a proprietary OCR engine with spatial awareness and layout preservation. It handles printed text, handwriting, and table structures. 

Layer 4 - Classification: Docsumo's classification engine uses machine learning and rules. You can train on your documents or start with pre-trained models for common document types (invoices, purchase orders, contracts, etc.).

Layer 5 - Extraction: Docsumo uses schema-driven extraction. You define the fields you want. Docsumo finds them. It handles key-value pairs, tables, line items, and nested structures. Learn more about data extraction.

Layer 6 - Validation: Docsumo includes a rule engine. Define field-level rules, cross-field rules, and cross-document rules. Anomaly detection flags unusual patterns. Read more on intelligent document processing.

Layer 7 - Integration: Docsumo integrates with ERPs, data lakes, RPA tools (UiPath, Automation Anywhere), and EDMSs. APIs are REST and webhooks. RPA integration patterns.

The result: invoices arrive in email. Docsumo processes them end-to-end. Validated invoices hit your ERP. Failed ones are routed to an approval queue. An RPA bot books approved invoices. Everything is audited.

Real-world use case: invoice processing automation cuts processing time from 4+ hours (manual) to under 30 minutes (automated).

For a deeper look at the platform, try Docsumo's agentic document processing platform for free.

Final thoughts

An IDP stack is powerful because it's all-encompassing. It's weak because it's intricate. You can build it piecemeal, but you'll spend time and money integrating seven fragile seams. Or you can use a platform that handles all seven layers and spend time on business logic, not plumbing.

The fintech startup in the opening learned the hard way. Their pre-processing layer was dropping 8% of documents. Nobody noticed because their validation layer had a blind spot. Six weeks of silent data loss. That's a real cost: customer trust, operational risk, remediation effort.

Docsumo's intelligent document processing solutions simplify this complexity. Don't build your own IDP stack unless you have the team, the budget, and the patience. If you do, use this article as your architecture guide. Test aggressively. Invest in validation and integration. And remember: extraction is one step out of seven. It's not the hardest one.

For workflow-specific implementation details, see Docsumo's intelligent document processing workflow guide.

According to AWS guidance on intelligent document processing, modular, API-first architectures are the future. No single tool does everything well. But a platform designed around integration patterns makes all seven layers work together.

FAQs

What's the difference between OCR and IDP?

OCR is layer 3 only. It reads images and outputs text. IDP is layers 1 through 7. It ingests, pre-processes, classifies, extracts, validates, and integrates. More on OCR vs. IDP.

Can we just use an open-source OCR tool?

Yes. You own the other six layers. That's two to six months of development, testing, and integration. Plus ongoing maintenance. Plan accordingly.

What's a realistic STP rate?

90%+ is achievable with proper validation and business rules. 95%+ is reachable with human-in-the-loop review of flagged items. Anything above that usually means your validation rules are too loose (you're missing real problems).

How long does it take to build an IDP stack?

Build from scratch: 3 to 6 months for a single use case (like invoices). Platform-based implementation: 2 to 4 weeks for a single use case. Integration with downstream systems adds 2 to 8 weeks either way.

Do we need machine learning for classification?

Not always. If you have 3 to 5 stable document types (invoice, credit memo, PO), rules work fine. Machine learning becomes useful when you have 10+ types or formats that drift over time. Hybrid (rules first, ML fallback) is the sweet spot.

What happens to documents that fail validation?

They go to a manual review queue. A human reviews the flagged fields, corrects them if needed, and approves or rejects. Some systems route to different queues based on the failure type (data quality issues to QA, missing vendor to procurement, etc.). The key is that nothing silently fails. Everything has a visible path.