Freelance IDP Engineer — automated document pipelines for businesses. PDF extraction, OCR, invoice automation. Based in Durham, UK.https://subhajitbhar.com/https://subhajitbhar.com/blog/idp/aws-textract-alternatives/https://subhajitbhar.com/blog/idp/aws-textract-alternatives/AWS Textract works on standard document types and integrates cleanly into AWS infrastructure. Where it falls short: domain-specific layouts, high-volume cost, and silent extraction failures with no built-in routing to human review.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/azure-document-intelligence-alternatives/https://subhajitbhar.com/blog/idp/azure-document-intelligence-alternatives/Azure Document Intelligence works well on standard document types. Here's where it falls short, and when a custom extraction pipeline is the better choice.Wed, 04 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/azure-document-intelligence-vs-custom-pipeline/https://subhajitbhar.com/blog/idp/azure-document-intelligence-vs-custom-pipeline/Azure Document Intelligence works well for standard document types. A custom pipeline is the better choice when edge cases matter, accuracy requirements are high, or layout variation is significant.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/certificate-of-analysis-extraction/https://subhajitbhar.com/blog/idp/certificate-of-analysis-extraction/Certificates of analysis are legally significant documents. Extracting data from them reliably — across multiple laboratory formats — requires more than regex or a cloud API. Here's what a production CoA extraction pipeline looks like.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/contract-data-extraction/https://subhajitbhar.com/blog/idp/contract-data-extraction/Contracts are the least standardised document type in most organisations. Extracting dates, parties, obligations, and clauses reliably — across varied formats — requires a different approach from invoice or form extraction.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/cost-of-manual-document-processing/https://subhajitbhar.com/blog/idp/cost-of-manual-document-processing/Manual document processing costs more than the staff hours spent on it. Errors, delays, and growth bottlenecks add up. Here's how to calculate what it's actually costing your business — and what the automation ROI looks like.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/customs-declaration-extraction/https://subhajitbhar.com/blog/idp/customs-declaration-extraction/Customs declarations are high-stakes, structured documents — but their extraction presents specific challenges: multilingual fields, tariff code complexity, and zero tolerance for errors. Here's what a production extraction pipeline looks like.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/docsumo-alternatives/https://subhajitbhar.com/blog/idp/docsumo-alternatives/Docsumo is a solid platform for financial document extraction, but its vertical focus is also its ceiling. Here's where it falls short and when an alternative makes more sense.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/document-processing-pipeline-llm-python/https://subhajitbhar.com/blog/idp/document-processing-pipeline-llm-python/A raw LLM call that extracts invoice data works in demos. It fails in production. The difference is the pipeline around it: schema-first design, rules baseline, confidence scoring, and human review. Here's what that looks like.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/confidence-scoring-document-extraction/https://subhajitbhar.com/blog/idp/glossary/confidence-scoring-document-extraction/Confidence scoring assigns a reliability score to each extracted field in a document pipeline. It's how production IDP systems distinguish trustworthy extractions from ones that need human review.Sun, 08 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/document-automation/https://subhajitbhar.com/blog/idp/glossary/document-automation/Document automation is the use of software to replace manual document handling — either generating documents from data, or extracting data from incoming documents. The second category is harder and more valuable for most businesses.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/document-classification/https://subhajitbhar.com/blog/idp/glossary/document-classification/Document classification is the step that identifies what type of document has arrived before extraction begins. In pipelines that handle multiple document types, it's what routes each document to the right extraction logic.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/document-extraction-pipeline/https://subhajitbhar.com/blog/idp/glossary/document-extraction-pipeline/A document extraction pipeline is the end-to-end system that takes documents in and produces structured data out. This is what separates a one-off extraction script from something that runs reliably in production.Sun, 08 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/document-validation/https://subhajitbhar.com/blog/idp/glossary/document-validation/Document validation is the step that checks whether extracted data is correct before it passes downstream. Without it, extraction errors propagate silently into your databases, reports, and compliance records.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/human-in-the-loop-document-processing/https://subhajitbhar.com/blog/idp/glossary/human-in-the-loop-document-processing/Human-in-the-loop (HITL) in document processing means routing uncertain extractions to a human reviewer before they pass downstream. It's not a fallback — it's an explicit design decision that makes automated extraction trustworthy.Sun, 08 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/layout-variation/https://subhajitbhar.com/blog/idp/glossary/layout-variation/Layout variation is when the same document type arrives in different formats from different sources — or from the same source at different times. It's the primary reason document extraction scripts break in production.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/ocr-post-processing/https://subhajitbhar.com/blog/idp/glossary/ocr-post-processing/OCR converts images to text, but raw OCR output contains errors — misread characters, broken words, formatting noise. OCR post-processing cleans and corrects this output before extraction logic runs against it.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/schema-first-extraction/https://subhajitbhar.com/blog/idp/glossary/schema-first-extraction/Schema-first extraction means defining exactly what you want to extract before you touch a single document. It's the design principle that separates reliable production pipelines from scripts that break on edge cases.Sun, 08 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/straight-through-processing/https://subhajitbhar.com/blog/idp/glossary/straight-through-processing/Straight-through processing is the automated handling of a document or transaction from start to finish without manual intervention. In IDP, it's the goal — but only for extractions the system is confident about.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/structured-vs-unstructured-documents/https://subhajitbhar.com/blog/idp/glossary/structured-vs-unstructured-documents/Structured documents have predictable, machine-readable layouts. Unstructured documents present information in free-form natural language. The distinction determines how you approach extraction — and how hard the problem is.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/table-extraction-pdf/https://subhajitbhar.com/blog/idp/glossary/table-extraction-pdf/Table extraction from PDFs is the process of identifying tabular structures in a document and converting them into structured data. It's one of the most technically demanding extraction problems because PDF tables have no single standard format.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/glossary/what-is-ocr/https://subhajitbhar.com/blog/idp/glossary/what-is-ocr/OCR converts images of text into machine-readable characters. It's the first step in processing scanned documents — but it's not the same as extracting structured data from them.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/google-document-ai-alternatives/https://subhajitbhar.com/blog/idp/google-document-ai-alternatives/Google Document AI handles standard document types well, but GCP lock-in, per-page pricing, and limited control over failures make it the wrong fit for many production pipelines. Here's what to consider instead.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/how-to-choose-idp-solution/https://subhajitbhar.com/blog/idp/how-to-choose-idp-solution/Three paths to document automation: SaaS IDP platforms, cloud provider APIs, or a custom-built pipeline. Each fits different document types, volumes, and accuracy requirements. Here's how to decide.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/intelligent-document-processing-environmental-consultancies/https://subhajitbhar.com/blog/idp/intelligent-document-processing-environmental-consultancies/Environmental and water consultancies process hundreds of lab reports, monitoring results, and compliance documents every month. Most of that data entry is still manual. Here's what IDP looks like in this context — and what it actually takes to make it reliable.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/intelligent-document-processing-legal/https://subhajitbhar.com/blog/idp/intelligent-document-processing-legal/Legal teams spend significant time extracting data from contracts, NDAs, and filings manually. IDP handles the extraction — but only if it's built to fail loudly, not silently. Here's what that looks like in practice.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/intelligent-document-processing-logistics/https://subhajitbhar.com/blog/idp/intelligent-document-processing-logistics/Logistics and customs operations run on documents. Bills of lading, customs declarations, commercial invoices, certificates of origin — each one processed manually is a delay and a risk. Here's how IDP handles them.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/invoice-data-extraction-python/https://subhajitbhar.com/blog/idp/invoice-data-extraction-python/Extracting data from one invoice format with Python is straightforward. Handling 20 different supplier formats reliably in production is a different problem. Here's what the production approach looks like.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/lab-report-data-extraction-python/https://subhajitbhar.com/blog/idp/lab-report-data-extraction-python/Lab reports from different testing laboratories look completely different. The same test results appear in different table structures, different units, different column names. Here's what reliable extraction looks like across multiple lab formats.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/nanonets-alternatives/https://subhajitbhar.com/blog/idp/nanonets-alternatives/Nanonets works well for AP automation on standard document types. When your documents are domain-specific, your volume is high, or you need to understand why extractions fail, you need something else.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/ocr-vs-intelligent-document-processing/https://subhajitbhar.com/blog/idp/ocr-vs-intelligent-document-processing/OCR converts images to text. IDP extracts structured data from documents. They're related but not interchangeable — and confusing them leads to building the wrong solution.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/purchase-order-extraction/https://subhajitbhar.com/blog/idp/purchase-order-extraction/Purchase order extraction looks simple until you're handling POs from dozens of customers with different formats, ERP systems, and approval workflows. Here's how a production PO extraction pipeline is structured.Thu, 12 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/what-is-document-automation/https://subhajitbhar.com/blog/idp/what-is-document-automation/Document automation is the practice of replacing manual document handling with software. It covers a wide range of use cases — from generating documents automatically to extracting data from incoming ones. Here's how to think about it.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/what-is-intelligent-document-processing/https://subhajitbhar.com/blog/idp/what-is-intelligent-document-processing/A plain-English guide to IDP: what it is, how it works, when you need it, and how to evaluate your options — from OCR and rules-based extraction to LLM-augmented pipelines.Wed, 04 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/idp/why-document-automation-breaks-on-edge-cases/https://subhajitbhar.com/blog/idp/why-document-automation-breaks-on-edge-cases/Simple scripts, enterprise platforms, and LLM-only extraction all break the same way: clean documents work, real ones don't. Here's why, and what production pipelines do differently.Wed, 04 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/pdf-extraction/extract-data-scanned-pdf-python/https://subhajitbhar.com/blog/pdf-extraction/extract-data-scanned-pdf-python/Scanned PDFs contain images, not text. pdfplumber returns nothing. You need an OCR layer first, then extraction logic on top. Here's the full pipeline.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/pdf-extraction/extract-tables-from-pdf-python/https://subhajitbhar.com/blog/pdf-extraction/extract-tables-from-pdf-python/Table extraction from PDFs in Python using pdfplumber — bordered and borderless tables, multi-page tables, post-processing, and the edge cases that break naive approaches.Wed, 04 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/pdf-extraction/handle-pdf-layout-variations-python/https://subhajitbhar.com/blog/pdf-extraction/handle-pdf-layout-variations-python/An extraction script that works on one supplier's invoice breaks on another's. The same document type, different layout. Here's how to build extraction logic that handles multiple formats without a separate script for each.Wed, 11 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/pdf-extraction/pdfplumber-vs-pymupdf-vs-pypdf2/https://subhajitbhar.com/blog/pdf-extraction/pdfplumber-vs-pymupdf-vs-pypdf2/A practical comparison of pdfplumber, PyMuPDF (fitz), and PyPDF2 — with code examples and a decision framework for production use.Wed, 04 Mar 2026 00:00:00 GMThttps://subhajitbhar.com/blog/pdf-extraction/schema-first-pdf-extraction-pydantic/https://subhajitbhar.com/blog/pdf-extraction/schema-first-pdf-extraction-pydantic/Define your output schema before writing extraction logic. Schema-first PDF extraction with Pydantic — the approach that keeps production pipelines maintainable.Wed, 04 Mar 2026 00:00:00 GMT