PDF Data Extraction Pipeline

This project tackled the pain of turning messy, mixed-layout documents into structured, searchable data—reliably and at scale. The solution provides an end-to-end pipeline for ingesting files, extracting text and structure, normalizing reading order, and syncing results back to SharePoint. I worked on the architecture, services and UI, Azure and Microsoft Graph integration, deployment, monitoring, and iteration. Along the way I hit (and solved) real-world hurdles around OCR fidelity, container memory limits, etc.

The system now converts large, varied PDFs (100-200+ pages) into clean JSON + text with coordinates, preserves tables, and enriches SharePoint with metadata—while remaining observable, maintainable, and secure.

This started as a practical need: teams were wasting valuable time extracting data manually from PDFs with high page counts. Search was brittle, and keying data into excel introduced delays. I wanted a pipeline that could accept anything we threw at it and produce consistent, trustworthy output.

The first challenge was choosing the extraction core. I evaluated several engines and picked tesseract for the initial OCR, then Azure AI Document Intelligence for layout awareness, tables, and stable bounding boxes. Pairing that with cosmos db for a reliable data store and retreival

Finally, I automated the entire the pipeline to where a watch folder lilves in SharePoint where a user can drop a PDF and in an interval of minutes that folder will be checked and processed, moved into a new folder and uploaded the extracted info to a SharePoint list. The key engineering story here is how I gained great experience in various fields... Cloud Engineering, Bakcend Programming/scripting in python, and automation engineering—all end-to-end.

Document Variability & Reading Order

Real documents aren’t linear. Multi-column layouts, headers/footers, stamps, and tables break naïve “top-to-bottom” text extraction.

Resource Constraints in Containers

The OCR pipeline (PDF rasterization, image preprocessing, layout analysis) spikes memory—especially on long PDFs. In Azure App Containers, RAM swap is not supported, leading to container crashes; I hit OOM kills and kernel thrash on heavy jobs.

SharePoint Integration

Graph API auth, SharePoint permissions, mapping to a list.

Reliable Extraction Across Layouts

Preserve structure (pages, lines, words, tables, selection marks) and deliver coordinate-accurate tokens for later use to extract data.

Scalable, Cost-Aware Compute

Right-size OCR workers to handle large and spiky jobs without runaway costs—or random failures due to memory pressure. Using tesseract on a VM, I was able to leverage RAM swap for big PDFs, although it takes longer it is a cost effective solution. Then later using azure document intelligence to analyze data on select pages reduces the costly service by delegating large ocr tasks to a cheaper service via VM, considering the scale of docs processed.

Deterministic Reading Order Normalization

When the OCR engine’s “content order” and “bounding-box order” disagree, apply stable rules so output is predictable and testable... Using the bounding box approach was the stable solution

Seamless SharePoint Round-Trip

Ingest from SharePoint, process, then write back data to a list and maintain a scalable solution

Architecture / Software

The system consists of: a collection of python/powershell scripts to create the pipeline. I initially containerized everything, but OCR workers routinely exceeded container memory (PDF rasterization + layout inference). After profiling attempts, I split the deployment: local processing on VM/ Document Intelligencethis workload split ensured cost effectiveness as well as relieving any previous system constraints like OOMs and iprevented over-provisioning the entire stack.

Extraction

Tesseract offers the first OCR passthrough allowing for unique keyword searches, reducing the pages for analysis from 100-200+ to under 15 pages. These pages are then sent to Azure Document Intelligence OCR tool for better OCR quality that is then used in the pulling of nessesary data

A user drops files into a SharePointfolder. The pipeline at an interval of n minutes and queues the doc(s) in the folder. An OCR worker (on the VM) pulls the job, downloads the file, hashes it (for deduplication), runs tessearact and writes raw results to cosmos db. The pipeline then grabs unique words to identify pages for extraction and writes these pages to cosmos for later reference. Then these pages are sent to Azure Document Intelligence OCR for second pass through OCR. The normalization layer reconciles reading order with bounding boxes. The pipeline then extracts the datapoints required by the business using keywords and bounding boxes to pick up the values needed and uploads them to cosmos db. The last step in the pipeline then updates a SharePoint list with the newly extracted data and moves the file to a processed folder.

Container OOM

Container scaling was not practical for this use case, I moved OCR workers to a RAM-rich VM(which supports swap if needed), added queueing and batch processing to process in paralell while remaining in bounds of the VMs RAM to avoid swap

Reading order vs bbox conflicts

Being quite new to the nuances of OCR I quickly learned that the text output of OCR systems aren't always accurate. I took this approach initially but the accuracy of the system failed as I was testing, it was not working at scale, led to reduced accuracy! I switched over to bounding boxes for a substantial increase in accuracy. By finding the key all I needed was to follow the x or y axis to get the value, this led to a substantial accuracy gain