What is file parsing? Techniques, tools and Python examples

File parsing reads what is inside a file by the rules of its format. It turns the content into structured data, such as rows or labeled fields, that a program can use. That means rows from a CSV, objects from JSON, elements from XML, or text and fields from a PDF. It's the first step in almost every data pipeline. Data arrives as files long before it reaches a database or report.

What is file parsing?#

A parser knows the rules of a format and uses them to break a file into meaningful parts. A CSV parser knows that commas separate fields and newlines separate records, and that a comma inside quotes is data, not a separator. A JSON parser knows about objects, arrays and types. A PDF parser knows how to find text objects on each page.

Parsing is usually one of three steps that together turn files into data you can load, analyze or act on.

• Parsing

  Reading the file by its format's rules into rows, objects, elements or page text.

• Extraction

  Picking out the specific values you need, such as the invoice total.

• Transformation

  Cleaning and reshaping values, such as converting "4,120.50" to a number or dates to one format.

Parsing by file type#

Every common format has a standard parser. What differs is where it goes wrong.

Python examples for common formats#

This script parses a CSV, a JSON file, an XML file and a digital PDF. It was tested on Python 3.11 with pypdf 6.19 (pip install pypdf); everything else is in the standard library.

import csv
import json
import re
import xml.etree.ElementTree as ET

from pypdf import PdfReader

# CSV: one dict per row
with open("invoices.csv", newline="", encoding="utf-8") as f:
    rows = list(csv.DictReader(f))
print(rows[0])

# JSON: nested keys
with open("invoice.json", encoding="utf-8") as f:
    doc = json.load(f)
print(doc["vendor"]["name"], doc["total"])

# XML: attributes and child elements
root = ET.parse("invoice.xml").getroot()
print(root.get("number"), root.findtext("vendor"), root.find("total").get("currency"))

# PDF with a text layer: extract text, then find fields
text = PdfReader("invoice.pdf").pages[0].extract_text()
total = re.search(r"Total Due:\s*\$([\d,]+\.\d{2})", text)
print(total.group(1) if total else "not found")

Here is the output from a small file in each format.

{'invoice_number': 'INV-20431', 'vendor': 'Acme Supplies', 'total': '4120.50'}
Acme Supplies 4120.5
INV-20431 Acme Supplies USD
4,120.50

Two things to notice. The CSV parser returns every value as a string ('4120.50'), so you still need to convert types. And the PDF step only works because we knew the exact label "Total Due". A different vendor's invoice would need a different pattern.

Parsing techniques and best practices#

Imagine a partner sends you a new copy of the invoices.csv from the script above every night. Your first version splits each line at every comma. It works until a vendor name arrives in quotes with a comma inside the name. A real parser, here csv.DictReader or pandas, knows that a quoted comma is data. Splitting HTML with a regex fails the same way when tags nest. Then a file arrives from an older system, and the accented names show the wrong characters. A text encoding is the way a file stores its letters. State the encoding every time you open a file, usually UTF-8. Decide in advance what to do with characters that can't be read.

Next, check each record as it arrives. Required fields must be present, types must convert and amounts must fall in range. Send a record that fails to a reject file, with the reason beside it. Log the error, keep going and report the counts at the end, so one bad row doesn't stop the night's run. The file will keep growing, so don't load all of it into memory. Read the CSV in chunks, for example pandas.read_csv(..., chunksize=100_000), use iterparse for large XML, and write the output as you go.

Treat any file from outside your organization as untrusted input. Python's documentation warns that its XML modules aren't secure against maliciously constructed data. Use a hardened parser such as defusedxml for those files. Keep field mappings in configuration, so a renamed column is a config change, not a code change. Finally, keep a set of tricky real files as regression tests, so you can rerun them after every change. Good test files have empty fields, unusual encodings or a very large size.

Where we land. Test on your worst files first, because the tidy file you wrote the parser against proves little. The file from the oldest system, with its odd encoding, shows what will fail, so keep it as a regression case. The same goes for documents. A phone photo, a long statement or a handwritten page tells you far more about a tool than a clean digital copy does.

Common file parsing problems#

• Schema drift

  A partner adds, renames or reorders columns. Validate headers before processing.

• Inconsistent formats

  Dates as 08/14/2026 in one file and 2026-08-14 in the next. Normalize on the way in.

• Tables in PDFs

  Text extraction loses table structure. Use a table-aware tool, or see table extraction from PDF.

• Scanned documents

  There's no text to parse. You need OCR, and then something that understands layout.

• Many layouts of the same document

  Invoices, bank statements and pay stubs come from thousands of issuers, each with its own layout. You can't handle them by writing one pattern for each layout.

When parsing isn't enough: documents#

Parsers work well when files are designed for machines. Business documents are designed for people. An invoice PDF has the invoice number somewhere on the page, and every vendor labels it differently. A scanned bank statement has no text at all until OCR reads it.

For these, intelligent document processing replaces rules written by hand. It classifies each file and runs OCR where needed. Models trained on many layouts find the fields and tables. The software then validates the values and returns structured JSON.

Docsumo does this for 250+ document types, including invoices and bank statements. Results come back through an API and webhooks, which are ways for one program to send data to another. To your other systems, the output looks like any other parsed file.

• 99%99%field-level accuracy across 250+ document types
• 95%+95%+of documents processed straight through, without manual review

For more on the document side, read document parsing and PDF parser.

When to stop writing patterns by hand#

Parse machine formats with standard libraries. Check every record and read large files in parts. When the input is a PDF, scan or photo in many layouts, you'd need a pattern for each one. Use OCR plus document AI instead to get the same structured output.

Book a demo with a few of your own documents, or start a free trial.

Frequently asked questions#