What is semi-structured data? Definition, examples and extraction

Semi-structured data is data that has some organization, such as tags, keys or labels that say what each value means, but doesn't follow a fixed schema of rows and columns. JSON, XML, emails and log files are classic examples. In business operations, documents such as invoices, bank statements and pay stubs are also semi-structured: they always contain the same kinds of fields, but every issuer lays them out differently.

This guide covers how semi-structured data compares with structured and unstructured data, common examples, where it's stored, and how to extract it, both from machine formats and from documents.

Structured vs semi-structured vs unstructured data#

For a longer comparison, read structured vs unstructured vs semi-structured data.

Examples of semi-structured data#

JSON

JSON stores values as key-value pairs and supports nesting and lists. APIs, web apps and IoT devices use it everywhere.

{
  "invoice_number": "INV-20431",
  "vendor": {"name": "Acme Supplies", "tax_id": "12-3456789"},
  "line_items": [
    {"description": "Paper, A4", "qty": 10, "amount": 45.00},
    {"description": "Toner", "qty": 2, "amount": 120.00, "discount": 10.00}
  ]
}

Notice that only the second line item has a discount. That's the flexibility, and the challenge, of semi-structured data.

XML

XML uses opening and closing tags to describe a hierarchy. It's common in older enterprise integrations, e-invoicing standards and configuration files.

<invoice number="INV-20431">
  <vendor>Acme Supplies</vendor>
  <total currency="USD">155.00</total>
</invoice>

YAML

A human-readable format for configuration files and data exchange, using indentation for nesting.

HTML

Web pages mark up content with tags. The structure describes presentation more than meaning, which is why extracting data from HTML usually needs scraping rules.

Emails

Headers (from, to, date, subject) are structured; the body is free text; attachments can be anything.

Log files

Each line usually follows a pattern, but fields vary by event:

2026-08-14 10:15:20 INFO user_login username=jdoe ip=192.168.1.100
2026-08-14 10:15:24 WARN payment_retry order=88731 attempt=2

Business documents

Invoices, bank statements, pay stubs, tax forms and insurance certificates carry predictable fields in unpredictable layouts. A bank statement always has an account number, a period, opening and closing balances and transactions, but there are thousands of bank templates. These documents are the largest source of semi-structured data most finance and operations teams deal with.

Why semi-structured data is useful#

• Flexible schema. New fields can be added without redesigning a database.
• Self-describing. Keys and tags travel with the values, so data is easier to exchange between systems.
• Handles nesting. An invoice with many line items, or a customer with many addresses, fits naturally.
• Good for fast-changing sources. APIs and event streams evolve without breaking storage.

The cost is that analysis and reporting usually need the data flattened into consistent tables first.

How semi-structured data is stored#

• Document databases such as MongoDB store each record as a JSON-like document with its own shape.
• Relational databases with JSON columns, such as PostgreSQL's jsonb type, mix fixed columns with flexible ones.
• Data warehouses support semi-structured types so JSON can be queried with SQL.
• Object storage and data lakes keep raw files (JSON, XML, CSV, PDFs) cheaply until they're processed.

How to extract semi-structured data#

From machine formats: parse and normalize

For JSON, XML and logs, parsers do the reading. The real work is normalization: flattening nested objects, handling optional fields, and producing consistent rows. This Python example (standard library only, Python 3.8+) turns the JSON invoice above into one row per line item:

import json

raw = """{
  "invoice_number": "INV-20431",
  "vendor": {"name": "Acme Supplies", "tax_id": "12-3456789"},
  "line_items": [
    {"description": "Paper, A4", "qty": 10, "amount": 45.00},
    {"description": "Toner", "qty": 2, "amount": 120.00, "discount": 10.00}
  ]
}"""

invoice = json.loads(raw)
rows = [
    {
        "invoice_number": invoice["invoice_number"],
        "vendor": invoice["vendor"]["name"],
        "description": item["description"],
        "qty": item["qty"],
        "amount": item["amount"],
        "discount": item.get("discount", 0.0),  # optional field
    }
    for item in invoice["line_items"]
]
for row in rows:
    print(row)

Output:

{'invoice_number': 'INV-20431', 'vendor': 'Acme Supplies', 'description': 'Paper, A4', 'qty': 10, 'amount': 45.0, 'discount': 0.0}
{'invoice_number': 'INV-20431', 'vendor': 'Acme Supplies', 'description': 'Toner', 'qty': 2, 'amount': 120.0, 'discount': 10.0}

From documents: OCR, models and validation

A PDF or scanned invoice has no keys to parse. Getting to the JSON above takes several steps:

1. Classify the document (invoice, bank statement, pay stub).
2. Read the page with OCR if it's a scan.
3. Locate fields and tables with models trained on many layouts, so one model reads invoices from thousands of vendors.
4. Validate the values: line items add up to the total, dates are valid, the vendor exists in your master data.
5. Review low-confidence fields with a person.
6. Output structured JSON to your systems.

This is what intelligent document processing platforms do. Docsumo reads 250+ document types, including invoices, bank statements and pay stubs, with 99% field-level accuracy, and returns JSON through its API and webhooks.

Challenges with semi-structured data#

• Schema drift. New or renamed keys break downstream code. Validate against a schema and alert on changes.
• Nesting. Deeply nested data is awkward in SQL and spreadsheets. Decide the grain (for example, one row per line item) up front.
• Inconsistent types. The same field can arrive as a number in one record and a string in another.
• Layout variety in documents. Templates don't scale across thousands of issuers; models do.
• Quality. Missing or wrong values need to be caught before they reach reports. Build in validation and review.

The bottom line#

Semi-structured data has labels but no fixed shape. For machine formats, parse it and normalize it into consistent tables. For business documents, which are the harder and more common case, use OCR and machine learning to find the fields, then validate them before they reach your systems.

Frequently asked questions#