What is semi-structured data? Definition, examples and extraction

Semi-structured data is data that has some organization, such as tags, keys or labels that say what each value means, but doesn't follow a fixed schema of rows and columns. JSON, XML, emails and log files are classic examples. Business documents such as invoices, bank statements and pay stubs are semi-structured too: they always contain the same kinds of fields, but every issuer lays them out differently.

Structured vs semi-structured vs unstructured data#

For the full side-by-side, read structured vs unstructured vs semi-structured data.

Examples of semi-structured data#

• JSON

  Key-value pairs with nesting and lists. The format of most APIs, web apps and IoT devices.

• XML

  Opening and closing tags describe a hierarchy. Common in enterprise integrations, e-invoicing and configuration.

• YAML

  Indentation-based and human-readable, used for configuration files and data exchange.

• HTML

  Tags describe presentation more than meaning, so pulling data from web pages needs scraping rules.

• Email

  Headers (from, to, date, subject) are structured; the body is free text; attachments can be anything.

• Log files

  Each line follows a pattern, but the fields vary by event.

• Business documents

  Invoices, bank statements, pay stubs and tax forms carry predictable fields in unpredictable layouts.

JSON example

{
  "invoice_number": "INV-20431",
  "vendor": {"name": "Acme Supplies", "tax_id": "12-3456789"},
  "line_items": [
    {"description": "Paper, A4", "qty": 10, "amount": 45.00},
    {"description": "Toner", "qty": 2, "amount": 120.00, "discount": 10.00}
  ]
}

Only the second line item has a discount. That's the flexibility, and the challenge, of semi-structured data.

XML example

<invoice number="INV-20431">
  <vendor>Acme Supplies</vendor>
  <total currency="USD">155.00</total>
</invoice>

Log file example

2026-08-14 10:15:20 INFO user_login username=jdoe ip=192.168.1.100
2026-08-14 10:15:24 WARN payment_retry order=88731 attempt=2

A bank statement is the document version of the same idea: it always has an account number, a period, opening and closing balances and transactions, but thousands of banks each use their own template.

Why semi-structured data is useful#

• Flexible schema. New fields can be added without redesigning a database.
• Self-describing. Keys and tags travel with the values, so data moves easily between systems.
• Handles nesting. An invoice with many line items fits naturally.

The cost is that reporting usually needs the data flattened into consistent tables first.

How semi-structured data is stored#

• Document databases such as MongoDB store each record as a JSON-like document with its own shape.
• Relational databases with JSON columns, such as PostgreSQL's jsonb type, mix fixed columns with flexible ones.
• Data warehouses support semi-structured types, so JSON can be queried with SQL.
• Object storage and data lakes keep raw files (JSON, XML, CSV, PDFs) until they're processed.

How to extract semi-structured data#

From machine formats: parse and normalize

For JSON, XML and logs, parsers do the reading. The real work is normalization: flattening nested objects, handling optional fields, and producing consistent rows. This Python example (standard library only, Python 3.8+) turns the JSON invoice above into one row per line item:

import json

raw = """{
  "invoice_number": "INV-20431",
  "vendor": {"name": "Acme Supplies", "tax_id": "12-3456789"},
  "line_items": [
    {"description": "Paper, A4", "qty": 10, "amount": 45.00},
    {"description": "Toner", "qty": 2, "amount": 120.00, "discount": 10.00}
  ]
}"""

invoice = json.loads(raw)
rows = [
    {
        "invoice_number": invoice["invoice_number"],
        "vendor": invoice["vendor"]["name"],
        "description": item["description"],
        "qty": item["qty"],
        "amount": item["amount"],
        "discount": item.get("discount", 0.0),  # optional field
    }
    for item in invoice["line_items"]
]
for row in rows:
    print(row)

Output:

{'invoice_number': 'INV-20431', 'vendor': 'Acme Supplies', 'description': 'Paper, A4', 'qty': 10, 'amount': 45.0, 'discount': 0.0}
{'invoice_number': 'INV-20431', 'vendor': 'Acme Supplies', 'description': 'Toner', 'qty': 2, 'amount': 120.0, 'discount': 10.0}

From documents: OCR, models and validation

A PDF or scanned statement has no keys to parse. Getting to JSON means classifying the document, reading it with OCR, locating fields and tables with models trained on many layouts, validating the values (do the transactions reconcile to the closing balance?), and sending low-confidence fields to a person.

That's what intelligent document processing platforms do. Docsumo reports 99% field-level accuracy on 250+ document types, including invoices, bank statements and pay stubs, and returns JSON through its API and webhooks.

Challenges with semi-structured data#

• Schema drift. New or renamed keys break downstream code. Validate against a schema and alert on changes.
• Nesting. Deeply nested data is awkward in SQL and spreadsheets. Decide the grain, such as one row per line item, up front.
• Inconsistent types. The same field can arrive as a number in one record and a string in another.
• Layout variety in documents. Templates don't scale across thousands of issuers; models do.
• Quality. Catch missing or wrong values before they reach reports, with validation and review.

The bottom line#

Semi-structured data has labels but no fixed shape. For machine formats, parse it and normalize it into consistent tables. For business documents, the harder and more common case, use OCR and machine learning to find the fields, then validate them before they reach your systems.

Book a demo with a few of your own documents, or start a free trial.

Frequently asked questions#