> ## Documentation Index
> Fetch the complete documentation index at: https://docs.hookstream.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Instant Database

Collections, upsert engine, schema inference, and data export.

Instant Database turns a webhook stream into a queryable table. Point a collection at a source, pick a primary key, and hookstream upserts every matching event into a record — ready to query, export, and analyze.

## What is Instant Database?

Instant Database lets you sync webhook events to queryable collections. Each collection is like a table: define a primary key field, and hookstream upserts events as records using a last-write-wins strategy. You get structured, queryable webhook data without running a database or writing ETL code.

## Collections

A collection is a container for records. Create one with:

- **`name`** — human-readable label
- **`source_id`** — which source to sync from
- **`primary_key_path`** — dot-notation path to the unique identifier (e.g. `body.data.id`)

Every event from that source is automatically upserted as a record, keyed by the primary key. Use one collection per logical entity — customers, orders, subscriptions, whatever your webhooks represent.

## Upsert Engine

The upsert engine uses last-write-wins semantics based on the `source_event_at` timestamp. When an event arrives:

1. Extract the primary key from the payload using `primary_key_path`.
2. Check whether a record already exists.
3. If **new** — insert the record.
4. If **existing and newer** — update the record.
5. If **existing and older** — skip (out-of-order protection).

This handles out-of-order webhook delivery without CRDTs or vector clocks. If Stripe sends `customer.updated` then `customer.created` in the wrong order, you still end up with the latest state.

Last-write-wins means the most recent `source_event_at` wins — not the time hookstream received it. This preserves the sender's intended ordering even when the network reorders things.

## Schema Inference

hookstream infers the JSON schema of a collection by sampling up to 200 records. The inferred schema surfaces field names, types, and nesting — useful for building dashboards or writing typed clients against your webhook data.

Schema is computed **on-demand**, not stored. If your payload shape changes, the next inference call picks up the new shape automatically. No migrations, no staleness.

## Data Export

Export collection records as NDJSON or CSV:

```bash
# NDJSON — one JSON object per line, streamed
curl https://hookstream.io/v1/collections/col_abc/export?format=ndjson \
  -H "Authorization: Bearer $HOOKSTREAM_API_KEY"

# CSV — flattened with dot-notation column headers
curl https://hookstream.io/v1/collections/col_abc/export?format=csv \
  -H "Authorization: Bearer $HOOKSTREAM_API_KEY"
```

CSV export uses json-flatten to convert nested JSON into dot-notation columns (e.g. `data.customer.id`). Large exports stream so you don't have to hold the whole thing in memory.

## Backfill

Backfill re-processes historical events into a collection — useful when you create a collection after events have already been received. The backfill engine processes events in batches using `ctx.waitUntil()`, and the frontend re-triggers batches until the job completes. Progress is visible in the dashboard.

## Next Steps

**[Collections API](/api/collections)**

Create collections, query records, and run exports.
  **[Filters & Transforms](/concepts/filters-transforms)**

Shape payloads before they hit your collections.
  **[Event Replay](/guides/event-replay)**

Replay historical events to backfill a collection.
  **[Events](/concepts/events)**

How events flow into the upsert engine.