Skip to content

LLM-ready export (format=llm)

Choose format=llm when you need normalized, deduplicated, versioned JSON for an LLM, RAG pipeline, or downstream system. It provides a consistent contract instead of raw parser output.

Request

GET /v1/key/jobs/{jobId}/export?format=llm
Authorization: Bearer <project-api-key>

The job must be complete, and the API key must belong to the same project as the job.

Response

  • Content-Type: application/json
  • Attachment: job-{jobId}-results.llm.json
curl -sS "$API_BASE/v1/key/jobs/JOB_ID/export?format=llm" \
  -H "Authorization: Bearer $API_KEY" \
  -o job-results.llm.json

Envelope shape

Field Meaning
schemaVersion Contract version (currently 1.0.0). Check the major version when integrating.
kind "search" or "details"
generatedAt Job completion time (ISO 8601) or null
marketplace Marketplace code (e.g. US) or null
language Crawl language or null
productCount Number of products after dedupe
products Array of normalized products

Every product includes the same keys (asin, title, brand, url, imageUrl, price, currency, rating, reviewCount, features, categories, keyword). A field that does not apply to the job type is set to null or an empty array; fields are never omitted.

Why use llm vs json?

format=json format=llm
Shape Raw stored crawl items Normalized profile
Dedupe No Yes (order-preserving)
Version field No schemaVersion
Key stability Depends on the parser Fixed keys and order

Schema & versioning

Machine-readable JSON Schema (for validation in your pipeline):

packages/crawler-core/schemas/llm-ready-output.schema.json

Breaking changes bump the major schemaVersion. Maintainer reference: repo docs/llm-ready-export.md.

See also