DataFlow
PricingGet started

DataFlow Documentation

DataFlow helps you unify supplier feeds (XML, CSV, XLSX, JSON APIs) into a stable output endpoint. Use visual mapping, filters, snapshots, and scheduling so your downstream imports stop breaking.

Start here
Create your first source → feed → published endpoint.
Open quick start
Test endpoints
Pull JSON/CSV/XML with optional API key headers.
See examples
Billing & limits
Understand usage, limits, and what happens on upgrades.
Billing guide

Overview

The platform is built around the idea that messy supplier data should be handled once: connect sources, map fields into a single schema, normalize values, and publish a unified endpoint.

Unify formats
Combine XML, CSV, XLSX and JSON into one output.
Map once
Visual mapping means less custom code.
Keep it stable
Snapshots and Hybrid caching reduce breakage.
Operate confidently
Runs, diagnostics and rejects explain what changed.

Core concepts

Workspace
A workspace contains sources, feeds, mappings, runs, and members. It’s the main tenant boundary.
Sources
A source points to a supplier file or API. You can fetch via URL or upload a file. Sources are previewed and parsed.
Feeds
A feed combines one or more sources into a unified output. Feeds define the published endpoint and scheduling mode.
Mappings
Mappings transform source fields into unified target fields. You can reorder fields and save templates for reuse.
Runs & snapshots
Runs are executions of the pipeline. Each run can create snapshots (raw + unified) for auditability and troubleshooting.
Endpoints & API keys
Every feed can publish a public endpoint. Optional API keys protect access; rate limits reduce abuse.

Getting started

  1. 1) Create a workspace
    Sign up, then create a workspace for your company/project. Workspaces help isolate data and permissions.
  2. 2) Add a source
    Go to Sources → New source. Choose URL or file upload, then pick the record list in the extraction wizard (for XML/JSON).
  3. 3) Create a feed
    Go to Feeds → New feed. Select the sources that contribute to your unified output and pick a scheduling mode (Live/Hybrid/Cron).
  4. 4) Map fields
    Open Mappings. Drag fields from the source preview into target fields. Add normalizations (trim, casing, number parsing) and filters.
  5. 5) Publish & test
    Open the feed page and copy the public endpoint. Use an API key if enabled. Verify that JSON/CSV/XML output is accepted by your downstream system.
Pro tip

Start with a minimal output schema (SKU, title, price, stock). Add fields gradually after you confirm your downstream import works.

Outputs and formats

Each feed exposes a stable public endpoint. Depending on your setup, you can request JSON, CSV, or XML. If API keys are enabled, include the key in an HTTP header.

# JSON
curl -H "X-API-Key: YOUR_KEY" https://YOUR_DOMAIN/f/your-feed-slug.json

# CSV
curl https://YOUR_DOMAIN/f/your-feed-slug.csv

# XML
curl https://YOUR_DOMAIN/f/your-feed-slug.xml
CSV for marketplaces
Many marketplaces still prefer CSV. Use your downstream field order.
JSON for internal tools
JSON integrates easily with APIs and headless commerce tooling.

Scheduling modes

Live
Generate on request. Best for low traffic and highly dynamic sources.
Hybrid
Serve cached output and refresh on schedule. Great for stability.
Cron
Only refresh on a schedule. Best for large feeds and predictable imports.

Filters & normalization

Real supplier feeds contain inconsistent formats: comma decimals, mixed casing, missing brands, or category drift. Use normalization and filters to keep your output consistent.

  • • Trim whitespace, normalize casing, remove control characters
  • • Parse numbers safely (prices, weights), enforce decimal separator
  • • Set defaults (e.g., stock = 0 when missing)
  • • Filter out deleted/out-of-stock items at source or unified level

Snapshots & run history

Runs show what happened when the pipeline executed. Snapshots store inputs and outputs so you can compare changes over time.

Auditability
Explain why a product changed or disappeared.
Debugging
Inspect the raw supplier payload for a failing source.

Webhook exports

Webhook exports let you push the unified output to another system when a run finishes. Typical use cases: updating a cache, triggering downstream imports, or notifying a pipeline.

Webhook payload shape

When a feed run completes, DataFlow sends a JSON body to your configured URL (best-effort — failures do not block the run).

{
  "feed": { "id": "...", "slug": "your-feed-slug" },
  "generated_at": "2026-02-03T...Z",
  "data": [
    { "sku": "...", "name": "...", "price": 123.45 }
  ]
}

Billing, usage & troubleshooting

Billing is enforced by plan limits (feeds, sources, runs/day, features). If you upgrade and don’t see changes: use the Billing page’s Re-sync button (it fetches your Stripe subscription and updates the workspace).

Upgrade checklist
When a user pays, the workspace is updated by Stripe webhooks and return-sync.
  • • Ensure Stripe keys are set in Admin → Billing
  • • Ensure you created plan prices (monthly/yearly)
  • • If needed, click Re-sync billing on the workspace billing page
Common errors
Most issues come from missing billing details required for tax calculation.
  • • Missing customer address → open Billing Portal and add address
  • • VAT ID collection requires name updates → handled automatically in Checkout
  • • Key rotation between Stripe accounts → re-save prices in Admin
Create an accountGo to Billing

Want a quick overview? Visit the homepage or check the pricing page.