LatticeOne
Back to Normalize

Normalize: setup, API and data handling

Start with the setup guide below, or jump to API authentication and limits. Review source data and exceptions before using any clean export. A successful reconciliation accounts for rows; it does not certify their business meaning.

Review and export

The single-file complete audit bundle is JSON containing all mapped, rejected and quarantined records, their raw values and unmapped fields, the mapping and schema, reconciliation report, clean CSV, source tables and original files encoded as base64. Keep the original input and this bundle together. The on-page table previews the first 50 rows of each bucket; downloads contain the full run.

Correction happens in your spreadsheet editor: inspect the source row and its reason, correct the original value or record why it must remain unchanged, and keep a dated change log with the reviewer’s name, row identifier, original value, revised value and decision. Upload the revised original file and rerun validation. An invalid value remains quarantined; approving a column mapping does not override validation. Retaining a value means retaining its exception, not certifying it as clean.

Whole-drop browser tables are capped samples; its audit, mappings and learned registry downloads do not contain all data rows. The offline edition writes full mapped.csv, report.json, rejected.json and quarantine.json per family under Out, plus mapping and layer artifacts. Source files move to Done and are not deleted.

Data handling

Public browser trials send files to Lattice’s server for processing in request memory. These routes do not save file bytes or rows and do not invoke AI. Responses are marked no-store. The browser retains the current run until navigation or reload; downloaded bundles remain wherever you save them. Do not upload sensitive documents unless this handling meets your requirements.

Usage logs contain endpoint, license identifier (or trial), row count, success and time. Website analytics may record request metadata and a conversion with row count. Hosting and network services also process request metadata; this page does not promise a fixed infrastructure-log deletion period or zero retention by every provider. See the privacy policy.

Saving a whole-drop project stores mappings, file fingerprints, names and run summaries, never data rows. Projects expire 180 days after their last save. You provide source files again on each visit.

Licensed API AI suggestions and document extraction use Anthropic and may transmit column samples or document text. They require the applicable AI entitlement; public trials and the offline edition use no AI. Server OCR temporarily writes PDF/images and removes its temporary directory after success or failure. Failed parsing returns an error or quarantine reason; source values are never silently replaced.

Hashes and versions

Source hashes use SHA-256 of file bytes; source IDs contain the first 16 hexadecimal characters, sheet name and source row. The mapping hash is the first 16 SHA-256 characters of recursively key-sorted JSON of the mapping excluding informational provenance. The output hash uses the same canonical JSON process over mapped records, including raw text, transformation rules and extras; rejected and quarantined rows are outside that hash. Array and file order matter.

The bundle records the engine version, complete target schema and its full SHA-256 hash. Transform rules are named per field and versioned with the engine. Replay guarantees apply to identical bytes, mapping, input order and engine version; a different year’s file may produce different output. Review hashes and reconciliation after an engine update. Model suggestions and extraction are advisory and are not a deterministic replay contract.

Offline setup and licensing

Use the Mac or Windows setup steps on the offline edition page. Read the installer and verify SHA-256 checksums before running. These are command-line installers; macOS signing is ad-hoc and is not Developer ID signing or notarization. Direct binaries: Mac Apple Silicon, Mac Intel, Windows x64. Verify filenames against checksums.

lattice-normalize setup
lattice-normalize activate lic_…
lattice-normalize status
lattice-normalize watch

Activation sends the license key and receives a signed token saved on your computer. Renewal is attempted when less than a day remains on that token; failed network refreshes use the saved token and its grace window. Defaults are 7 days for subscriptions and 14 days for managed licenses, with license-specific overrides. PURCHASE licenses do not expire offline. The status command shows your actual offline deadline and row limit. Once grace lapses, new runs stop until a successful refresh. Your data files stay local. PDFs and emails require API extraction and are set aside locally.

API quick start

Base path: /api/v1/normalize. Authenticate using Authorization: Bearer lic_… or x-api-key. The key must have Normalize entitlement. Public trials permit 2 MB and 2,000 rows; licensed requests permit 25 MB and default to 100,000 rows, subject to license limits. Keyed routes allow 60 requests per minute per endpoint/key; trials allow 20. Retry 429 responses after Retry-After.

  • GET /schemas: public built-in contact, customer, job and invoice_line schemas.
  • POST /profile: file types, fill rates, samples and header drift.
  • POST /propose: files and target; returns a proposed mapping for review. AI is opt-in.
  • POST /run: files, reviewed mapping and format=json|csv|ndjson. JSON includes all buckets and report; CSV/NDJSON contain mapped rows only.
  • POST /pipeline: whole drop, optional reviewed mappings, registry, target or partial-layer controls. Returns ranked families, capped samples, gates and audit.
  • POST /extract: up to 25 PDF, EML or TXT documents and target. Values without source-supported quotes are quarantined.
curl https://teamlattice.ai/api/v1/normalize/propose   -H 'Authorization: Bearer lic_…' -F file=@customers.csv -F target=customer
# Save and review the returned mapping as mapping.json, then:
curl https://teamlattice.ai/api/v1/normalize/run   -H 'Authorization: Bearer lic_…' -F file=@customers.csv   -F 'mapping=<mapping.json' -F format=json

Multipart uploads use repeated file parts. JSON accepts files as objects with name and text or base64. A mapping contains engine, target, fields (target field to source-header alternatives), dateOrder, ignore and optional decimalMark, synonyms and nullTokens. GET /schemas returns field definitions and required fields.

{
  "ok": true,
  "mapped": [
    {
      "sourceId": "hash:Sheet1:2",
      "data": {
        "customer_id": "C-001"
      },
      "raw": {
        "customer_id": "C-001"
      },
      "rules": {
        "customer_id": "string:trim"
      },
      "extra": {}
    }
  ],
  "rejected": [],
  "quarantine": [],
  "report": {
    "ok": true,
    "engineVersion": "lattice-normalize/1",
    "counts": {
      "input": 1,
      "mapped": 1,
      "rejected": 0,
      "quarantine": 0
    }
  }
}

The response above is an abbreviated shape example, not a complete report. Full JSON reports include checks, input/mapping/output hashes, rule and issue counts, table provenance and totals. A run with report.ok=false is not complete; inspect every check and all exceptions.

Errors return JSON with an error string: 400 invalid request or mapping; 401 missing/invalid key; 402 inactive license; 403 missing entitlement or AI disabled; 413 size/row/document limit; 422 unreadable or empty input; 429 rate limited. Schema validation errors include the invalid mapping path.

Versioning: URLs use v1 and mappings declare an engine version. Preserve your reviewed mapping and record the returned engine version. Run/profile/propose calls create no saved records and accept no Idempotency-Key; retrying a run recomputes it and logs another usage event. Identical inputs and engine reproduce deterministic run output. AI proposals/extraction can vary. Saved-project writes are separate and use revision-based compare-and-set; stale writes return 409.