Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 83 additions & 0 deletions .changeset/20281-connector-sync-moved-to-mapping.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
'@objectstack/spec': minor
---

feat(spec)!: connector-attached sync leaves the connector — `syncConfig` and `fieldMappings` are retired, and a `mapping` gains the `connectorSource` pull binding (#20281)

**BREAKING** — `connector.syncConfig` (the `DataSyncConfig` block: `strategy`,
`direction`, `realtimeSync`, `timestampField`, `conflictResolution`, `batchSize`,
`deleteMode`, `filters`) and `connector.fieldMappings` (the `ConnectorFieldMapping`
list: `source`, `target`, `defaultValue`, `dataType`, `required`, `syncMode`) are
removed from `ConnectorSchema` and `DeclarativeConnectorEntrySchema` — so from
`defineConnector`, `stack.connectors[]`, the `PUT /api/v1/meta/connector/:name` door
and `AutomationEngine.registerConnector`. The `DataSyncConfigSchema`,
`SyncStrategySchema`, `ConnectorConflictResolutionSchema` and
`ConnectorFieldMappingSchema` exports (and their `DataSyncConfig`, `SyncStrategy`,
`ConnectorConflictResolution`, `ConnectorFieldMapping` types and `…Parsed` aliases)
leave `@objectstack/spec/integration` with them. ADR-0049, ruled ENFORCE on the
maintainer's criterion for a declared-but-unenforced family, with the definition
MOVED: every mainstream platform binds a sync to its TARGET, not to the connection.

Measured before removal: no engine ever ran a connector-attached sync or moved a
value through a connector field mapping — outside the spec package `syncConfig`
appeared only in two comments and `fieldMappings` nowhere, the automation service's
declared-connector item carried neither key, and the def a provider registers is its
own. The `latest_wins` and `soft_delete` defaults read as configured policy and did
nothing; the platform has no soft delete.

**Added (declared, not yet executed):** `mapping.connectorSource` — the pull
binding on the target side, beside the mapping's existing `targetObject`,
`fieldMapping`, `mode` and `upsertKey`: `connector` (a `rest` or `openapi`
connector instance), `action` (the action that reads the records), optional
`input`, optional `recordsPath`, and an optional `watermark` (`field` on the
record, `param` on the request) for a timestamp-incremental pull. Version 1 is a
one-way pull. It carries no cadence (a `job` sets that), no credential (the
connector instance holds it) and no delete or conflict policy. Nothing executes it
in this release, and `os validate` / `os build` warn when it is authored.

### FROM → TO

| removed | what to write instead |
| --- | --- |
| `connector.syncConfig` | delete the key. A sync you still want is a `mapping` on its target object, with `connectorSource` naming the connector and its read action, `watermark` for an incremental pull, and a `job` for the cadence. `direction`, `conflictResolution` and `deleteMode` have no counterpart: the pull is one-way and writes through `mode` / `upsertKey`. |
| `connector.fieldMappings` | delete the key, and carry its `source` → `target` pairs into that mapping's `fieldMapping` (a `defaultValue` becomes a `constant` transform). |
| `DataSyncConfigSchema`, `SyncStrategySchema`, `ConnectorConflictResolutionSchema`, `ConnectorFieldMappingSchema` and their types | no replacement — nothing parsed a sync or a connector field mapping into anything that ran. |

**The one-line fix: delete `syncConfig:` and `fieldMappings:` from every connector.**
`os migrate meta --from 17` lists the mechanical edits for existing sources.

⚠️ Runtime behaviour is deliberately **unchanged**: no connector sync ever ran.
What changes is the answer an author gets — both keys are refused at parse with a
prescription naming the target-side binding, and in `tsc` (their input type is
`never`), instead of being saved with no effect.

### The retirement kit

- **Tombstones.** `syncConfig` and `fieldMappings` are `retiredKey()` tombstones on
the private `ConnectorBaseSchema` both published carriers wrap (the schema is not
`.strict()`, so a bare deletion would be a silent strip, ADR-0104).
`RETIRED_KEYS_BY_MAJOR[18]`: `integration/Connector:syncConfig`,
`integration/Connector:fieldMappings`,
`integration/DeclarativeConnectorEntry:syncConfig` and
`integration/DeclarativeConnectorEntry:fieldMappings`. Neither key had a default,
so no retired-default residue is owed.
- **Four defs leave whole** (`RETIRED_DEFS_BY_MAJOR[18]`): `integration/DataSyncConfig`,
`integration/SyncStrategy`, `integration/ConnectorConflictResolution`,
`integration/ConnectorFieldMapping`. The two `RENAMED_DEFS` entries that targeted
them left the rename table.
- **D2 conversion `connector-sync-keys-removed`** (step 18, retired from the load
path): strips both keys from `connectors[]` and from stored `sys_metadata`
connector rows (the rehydration seam replays it), one notice per key, as a
lossless delete. It never writes a `mapping`: a pulled mapping would start writes
that never happened.
- **D3 entry `connector-sync-keys-retired`** carries the family's judgement: which
syncs should now exist as target-side mappings, and what an author who relied on
`export`, `bidirectional`, `soft_delete` or a conflict policy does without them.
- **No deprecation window**, per the project's startup-stage posture.

⚠️ **The out-of-repo consumer population is NOT MEASURED.** `@objectstack/spec` is
published, so this is breaking for consumers no telemetry was consulted for.

Clause-②: yes (narrowing)

<!-- adr-0087: registered connector-sync-keys-removed, connector-sync-keys-retired -->
11 changes: 11 additions & 0 deletions content/docs/references/data/mapping.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ const result = ImportFieldMappingSchema.parse(data);
| **fieldMapping** | `{ source: string \| string[]; target: string \| string[]; transform: Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>; params?: object }[]` | ✅ | |
| **mode** | `Enum<'insert' \| 'update' \| 'upsert'>` | optional (default: `"insert"`) | |
| **upsertKey** | `string[]` | optional | Fields to match for upsert (e.g. email) |
| **connectorSource** | `{ connector: string; action: string; input?: Record<string, any>; recordsPath?: string; … }` | optional | Pull binding: the rest/openapi connector this mapping pulls rows from (one-way, full or timestamp-incremental; a `job` sets the cadence). Declared; the pull is not executed yet |
| **_lock** | `Enum<'none' \| 'no-overlay' \| 'no-delete' \| 'full'>` | optional | Item-level lock — controls overlay & delete (ADR-0010). |
| **_lockReason** | `string` | optional | Human-readable reason shown when a write is refused by _lock. |
| **_lockSource** | `Enum<'artifact' \| 'package' \| 'env-forced'>` | optional | Layer that set _lock (artifact \| package \| env-forced). |
Expand All @@ -66,6 +67,16 @@ const result = ImportFieldMappingSchema.parse(data);
| **transform** | `Enum<'none' \| 'constant' \| 'lookup' \| 'split' \| 'join' \| 'javascript' \| 'map'>` | optional (default: `"none"`) | |
| **params** | `{ value?: any; valueMap?: Record<string, any>; separator?: string }` | optional | |

### Nested Shape: `Mapping.connectorSource`

| Property | Type | Required | Description |
| :--- | :--- | :--- | :--- |
| **connector** | `string` | ✅ | Name of the connector instance the rows are pulled from — a `connectors[]` entry whose `provider` is `rest` or `openapi` (version 1) |
| **action** | `string` | ✅ | Key of the connector action that reads the records: `request` on a `rest` connector, an operation key on an `openapi` connector |
| **input** | `Record<string, any>` | optional | Fixed input passed to the action on every pull (e.g. `{ method: 'GET', path: '/contacts' }` for `rest`) |
| **recordsPath** | `string` | optional | Dot path, within the action's result, to the array of records (e.g. `body.results`); omitted ⇒ the result's `body` is the array |
| **watermark** | `{ field: string; param: string }` | optional | Timestamp-incremental pull; omitted ⇒ every pull reads the full set |


---

Expand Down
10 changes: 5 additions & 5 deletions content/docs/references/index.mdx
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Protocol reference — every schema by module
navTitle: Protocol Reference
description: Every schema published by @objectstack/spec — 1525 schemas across 14 protocol modules
description: Every schema published by @objectstack/spec — 1521 schemas across 14 protocol modules
---

{/* ⚠️ AUTO-GENERATED — DO NOT EDIT. Run build-docs.ts to regenerate. Hand-written docs live in the module folders under content/docs/. */}
Expand All @@ -25,7 +25,7 @@ counts are sums of the rows they head. Regenerate with
| [Automation Protocol](/docs/references/automation) | 14 | 75 | Flows and their nodes, approvals, ETL pipelines, webhooks, state machines, execution records. |
| [Data Protocol](/docs/references/data) | 30 | 178 | Objects, fields, queries, filters, datasources and drivers — the ObjectQL layer. |
| [Identity Protocol](/docs/references/identity) | 5 | 27 | Users and accounts, organizations, positions, SCIM provisioning. |
| [Integration Protocol](/docs/references/integration) | 1 | 16 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. |
| [Integration Protocol](/docs/references/integration) | 1 | 12 | The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances. |
| [Kernel Protocol](/docs/references/kernel) | 30 | 157 | Plugin lifecycle and manifests, capabilities and security, metadata loading, service registry. |
| [Marketplace Protocol](/docs/references/marketplace) | 4 | 30 | The package & marketplace format — package identity and versions, listing, publish, review, search, install, template manifests. |
| [QA Protocol](/docs/references/qa) | 1 | 8 | Declarative test suites — scenarios, steps, actions and assertions. |
Expand All @@ -34,7 +34,7 @@ counts are sums of the rows they head. Regenerate with
| [Studio Protocol](/docs/references/studio) | 3 | 35 | Studio designer metadata — the authoring surfaces for the protocols above. |
| [System Protocol](/docs/references/system) | 34 | 275 | The runtime environment — logging, jobs, cache, metrics, notifications, i18n and compliance. |
| [UI Protocol](/docs/references/ui) | 16 | 166 | Apps, pages, views, dashboards, reports, actions and themes — the ObjectUI layer. |
| **Total** | **197** | **1525** | 14 protocol modules |
| **Total** | **197** | **1521** | 14 protocol modules |

---

Expand Down Expand Up @@ -188,13 +188,13 @@ Users and accounts, organizations, positions, SCIM provisioning.

## Integration Protocol

**Source:** `packages/spec/src/integration/` · **Import:** `@objectstack/spec/integration` · **1 page, 16 schemas**
**Source:** `packages/spec/src/integration/` · **Import:** `@objectstack/spec/integration` · **1 page, 12 schemas**

The single connector protocol (ADR-0097) — catalog descriptors and provider-bound instances.

| File | Schemas |
| :--- | :--- |
| [`connector.zod.ts`](/docs/references/integration/connector) | `Connector`, `ConnectorAction`, `ConnectorActionEffect`, `ConnectorConflictResolution`, `ConnectorFieldMapping`, `ConnectorInstanceAPIKeyAuth`, `ConnectorInstanceAuth`, `ConnectorInstanceBasicAuth`, `ConnectorInstanceBearerAuth`, `ConnectorInstanceNoAuth`, `ConnectorRetryStrategy`, `ConnectorType`, `DataSyncConfig`, `DeclarativeConnectorEntry`, `RetryConfig`, `SyncStrategy` |
| [`connector.zod.ts`](/docs/references/integration/connector) | `Connector`, `ConnectorAction`, `ConnectorActionEffect`, `ConnectorInstanceAPIKeyAuth`, `ConnectorInstanceAuth`, `ConnectorInstanceBasicAuth`, `ConnectorInstanceBearerAuth`, `ConnectorInstanceNoAuth`, `ConnectorRetryStrategy`, `ConnectorType`, `DeclarativeConnectorEntry`, `RetryConfig` |

---

Expand Down
Loading
Loading