Sync modes and optimization

Every table syncs in one of four modes — from rebuilding the whole table each run to reading only the source's change feed. Elglide starts each table on the safe default and lets you climb the ladder to move less data as the table qualifies. You'll find these controls on each task under Settings → Sync mode and optimization.

Higher on the ladder means less data moved per run, not less accuracy. Every mode delivers the same rows to Snowflake — they differ only in how much they read from the source and write to the warehouse to get there.

The sync ladder

The four modes form a ladder. Each rung reads and writes less than the one below it, but has stricter requirements a table must meet to use it.

Full — rebuilds the whole target table every run. The safe default: it always works and needs nothing from the source, but it re-reads and re-writes the entire table each time. Best for small or keyless tables.
Incremental · RowHash — reads the whole source table, but compares a hash of each row against the last sync and writes only the rows that changed to Snowflake. Saves the warehouse-side rewrite; the source is still read in full.
Incremental · Watermark — extracts only new and changed rows using watermark columns (typically an identity key plus a last-modified date), instead of scanning the whole table. The least data read of the table-scan modes.
CT/CDC · native — reads changes straight from the source's own change feed. Inserts, updates, and deletes are captured natively with no watermark and no full-table scan. The top rung — a source-side capability Elglide detects and manages automatically. See Change Data Capture to enable it.

The row-matching key

Every incremental mode identifies a row by a key — the column (or columns) that say “this is the same row.” Elglide detects it automatically, almost always your table's primary key; for a table or view without one, a unique key. It's shown read-only in the panel (as Row-hash key or Merge key), and a composite key lists every column it spans.

The key is read-only on an active pipeline because changing how rows are matched requires a full rebuild. To change it, step the table back to Full and promote again — the key is re-derived at that point.

Promoting and demoting

You move a table up or down the ladder from the panel. Elglide re-checks eligibility server-side before applying a promotion.

Full → Incremental (RowHash) — offered once a primary or unique key is detected. The next run switches to change-only writes.
RowHash → Watermark — offered once the table has enough run history to show the move is safe: 20 completed runs spanning 30 days, with row changes actually observed and no deletes seen. Below that bar the panel shows a progress line instead — a handful of runs can't rule out a periodic delete job (a month-end purge, for example). Once the bar is met, promoting also needs usable watermark columns; very small tables aren't offered the switch even then, since there's little data to save either way.
Demote — a manual step-back button on each mode (Watermark → RowHash, RowHash → Full) if you want to move back.

Automatic safety demotes. Correctness is never gated behind a click. A per-run count audit compares source and target; if a watermark ever misses changes — or physical deletes appear on a watermark table — Elglide falls back to RowHash automatically and reports it. You only choose the cost transitions; the safety ones happen on their own.

When a table qualifies

While a table runs on RowHash, the panel shows what it has actually been doing — runs analyzed, days spanned, average rows scanned, and the insert / update / delete counts observed. Those numbers drive two decisions: whether the table has earned enough history to be offered Watermark at all, and, once offered, whether it's worth switching. A projected savings figure appears only when the table is large enough for the saving to be meaningful — on a small table, watermark extraction saves little, so no figure is shown.

Deletes and watermarks

Watermark sync follows a moving cursor (for example, a last-modified timestamp), so it can only see rows that were touched.

Soft deletes (a status flag set by an UPDATE) advance the update watermark and are captured normally.
Physical deletes (DELETE removes the row) leave no trace for a watermark to find — so tables with physical deletes don't qualify for Watermark, and the count audit demotes any table that starts physically deleting.

Need deletes captured with the least data movement? Use your database's native change data capture — it records deletes at the source and sits at the top of the ladder.

Which mode fits

Small or keyless table — Full is fine; the rebuild is cheap.
Has a key, moderate size, mixed changes — RowHash: change-only writes without any source setup.
Large, append-or-update-only, delete-free — Watermark: skip the full scan entirely.
Needs deletes captured, or you want the lowest overhead — enable native CDC.

See it in the app

Open any task, go to Settings → Sync mode and optimization. The stepper shows the current mode, the row-matching key, the observed metrics, and any promote or demote available for that table. Open your tasks (sign-in required).