Schema evolution

Source schemas change — a team adds a column, widens a number, retires a field. Schema evolution is how Elglide reacts to those changes on the next sync without you rebuilding the pipeline by hand. You set the policy per task under Schema → Schema Evolution.

Elglide is net-additive: your destination is never silently destroyed. Columns are added to the warehouse when the source gains them, but a column is never automatically dropped and a change that could truncate data never applies on its own — the run stops and tells you instead.

The three settings

Each task has three independent controls, one per kind of column change. All three ship with a conservative default; you only change the ones you want to behave differently.

On new source column(s)

What happens when a column appears at the source that the destination doesn't have.

Refresh destination table (default) — add the column to the destination and reload the table so its history is populated.
Ignore — leave the column out; the sync continues without it. The column is marked excluded so it isn't re-flagged every run. Switch back to Refresh and re-sync to pull it in later.
Stop and notify — pause the run so a human decides.

On dropped source column(s)

What happens when a column that used to exist disappears from the source.

Stop and notify (default) — the conservative choice: pause so someone can confirm the drop was intentional.
Mark destination column deprecated — keep syncing the remaining columns and leave the destination column in place, so its historical data is preserved. It simply stops receiving new values.

On source column type change

What happens when an existing column changes type. Because every destination is Snowflake, many source type changes don't actually change the destination type at all — see what counts as a type change below.

Widen or keep destination (never narrow) (default) — a genuine widening (for example a whole number becoming a decimal, a date becoming a timestamp, or any column becoming text) rebuilds the destination column at the wider type and reloads so the values repopulate. A change that would narrow the column (decimal back to whole number) keeps the destination at its current, wider type — the narrower values still fit, so the sync just continues. Only a truly incompatible change (for example text back to a number, with no safe common type) falls through to stop and notify.
Stop and notify — pause on any real type change.

Widening always follows the supertype rule: the destination column is only ever moved to a type that holds both the old and the new values. Elglide never narrows a column automatically, so a type change can't quietly truncate what's already landed.

Renamed columns

A rename can't be told apart from an unrelated drop-and-add with certainty, so Elglide doesn't guess — guessing wrong would corrupt data. A renamed source column is handled as a drop plus an add, governed by the two settings above: the new name flows through On new source column(s) and the old name through On dropped source column(s).

With Refresh + Mark deprecated, a rename flows through hands-off: the new column is added and backfilled while the old one is kept with its history frozen. With the defaults, it stops and notifies so you can confirm.

What counts as a “type change”

The destination is always Snowflake, and Elglide maps source types onto a small set of generous Snowflake types. So the most common source type changes land on the same Snowflake type and need no action — the sync just continues:

int → bigint — both are the same Snowflake number.
varchar(50) → varchar(100) — both are the same Snowflake text.
Even a source narrowing like bigint → int — the destination column already holds it.

The On source column type change setting only governs the rarer cases where the destination type genuinely has to move — a whole number becoming a decimal, a date becoming a timestamp, and so on.

Picking a policy

Hands-off, additive pipeline — Refresh + Mark deprecated + Widen: new and widened columns flow through automatically; nothing is ever destroyed.
Change-controlled pipeline — keep the Stop-and-notify defaults so every real schema change pauses for review.
Fixed column set — Ignore new columns so only the columns you mapped ever reach the warehouse.

See it in the app

Open any task, go to Schema → Schema Evolution. Set the policy per column-change type; it takes effect on the next sync. Open your tasks (sign-in required).