File Connector

Sync CSV, JSON, NDJSON, XML and Parquet files from a folder on your own network into Snowflake. The files are read in place by an agent inside your network — nothing is copied, moved, renamed or deleted, and nothing reaches Elglide except the columns your Schema Map selects.

How it fits together

• Connection — points at one agent group. The folder itself is set on the agent host, not in the portal.
• File set — one set produces one target table. A connection can have many. This is where the subfolder, patterns, format and limits live.
• Pipeline — picks a file set the way it would pick a table.

1. Point the agent at your folder

On the agent host, set FILE_ROOT in the bundle's .env and restart the bundle:

FILE_ROOT=/srv/exports

It can be a local path or a mounted network share — mount it on the host first, and the agent reads it like any other folder. The connector opens it read-only.

2. Create the connection

Connections → New → Files (Mounted Folder). Give it a name, pick the agent group, then use Test Connection: a successful test lists the folders the agent can see beneath the root and how many files are in each. That is the fastest way to confirm the mount is right before configuring anything else.

3. Add a file set

Edit connection → Manage file sets → New. One file set feeds one target table.

Field	What it does
Subfolder	Where to look, relative to the root. Blank means the root itself.
Include patterns	One per line. Any match selects a file, so several filename shapes can feed one table.
Exclude patterns	Applied after includes. Any match rejects the file — useful for *.tmp partial writes.
Modified after	Ignore anything older, even if never loaded. The "start from here" filter for a folder with years of history.
Size range	Optional. A minimum of 1 byte skips empty placeholder files.
File format	A file whose extension disagrees with this fails the run rather than being parsed on a guess.

Patterns are case-sensitive and use / separators on every host, including Windows. A glob is translated to a regular expression when you save and shown back to you, so what actually runs is never hidden.

Wildcard	Matches
*	Any run of characters within one folder — it stops at a /.
**/	Any number of folders, including none. **/*.csv finds sales.csv and 2026/q1/sales.csv alike.
?	Exactly one character, not a /.

Preview Matches runs the real selection code against the real folder, through your agent, before anything is saved — including a warning when a pattern matches nothing.

4. Detect the schema

On the pipeline's Schema tab, Detect Schema samples the newest files and proposes a column type for each field. A proposal is a suggestion in a dropdown — it never silently rewrites a mapping you already saved.

What counts as drift depends on the format. CSV is positional, so a missing column is fatal — every later value would shift one place left. JSON, NDJSON, XML and Parquet are matched by name, so a record that omits a field simply gets NULL for it; only a field that is not in the map at all fails the run, because it has nowhere to go.

Every row carries its origin

Four columns are added to every file-sourced table and cannot be removed:

Column	Contains
_ELGLIDE_FILE	Path of the source file, relative to the subfolder
_ELGLIDE_LINE	Record number within that file
_ELGLIDE_FILE_MODIFIED	The file's last-modified time
_ELGLIDE_EXTRACTED_AT	When the sync ran

They let you trace any row back to the file it came from, and tell two versions of the same filename apart.

Each file loads once

Every file that loads successfully is recorded in a ledger, keyed by path, size and modified time. The next run skips it without opening it. Replace a file with a newer version and it is treated as new work.

Those three facts come from the directory listing, so a run never has to open a file just to decide to skip it. The trade-off is worth knowing: a file edited in place that ends up the same size with its timestamp preserved looks unchanged and will not be re-read. Tools that restore timestamps when copying can produce this; if yours does, write a new filename instead.

Nothing enters the ledger until the load has committed, so a run that fails at any point leaves it untouched and the same files are simply picked up next time. You can review what has been loaded — and reset it to force a reload — on the pipeline's Ledger tab.

Format options

Optional JSON on each file set. Leave it blank for the defaults below.

Format	Option	Default
CSV / TSV / TXT	Delimiter	,
	Quote	"
	Escape	"
	NullSequence	none
	HasHeader	true
	SkipLeadingLines	0
	SkipTrailingLines	0
JSON / NDJSON	RecordPath	root array
XML	RecordPath	repeating element
Parquet	No options — the file describes itself

Run limits

Each file set can cap how much one sync does. Leave them blank to use the agent's own defaults, which is right for almost everyone.

Limit	Effect
Files per run	Files beyond the cap are not skipped — they are left for the next run, which picks up where this one stopped.
Bytes per run	The same, measured in bytes read.
Rows per file written	Rows per file uploaded to your warehouse. Lower uses less scratch space on the agent and uploads sooner.

These are per file set, so a large nightly export and a small hourly drop on the same agent can be tuned independently.

When a run fails

• A column moved or disappeared. The run fails naming the file and the column. Re-run Detect Schema and save to accept the new shape, or exclude that file.
• The files disagree with each other. If the files in one set do not all share a column layout, no single Schema Map fits them. Narrow the set — tighten the pattern, set Modified after, or split it into two file sets.
• A value will not load. The run fails and nothing is committed, so you never get a partial table. Fix the file and re-run; the ledger was not written, so it is picked up again.
• The agent cannot see the folder. Test Connection reports it. Check FILE_ROOT, that the mount survived a reboot, and that the agent's user can read it.