n8n for Data Pipelines: Database Sync & ETL Automation

n8n replaces lightweight ETL scripts with visual workflows that extract data from databases and APIs, transform it through Set and Code nodes, and load it into data warehouses, Google Sheets, or Airtable. Scheduled triggers or webhooks drive the pipeline, and built‑in nodes for PostgreSQL, MySQL, and over 400 connectors make database sync achievable without writing SQL orchestration code. [1]

How do you extract data from PostgreSQL and transform it with n8n?

Connect the Postgres node with SSL credentials and write a SELECT query in the “Execute SQL” operation. The output JSON then passes through a Set node that renames or reshapes fields—for example converting timestamps to ISO strings or masking PII columns. Finally, a second Postgres node inserts the cleaned data into another table. [1]

For large datasets, use the “Execute in Batches” option on the source node to fetch records in chunks, avoiding memory issues. This pattern is also detailed in the SplitInBatches loop guide for processing thousands of rows.

How do you sync data between two MySQL databases using n8n?

Set up two MySQL nodes—one for source and one for target. The source runs a SELECT with a WHERE clause filtered by the last processed timestamp; the target uses an INSERT … ON DUPLICATE KEY UPDATE (upsert) for each row. A Set node maps column names between schemas before the insert. [2]

Schedule this workflow with a cron trigger for nightly sync. To guarantee idempotency, always store the last synced ID or timestamp in a static data node or external key‑value store. For more on reliable scheduling, see our n8n trigger types guide.

How do you use Google Sheets as an automated reporting database with n8n?

Use the Google Sheets node with a service account. The “Append” operation adds new rows to a sheet, while “Update” overwrites specific cells. A workflow can pull data from an API, transform it with a Set node, and append a summary row to a tracking sheet every hour. [3]

For dashboards, combine multiple data sources into one sheet by running parallel branches that each write to a separate tab. This approach replaces expensive BI connectors for small‑team reporting, as demonstrated in our marketing and data use cases.

How do you automatically sync records between a database and Airtable?

Create a workflow with a database trigger (e.g., new row via polling) that feeds into the Airtable node. Configure the Airtable node with a Personal Access Token and select the base/table. Use the “Create” operation for new records and “Update” for existing ones identified by a lookup on the Airtable record ID. [4]

For bidirectional sync between a database and Airtable, pattern the workflow as described in the two‑way CRM sync guide —listening on both ends and using upsert logic to avoid duplicate rows.

How do you map and reshape fields with the Set node in an ETL pipeline?

The Set node maps, renames, and deletes columns using a visual field editor. Enable “Keep Only Set” to drop all unlisted fields, or toggle individual fields to add defaults, convert types, or compute expressions like {{ $json.orderTotal * 1.2 }}. This turns raw API responses into clean, load‑ready records. [5]

For advanced reshaping, chain a Code node after the Set node to run JavaScript transformations on nested structures. The combination covers both simple and complex ETL mappings. Learn more about expressions in the node expressions & techniques guide.

How do you perform an upsert (update or insert) in n8n for databases?

Use the “Upsert” operation available on the Postgres or MySQL node. Set the “Column to Match On” to a unique key like email or id. n8n generates a MERGE/ON CONFLICT statement that inserts new rows or updates matched rows with the specified column values, all within a single database call. [1]

This pattern avoids the need for separate SELECT‑then‑INSERT logic. When the datasource is an API, first transform the payload with a Set node to match the database column names, then feed it directly into the upsert node. For error handling during bulk upserts, wrap the execution in an error workflow with retry logic.