Azure SQL ETL Target

Azure SQL target loads transformed data into Azure SQL and SQL Server databases.

Target Configuration

Configure the target parameters that are explained below.

Azure SQL target loads transformed data into Azure SQL and SQL Server databases.

Target Configuration

Configure the target parameters that are explained below.

Connection Name

Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Azure SQL earlier. Or create one as explained in the topic - Azure SQL Connection →

Use the Test Connection option to ensure that the connection with the Azure SQL database is established successfully.

A success message states that the connection is available. In case of any error in test connection, edit the connection to resolve the issue before proceeding further.

Schema Name

Select the schema name where the target table resides. The schema metadata is retrieved from the target system.

Table Name

Select the table name to load data at runtime. The table metadata is retrieved from the target system.

Enable Batch

Enable this option to process and send records in batches.

Batch Size

Specify the number of records sent to the database in a single batch.

Increasing batch size boosts performance by reducing database round trips, yet large batches may consume excessive memory.

Output Mode

Specifies how the data changes in a streaming source are written to the sink. Options include:

  • Append: Only new rows of data are written to the sink.

  • Complete: The entire dataset is written to the sink with each update.

  • Upsert: New rows are added, and existing rows are updated based on key matches.

  • Update: Only the rows with changes are written to the sink.

Save Mode

Save mode specifies how to handle any existing data in the target. The options are:

  • Append: Appends data in the configured target.

  • Overwrite: Overwrites existing data if any, in the target.

  • Upsert: The existing data in the designated target table will be updated, and any new data will be inserted.

  • Update: Update the data rows in the target.

Ignore Error

Selecting this option ingestion errors will be ignored e.g. duplicate key constraint.

Enable Trigger

Trigger defines how frequently a streaming query should be executed.

Trigger Type

Supported Trigger Types are:

  • One-Time Micro-Batch: A trigger that process only one batch of data in a streaming query then terminates the query.

  • Fixed Interval Micro-Batches: A trigger policy that runs a query periodically based on an interval in processing time.

Processing Time

Specifies the time interval that governs the trigger policy.

Ignore Values

Enable this option to exclude specified values from the data. This option is applicable for streaming only.

Values to Ignore

Enter comma-separated values to exclude from the data. If a record contains any of these values in columns, even partially, then matching column values will be ignored and set as null.

Priority

Priority defines the execution order for the emitters

ADD CONFIGURATION: Enables to configure additional custom properties.

Post Action

To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Post-Actions →

Notes

Optionally, enter notes in the Notes → tab and save the configuration.

If you have any feedback on Gathr documentation, please email us!

← Azure SQL Incremental Configuration
Azure Synapse Connection →