Doris ETL Source

In Gathr, Doris can be added as a channel. It is an open-source real-time data warehouse for high-query speed.

Schema Type #

See the topic Provide Schema for ETL Source → to know how schema details can be provided for data sources.

After providing schema type details, the next step is to configure the data source.

Data Source Configuration #

Configure the data source parameters as explained below.

Connection Name #

Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details for Dynamics 365 earlier. Or create one as explained in the topic - Dynamics 365 Connection →

Use the Test Connection option to ensure that the connection with the Dynamics 365 channel is established successfully.

A success message states that the connection is available. In case of any error in test connection, edit the connection to resolve the issue before proceeding further.

Database Name #

Select Database from the available list of the selected connection.

Table #

Select Table from the available list of the selected Database.

Limit #

Limit of records to be fetched.

Run Time Filter Query #

Query filter to be applied on table during the pipeline execution/run time. Example: _C0 > 20.

Design Time Filter Query #

Query filter to be applied on table during the pipeline design/inspection. Example: _C0 > 20.

Batch Size #

The batch size determines the number of rows to be fetched per round trip. The default value is 1000.

Add Configuration: Additional properties can be added using this option as key-value pairs.