Template as Component in an ETL Application
For users requiring consistent and reusable data processing configurations, Gathr provides the Template as Component feature.
This capability enables seamless modification of various data processing elements, including data sources, processing steps, and output methods.
In this setup, a Template can be used in an ETL Application, and it can have multiple steps. You can choose specific parts of the Template to adjust or change within your data processing setup.
Key Functionality
Utilize a Template as channel, transformation, and target.
Choose the part of a template that is to be used in ETL Application.
If a template is updated, the effective changes will reflect once the ETL Application is started.
Steps to Use Template as Component
On the ETL Applications canvas, click Add Operator and select the Templates tab to use the listed templates.
Click on any data source or use a template as a data source, transformation or target.
For an instance, once any component is connected to the Template, then the window opens displaying the template view.
Upon clicking the TEM icon, the template flow will appear.
Select a template component and the flow next to it will adjoin the ETL Application’s component.
Customize Template
Click on the template to customize it:
| Field | Description |
|---|---|
| Key | Name of the parameter. |
| Value | Value associated with the parameter is reflected. The value can be edited by hovering over the value and clicking Edit. An editable window opens where the value can be edited and saved. |
| Default Value | The default value of the parameter is reflected. |
| Action | In case of external configuration, the key used will be reflected in red color. Download sample JSON to update the external configuration of the parameter. Please add For example, suppose an SQL processor is externalized and has query “ |
On the Decide template components for the pipeline window, right-click the component that needs to be terminated or skipped from the template flow. The options available are:
Terminate Succeeding Flow: Terminates the flow that follows the selected component.
Terminate Preceding Flow: Terminates the flow that precedes the selected component.
Skip Component: Skips the selected component from the flow.
The template components are given color coding as per their relationship with the components present in the ETL Application.
Upon terminating the succeeding or preceding flow, the terminated components will be grayed out, while the skipped component will be reflected in brown color.
Template Components Color Coding
UI Legends:
The template component(s) reflected in red are terminated from the ETL Application flow along with the chosen succeeding or preceding components in template flow.
The template component(s) reflected in brown are skipped from the ETL Application flow. However, its succeeding/preceding component(s) will be included in the flow unless you choose to terminate the succeeding/preceding component(s) or skip them from the flow.
After the template has been customized as per requirement, connect it with the ETL Application on the canvas with the exit point.
In certain scenario the option to pair two templates by connecting them in serial order is required. In that case, option to select another template and pair it with the ETL Application is available.
Upon clicking Pair, the window appears showing details of both Template 1 and Template 2. In Template 1 the exit point of the flow is selected and in Template 2 the entry point of the template flow is selected.
Entry point: Multiple inputs can be provided using Union, Join (only two) and SQL processors. Likewise, for Exit point: Multiple exit points can be provided using Add New Flow option.
Using SQL and Router processor as exit point, if user selects Add new flow option, then provide the query name.
Node Alias
In Gathr, while using template as a component within an ETL Application, you select connecting point to connect it with ETL Application’s component.
For example, in a certain scenario, you may want to update a particular template that is utilized in an ETL Application. In that particular template you may want to remove the component that is a connecting point for the ETL Application.
The ETL Application in that case will not be able to find the missing/removed component in the updated template.
Thus, with Node Alias option, you can handle the scenario of missing template component by creating a Node Alias of the ETL Application’s connecting point component.
Therefore, upon removing the component of the template which is also the connecting point of a ETL Application, the user will have to assign new component to that alias so that the ETL Application can connect to the newly defined component.
In template component, click upon the component and select Node Alias.
Provide details of Alias and click OK.
On the template canvas, click the ellipsis at the top right of the screen to see the Alias List.
The Alias name and Component name is reflected in the Alias List window.
The selected component in Alias List can be connected with the template flow that will be used as connecting point in the ETL Application.
If you have any feedback on Gathr documentation, please email us!