Create and Manage Templates
Before creating a template, make sure you have:
A working Gathr ETL Application that you want to convert into a template.
Appropriate permissions to create and manage templates.
A clear understanding of which parameters should be configurable.
Accessing Templates
Navigate to Applications > Advanced ETL section.
Click on the Templates tab.
You’ll see the template listing page with these options:
+ (Create Template)
Refresh (to update the template list)
Save User Preferences (to save your current view settings)
Creating a New Template
Method 1: From Template Listing Page
Click the + button on the template listing page.
A new template canvas will open where you can build your ETL Application.
Method 2: From Existing ETL Application
Locate your ETL Application in the ETL Application listing.
Click the ellipsis (…) on the ETL Application tile.
Select Create Template.
The ETL Application will open in template mode.
Building Your Template
Add Components:
Use the Components tab to add sources, transformations, and targets.
Configure each component as needed.
Parameterize Your Template:
Identify values that should be configurable (e.g., connection strings, table names, file paths).
Replace these values with parameters.
Add Parameters:
Click the Parameters option.
Click + Add Parameter.
Fill in the parameter details (see Parameter Configuration below).
Parameter Configuration
When adding a parameter, you’ll need to provide:
Field | Description | Example |
---|---|---|
Name | Enter a unique name for the parameter. This name will be used to reference the parameter within the ETL application components. | source_database |
Value | Enter the actual value for the parameter. This value must match the configuration in the existing ETL application to ensure correct functionality when creating the template. | production_db |
Default Value | Enter a default value for the parameter. This value will be used if no other value is specified during the template creation or execution. It’s optional but provides a fallback if the parameter is not assigned a value. | dev_database |
Applying Parameters to Components
Right-click on a component and select Parameterize.
In the parameterization table:
Identify the field you want to parameterize from the list.
Click Validate to validate that the values match.
Click Apply to save your changes.
Externalize a Template Component’s Configurations
A template’s component configurations can be externally provided in the same manner as done for ETL applications.
For more details, see the topic Externalize ETL Component Configurations.
Saving and Managing Templates
Saving Your Template
Click Update (save icon) in the top.
Provide a name, update configurations, and add optional description.
Click Save & Exit to confirm.
Templates Listing Page
The template listing page provides several views and actions:
Search: Find templates by Application Name.
Filter: Sort by Application Name, Source, Target,Processor (Transformations), or Analytics components.
Pagination: Navigate through multiple pages of templates.
Template Actions
Each template tile includes an actions menu (…) with these options:
Action | Description |
---|---|
View | View the template configurations. |
Edit | Modify the template configurations. |
Start All Instances | Start all instances of the template. |
Stop All Instances | Stop all instances of the template. |
Schedule All Instances | Schedule all instances of the template. |
Create Instances | Generate new instances of this template. |
Update Instances Parameters | Update the parameters of the existing instances. |
Summary | View the summary of the template. |
Delete | Remove the template (requires confirmation). |
Delete All Instances | Remove all instances of the template (requires confirmation). |
Next Steps
- Learn how to use templates as components in other ETL applications.
Create Instance
Click the template actions menu (…) to select the Create Instances option. Provide the below details for instance creation:
Field | Description |
---|---|
Instance Type | Option to select between creating single or multiple instances. |
Name | Provide a Unique name of the instance to be created. The template name will be prefixed before the instance name. |
Instance Count | Enter the number of instances to create when choosing the multiple instance option. |
Parameter Details | Provide value(s) against the parameter(s) Key for which the instance has to be created. |
Note: When creating multiple instances, the parameter details will remain the same for all the newly created instances.
Under the Component Used column, the variables used in the listed components are mentioned.
Click Create to create the instance(s).
Schedule All Instances
Field | Description |
---|---|
Frequency | Option to periodically schedule the Template Instance(s) run at fixed intervals of minutes, hour(s), day(s), week(s), month(s), or year(s). |
Every | Option to specify the interval at which the Template Instance(s) should run. The interval should be specified in minutes, hour(s), day(s), week(s), month(s), or year(s). |
Start Time | Option to specify the start time for the Template Instance(s). |
End Date & Time | Option to specify the end date and time for the Template Instance(s). |
Time Zone | Option to specify the time zone for the Template Instance(s). |
Cron Expression | A non-editable field that displays the cron expression for the scheduled Template Instance(s). |
Update Instances Parameters
Option to update all the instances of template.
On the Update Instance Parameter window you can select the instances and Download JSON for Selected Instance’s Parameters and Upload modified JSON.
View Templates
From the ellipsis of the template tile, you can select View to view its details and the existing instances of a particular template.
Template Details Tab
Details | Information |
---|---|
Current Job Progress | You can track an ongoing job’s progress shortly after the application becomes Active. |
Schedule Frequency | The frequency of the template’s execution schedule. |
Next Schedule | The next scheduled execution time. |
End Schedule | The end schedule time. |
Description | The description of the template. |
Instances Tab
Once the instances are created, you can view them under Instances tab.
Perform various actions by clicking the ellipses of the instance tile as explained below:
Action | Description |
---|---|
Start | Start the instance. |
View | View the details of the instance. The details shown for an instance are similar to ETL Application Details. |
Schedule | Set up automated execution schedules for the instance. Refer to Schedule All Instances for more details. |
Edit | Edit the instance. |
Inspect | Inspect the instance. The data source(s) with configuration details will be listed which can be updated as per requirement. |
Change Cluster | Change the cluster where your Template Instance should be deployed. |
Delete | Permanently remove the instance (requires confirmation) |
If you have any feedback on Gathr documentation, please email us!