Create and Manage Templates

Before creating a template, make sure you have:

  • A working Gathr ETL Application that you want to convert into a template.

  • Appropriate permissions to create and manage templates.

  • A clear understanding of which parameters should be configurable.


Accessing Templates

  1. Navigate to Applications > Advanced ETL section.

  2. Click on the Templates tab.

  3. You’ll see the template listing page with these options:

    • + (Create Template)

    • Refresh (to update the template list)

    • Save User Preferences (to save your current view settings)


Creating a New Template

Method 1: From Template Listing Page

  1. Click the + button on the template listing page.

  2. A new template canvas will open where you can build your ETL Application.


Method 2: From Existing ETL Application

  1. Locate your ETL Application in the ETL Application listing.

  2. Click the ellipsis (…) on the ETL Application tile.

  3. Select Create Template.

  4. The ETL Application will open in template mode.


Building Your Template

  1. Add Components:

    • Use the Components tab to add sources, transformations, and targets.

    • Configure each component as needed.

  2. Parameterize Your Template:

    • Identify values that should be configurable (e.g., connection strings, table names, file paths).

    • Replace these values with parameters.

  3. Add Parameters:

    • Click the Parameters option.

    • Click + Add Parameter.

    • Fill in the parameter details (see Parameter Configuration below).


Parameter Configuration

When adding a parameter, you’ll need to provide:

FieldDescriptionExample
NameEnter a unique name for the parameter. This name will be used to reference the parameter within the ETL application components.source_database
ValueEnter the actual value for the parameter. This value must match the configuration in the existing ETL application to ensure correct functionality when creating the template.production_db
Default ValueEnter a default value for the parameter. This value will be used if no other value is specified during the template creation or execution. It’s optional but provides a fallback if the parameter is not assigned a value.dev_database

Applying Parameters to Components

  1. Right-click on a component and select Parameterize.

  2. In the parameterization table:

    • Identify the field you want to parameterize from the list.

    • Click Validate to validate that the values match.

  3. Click Apply to save your changes.


Externalize a Template Component’s Configurations

A template’s component configurations can be externally provided in the same manner as done for ETL applications.

For more details, see the topic Externalize ETL Component Configurations.


Saving and Managing Templates

Saving Your Template

  1. Click Update (save icon) in the top.

  2. Provide a name, update configurations, and add optional description.

  3. Click Save & Exit to confirm.


Templates Listing Page

The template listing page provides several views and actions:

  • Search: Find templates by Application Name.

  • Filter: Sort by Application Name, Source, Target,Processor (Transformations), or Analytics components.

  • Pagination: Navigate through multiple pages of templates.


Template Actions

Each template tile includes an actions menu (…) with these options:

ActionDescription
ViewView the template configurations.
EditModify the template configurations.
Start All InstancesStart all instances of the template.
Stop All InstancesStop all instances of the template.
Schedule All InstancesSchedule all instances of the template.
Create InstancesGenerate new instances of this template.
Update Instances ParametersUpdate the parameters of the existing instances.
SummaryView the summary of the template.
DeleteRemove the template (requires confirmation).
Delete All InstancesRemove all instances of the template (requires confirmation).

Next Steps


Create Instance

Click the template actions menu (…) to select the Create Instances option. Provide the below details for instance creation:

FieldDescription
Instance TypeOption to select between creating single or multiple instances.
Name

Provide a Unique name of the instance to be created.

Instance CountEnter the number of instances to create when choosing the multiple instance option.
Parameter DetailsProvide value(s) against the parameter(s) Key for which the instance has to be created.

Note: When creating multiple instances, the parameter details will remain the same for all the newly created instances.

Under the Component Used column, the variables used in the listed components are mentioned.

Click Create to create the instance(s).


Schedule All Instances

FieldDescription
FrequencyOption to periodically schedule the Template Instance(s) run at fixed intervals of minutes, hour(s), day(s), week(s), month(s), or year(s).
Every

Option to specify the interval at which the Template Instance(s) should run.

Start TimeOption to specify the start time for the Template Instance(s).
End Date & TimeOption to specify the end date and time for the Template Instance(s).
Time ZoneOption to specify the time zone for the Template Instance(s).
Cron ExpressionA non-editable field that displays the cron expression for the scheduled Template Instance(s).

Update Instances Parameters

Option to update all the instances of template.

On the Update Instance Parameter window you can select the instances and Download JSON for Selected Instance’s Parameters and Upload modified JSON.


View Templates

From the ellipsis of the template tile, you can select View to view its details and the existing instances of a particular template.

Template Details Tab

DetailsInformation
Current Job ProgressYou can track an ongoing job’s progress shortly after the application becomes Active.
Schedule FrequencyThe frequency of the template’s execution schedule.
Next ScheduleThe next scheduled execution time.
End ScheduleThe end schedule time.
DescriptionThe description of the template.

Instances Tab

Once the instances are created, you can view them under Instances tab.

Perform various actions by clicking the ellipses of the instance tile as explained below:

ActionDescription
StartStart the instance.
ViewView the details of the instance. The details shown for an instance are similar to ETL Application Details.
ScheduleSet up automated execution schedules for the instance. Refer to Schedule All Instances for more details.
EditEdit the instance.
InspectInspect the instance. The data source(s) with configuration details will be listed which can be updated as per requirement.
Change ClusterChange the cluster where your Template Instance should be deployed.
DeletePermanently remove the instance (requires confirmation)
Top