Core Concepts

These common terms must be understood in context of Gathr documentation. They are arranged in a sequential order based on user’s journey, as they would come across these frequently used terms while navigating further.

Account #

It refers to your Gathr account that you have created with us post sign up. You can manage it from the User Profile option that is available in the main menu.

User #

A user is the owner of an account that is registered with us, who will be utilizing all the features and services that are offered by Gathr as per the subscribed plan.

Organization Administrator #

A user who has signed up for an organization with Gathr is called as the organization administrator in documentation.

Dashboard #

The landing page that will appear for the users once they either sign up or sign in, displaying certain information and set of actions that can be performed directly from the dashboard.

For more details, see Applications →

Main Menu #

The collapsible sidebar menu which can be used to navigate to the required feature once the user has signed in. When expanded, it will also show the user’s email ID.

Engine #

Gathr introduces two engines, Gathr and Databricks, to simplify ETL, Ingestion, and CDC application development and deployment.

The Gathr engine allows users to design and deploy applications within the Gathr platform, while the Databricks engine integration enables users to utilize their existing Azure Databricks clusters.

Visible under the User Settings option in the main menu, it displays status whether the engine is connected to the server or not. Gathr engine is selected by default once a user signs in. Azure Databricks accounts can be added to Gathr platform using the Compute Setup feature for utilizing the Databricks engine.

It is essential that the an engine is connected for smoothly operating Gathr applications.

If the engine is not connected, you can reach out to Gathr support team at:
support@gathr.ai

Engine Configurations #

Engine Configurations allow you to customize how your selected engine operates within Gathr.

Interpret Delta Components

Hive components will not work with this flag.

Enable Hive Catalog

Enable this option to run Hive components.

Custom Java Home

Select this option to use the specified Java home path to create a new inspect session. This is useful when multiple Java versions are available on the system.

Java Home Path

Enter the full path to the Java installation to be used when creating a new inspect session. This setting applies only to newly created sessions.

Log Level

Select the log level for the current inspect session.

Extra Driver Java Options

Specify a string of extra JVM options to pass to the driver. For instance, GC settings or other logging. This is equivalent to passing extra options to spark submit using the spark.driver.extraJavaOptions key. Default value is (none).

Example: -Dlog4j.c

Extra Executor Java Options

Specify a string of extra JVM options to pass to executors. For instance, GC settings or other logging. This is equivalent to passing extra options to spark submit using the spark.executor.extraJavaOptions key. Default value is (none).

Example: -Dlog4j.c

Extra Spark Submit Options

Supply Spark configurations (SparkConf) to spark-submit at the runtime of Spark application. spark-submit can accept any Spark property using the --conf flag.

Example:

--conf spark.eventLog.enabled=true --conf spark.eventLog.overwrite=true

Metering #

Statistics displaying pattern of credit points utilization history against the days of each month.

Credits #

Credits are alloted based on the subscribed plan to the user accounts that can be consumed for utilizing Gathr applications.

Support #

For any assistance that may be required, the Gathr support team can be reached at: support@gathr.ai

Help #

The live chat option for users to connect with the Gathr support team.

Applications #

Ready-to-use design templates in Gathr, offering solutions for data ingestion, capturing changed data, Advanced ETL solutions, and creating reusable workflows.

Connection #

Connections are the keys that allow Gathr to communicate with various source and targets. It is required for users to configure a particular source or target connection and test if it is established successfully when they are working with applications.

Source #

A specific database, application, or file storage service supported by Gathr that can be queried to fetch data as per requirement.

Transformations #

Operations available in Gathr that can be applied to transform data from one format or structure into the desired format and structure after querying it from a source.

Functions #

Different kind of functions like Date, Lookup, String, Math, Array and Miscellaneous functions that can be used to apply computations on the collected source dataset columns to get desired result.

Target #

Target defines the destination stage of an application which could be a database, data warehouse or a data lake supported by Gathr to ultimately sink the transformed data.

Cluster #

The applications once created can be run or scheduled on a cluster. Gathr has a free tier cluster for users who are on a 14-day free trial. The users on Advanced or Business plan can utilize extra small, small, medium or large clusters as per requirement.

Also, in Business Plan, there is an option to utilize clusters that are registered with Gathr for running Data Ingestion, CDC and ETL applications. Its prerequisite is to setup compute environments in Gathr from User Settings > Compute Setup → tab.