Custom ETL Target
Create a custom business logic for targets. Implement the com.Gathr.framework.api.processor.JSONProcessor interface and provide custom business logic in implemented methods.
Prerequisites
To use a custom target, you need to create a jar file containing your custom code. Then upload the jar file as a registered component.
To write a custom logic for your custom Data Target, download a sample component from Register Entities > Components page.
Import the downloaded sample component as a maven project in Eclipse. Ensure that Apache Maven is installed on your machine and that the PATH for the same is set.
Implement your custom code. Build the project to create a jar file containing your code, using the following command:
mvn clean install –DskipTests
If the maven build is successful, you need to upload the jar on the Register Entities > Components page.
Custom Target for Spark
Custom target for Spark supports writing code in Java programming language.
While using a Custom processor in the pipeline, you need to create a class that implements com.Gathr.framework.api.spark.processor.CustomProcessor interface. Add unimplemented methods in this class to write your business logic in it.
Shown below is a sample class structure:
There are three methods to implement.
Init: Enables entering any initialization calls.
Process: Contains actual business logic. This method is called for each tuple.
Cleanup: All resource cleanup occurs in this method.
Custom Target Configuration
Configuring Custom Data Target.
Connection Type
Type of connection to be selected for custom component.
Connection Name
Connections are the service identifiers. A connection name can be selected from the list if you have created and saved connection details earlier, or create a new connection for the type of connection selected above.
Emitter Plugin
Fully qualified name of a custom code class.
Upload the custom code jar using the Upload button on the Register Entities > Components page.
You can use this Custom Data Target in any application.
Priority
Priority defines the execution order for the emitters.
Click on the NEXT button.
Post Action
To understand how to provide SQL queries or Stored Procedures that will be executed during pipeline run, see Post-Actions →
Notes
Optionally, enter notes in the Notes → tab and save the configuration.
Click DONE for saving the configuration details.
If you have any feedback on Gathr documentation, please email us!