Cluster Configuration for CDH/CDP Using Cloudera Manager

💡 The component versions that are mentioned in this sub-topic are for representational purpose only. For the Gathr supported component version details, see Component Versions Supported →

Login to Gathr using Superuser credentials and go to Setup from the main menu.

1. Select Cluster Configuration tile and configure the properties as given below:

   

   Provide values for the following properties:

   Field	Description
   Cluster Manager	Select the cluster manager as Cloudera Manager.
   URL	Provide the Cloudera Manager URL as below
   User Name	Username for Cloudera Manager
   Password	Password for Cloudera Manager
   Cluster Name	Provide the Cloudera Manager cluster name
   Enable Kerberos	Enable Kerberos allows Kerberos configuration from cluster manager be configured in Gathr.

   

   Click Save and configure all managed services that are supported in Gathr with its progress.

   💡

   Livy configuration is not supported through setup simplification process; please make sure that Gathr is pointing to the correct Livy URL that looks as follows:

   

   Livy URL for CDH/CDP/Apache: http://localhost:8998

To install Livy on CDH/CDP environments and configure it in Gathr, do as follows:

1. Download the apache binary. Click here to download.

2. Extract the zip file using the below command:

   Cmd: unzip apache-livy-0.6.0-incubating-bin.zip
   

3. Update livy config file with below parameters.

   # What host address to start the server on. By default, Livy will bind to all network interfaces.
   livy.server.host = <hostname>
   
   # What port to start the server on.
   livy.server.port = 8998
   
   # What spark master Livy sessions should use.
   livy.spark.master = yarn
   
   # What spark deploy mode Livy sessions should use.
   livy.spark.deploy-mode = cluster
   
   # How long a finished session state should be kept in LivyServer for query.
   livy.server.session.state-retain.sec = 30s
   
   # If livy should impersonate the requesting users when creating a new session.
   livy.impersonation.enabled = false
   
   # on user request and then livy server classpath automatically.
   livy.repl.enable-hive-context =true
   

4. To enable Kerberos authentication, update the below configuration.

   💡

   Generate keytabs before updating the configuration.

   # Authentication support for Livy server
   # Livy has a built-in SPnego authentication support for HTTP requests with below configurations.
   livy.server.auth.type = kerberos
   livy.server.auth.kerberos.principal = HTTP/impetus-i0141.impetus.co.in@IMPETUS
   livy.server.auth.kerberos.keytab = /etc/security/keytabs/HTTP-impetus-i0141.keytab
   livy.server.launch.kerberos.principal = livy@IMPETUS
   livy.server.launch.kerberos.keytab = /etc/security/keytabs/livy.service.keytab
   

5. Start livy using below command

   Cmd : <installation\_path>/bin/ livy-server start
   

6. Livy server URL is: http://:8998

   💡

   This URL needs to be configured in the Gathr application under the Processing Engine configuration.