Topics

See more

Athena Connector

Introduction:

The document will provide you the step wise guide in order to use python script AWS Athena connector.

Supported Version:

AWS Cloud.

Pre-requisites:

  1. Klera Script Integration Service v2.3.4 or above.
  2. AWS User with Client ID, Client Secret Key and the AWS Region of Athena (for ex. us-west-1).
  3. Specified S3 bucket path for output of Athena queries.
  4. Full Access permissions for AWS user for Athena and S3.
  5. The AWS region of Athena should be accessible from the Klera server (the server where Klera is deployed).
  6. Boto3 library package (Internet connectivity should be thorough to download it using below steps).The below URL should be accessible from the Klera Server: 
    https://pypi.org/
  7. Python3 should be present.

IMPORTANT: Please make sure your Athena Connector instance must be accessible from the Klera Machine.

How to use:

In order to use the AWS Athena connector the user is required to follow the below steps:

Register Script on Klera:

  1. Execute any operation and bring any container on your exploration.
  2. Right click on the header of an existing container on the Klera and go to the “Scripts -> Register” and run.

    mceclip0.png
  3. It will first ask you to reduce data using prefilter but just select apply and proceed further.

    Enter the Operation name, Short Description for your script, hierarchy(where you want the operation to be shown later) and browse the Athena_ConnectorV1.0.0.py file to submit.

    The below prompt will open up, please fill the details as shown:

    mceclip1.png
  4. Click on the PACKAGES tab besides the SCRIPT tab in the current form and write the value for the Package name as ‘boto3’.

    mceclip2.png
  5. Click on the Submit to submit the current form.

  6. The below prompt will open up. Click on submit again and proceed further.

    mceclip3.png

  7. Finally validate the script by providing the Client ID, Client Secret, Hosted Athena AWS region, a simple SQL Query used in the Athena for any table, Output Location – the S3 bucket path where you want your output to be stored and the WorkGroup for the Athena and Submit the form.

    mceclip4.png

  8. Finally, you will see the two datasets in the View Panel. One is the actual results dataset for the query you submitted in the previous form and the other one containing the details for your script registration.

Use Operation on Klera:

  1. Right click on the floor and go to Script->Operations->{The hierarchy mentioned while registering the script}.

  2. Once you see the operation, please click on it and execute.

  3. It will open up the form where you need to provide the same details mentioned in the step #f of the Register Script on Klera and submit.

  4. You will see the required dataset in the View Panel of the Klera.

Important Points:

  1. There is always a cost associated with Athena with respect to the size of the data scanned. For more information:

    https://aws.amazon.com/athena/pricing/

  2. User always needs to provide a S3 bucket path for script execution on Klera. Query results are stored in an S3 bucket of your choice and are also billed at standard Amazon S3 rates. For more information:

    https://aws.amazon.com/s3/pricing/

 

↑