IHadoopStatement

IHadoopStatement is an interface implemented by the connector's statement class. It provides access to methods that allow for asynchronous execution of queries and the retrieval of the Yarn ATS GUID associated with the execution.

The IHadoopStatement interface is defined by the IHadoopStatement.java file. This file should look like the following example:

// =================================================================================================

/// @file IHadoopStatement.java /// /// Exposed interface for asynchronous query execution. /// /// Copyright (C) 2017 Simba Technologies Incorporated.

// =================================================================================================

package com.simba.hiveserver2.hivecommon.core;

import java.sql.ResultSet; import java.sql.SQLException;

import java.sql.Statement;

/**

* An interface that extends the standard SQL Statement Interface but allows for asynchronous

* query execution.

* The polling for query execution will occur when {@link ResultSet#next()} or

* {@link ResultSet#getMetaData()} is called.

*/ public interface IHadoopStatement extends Statement

{

/**

* Executes the given SQL statement asynchronously.

* <p> * Sends the query off to the server but does not wait for query execution to complete.

* A ResultSet with empty columns is returned.

* The polling for completion of query execution is done when {@link ResultSet#next()} or

* {@link ResultSet#getMetaData()}is called.

* </p>

*

* @param sql

An SQL statement to be sent to the database, typically a

* static SQL SELECT statement.

*

* @return A ResultSet object that DOES NOT contain the data produced by the given query; never null.

*

* @throws SQLException If a database access error occurs, or the given SQL

* statement produces anything other than a single

* <code>ResultSet</code> object.

*/

public ResultSet executeAsync(String sql) throws SQLException;

/**

* Returns the Yarn ATS guid.

*

* @return String The yarn ATS guid from the operation if execution has started,

* else null.

*/

public String getYarnATSGuid(); }

The following methods are available for use in IHadoopStatement:

  • executeAsync(String sql)

    The connector sends a request to the server for statement execution and returns immediately after receiving a response from the server for the execute request without waiting for the server to complete the execution.

    The connector does not wait for the server to complete query execution unless getMetaData() or next() APIs are called.

    Note that this feature does not work with prepared statements.

For example:

import com.simba.hiveserver2.hivecommon.core.IHadoopStatement;

 

public class TestExecuteAsyncClass

{

public static void main(String[] args) throws SQLException

{

// Create the connection object.

Connection connection = DriverManager.getConnection("jdbc:hive2://localhost:10000");

 

// Create the statement object.

Statement statement = connection.createStatement();

 

// Unwrap the java.sql.Statement object to an implementation of IHadoopStatement so the

// execution can be done asynchronously.

//

// The connector will return from this call as soon as it gets a response from the

// server for the execute request without waiting for server to complete query execution.

ResultSet resultSet =

statement.unwrap(

IHadoopStatement.class).executeAsync(

"select * from example_table");

 

// Calling getMetaData() on the ResultSet here will cause the connector to wait for the server

// to complete query execution before proceeding with the rest of the operation.

ResultSetMetaData rsMetadata = resultSet.getMetaData();

 

// Excluding code for work on the result set metadata...

 

// Calling getMetaData() on the ResultSet here, and if getMetaData() was not call prior to

// this, will cause the connector to wait for the server to complete query execution before

// proceeding with the rest of the operation.

resultSet.next();

 

// Excluding code for work on the result set ...

}

}

  • getYarnATSGuid()

    Returns the Yarn ATS GUID associated with the current execution. Returns null if the Yarn ATS GUID is not available.

For example:

public class TestYarnGUIDClass

{

public static void main(String[] args) throws SQLException

{

// Create the connection object.

Connection connection = DriverManager.getConnection("jdbc:hive2://localhost:10000");

 

// Create the statement object.

Statement statement = connection.createStatement();

 

// Execute a query.

ResultSet resultSet = statement.executeQuery("select * from example_table");

 

// Unwrap the java.sql.Statement object to an implementation of IHadoopStatement to access the

// getYarnATSGuid() API call.

String guid = statement.unwrap(

IHadoopStatement.class).getYarnATSGuid();

}

}

The connector can retrieve the query logs from the server when executing a SQL statement using the following functions:

 

getQueryLog(boolean incremental,int fetchSize)

The connector gets the execution logs of the given SQL statement.

 

hasMoreLogs() ;

The connector checks whether the query execution produces more logs to be fetched.

 

/**

* Get the execution logs of the given SQL statement.

*

* @param incremental      True to indicate to get logs incrementally,

*                         otherwise false to indicate to get logs from the beginning,

* @param fetchSize        The number of lines to fetch

*

* @return A list of logs. It can be empty if there are no new logs to be retrieved at that time.

* @throws ErrorException   if an error occurs fetching logs from the server

*/

public List<String> getQueryLog(boolean incremental, int fetchSize) throws ErrorException;

 

/**

* Check whether query execution produces more logs to be fetched.

*

* @return true if the query execution produces more logs, false otherwise

*/

public boolean hasMoreLogs();

 

For example:

 

public class TestGetQueryLog

{

public static void main(String[] args) throws SQLException

{

// Create the connection object.

Connection connection = DriverManager.getConnection("jdbc:hive2://localhost:10000");

 

// Create statement object.

Statement statement = connection.createStatement();

 

try

{

// Create and start the class that gets the query logs in the background

// while the query is running.

HiveLogRunnable runnable = new HiveLogRunnable(statement);

Thread t1 = new Thread(runnable);

t1.setDaemon(true);

t1.start();

 

// Execute the query.

ResultSet resultSet = statement.executeQuery("select * from example_table");

while (resultSet.next()) {}

}

   finally

  {

// Close the statement.

if (statement != null)

  {

  statement.close();

    }

  }

 }

}

 

class HiveLogRunnable implements Runnable {

private Statement stmt = null;

public HiveLogRunnable(Statement stmt) {

this.stmt = stmt;

}

@Override

public void run() {

    try {

      // Print the query logs to the console while the query has logs.

      while (stmt.unwrap(IHadoopStatement.class).hasMoreLogs()) {

          List<String> logLists = stmt.unwrap(IHadoopStatement.class).getQueryLog(true, 0);

          for (String string : logLists) {

              System.out.println(string);

         }

         Thread.sleep(0);

       }

    } catch (Exception e) {

        e.printStackTrace();

    }

  }

}

Related topics