spark_system Table

A pseudo-table called spark_system can be used to query for Spark cluster system environment information. The pseudo-table is under the pseudo-schema called spark_system. The table has two STRING type columns, envkey and envvalue. Standard SQL can be executed against the spark_system table. For example:

SELECT * FROM SPARK.spark_system.spark_system WHERE envkey LIKE '%spark%'

The above query returns all of the Spark system environment entries whose key contains the word "spark". A special query, set -v, is executed to fetch system environment information. Some versions of Spark do not support this query. For versions of Spark that do not support querying system environment information, the connector returns an empty result set.