[hive] How to set variables in HIVE scripts

Most of the answers here have suggested to either use hiveconf or hivevar namespace to store the variable. And all those answers are right. However, there is one more namespace.

There are total three namespaces available for holding variables.

  1. hiveconf - hive started with this, all the hive configuration is stored as part of this conf. Initially, variable substitution was not part of hive and when it got introduced, all the user-defined variables were stored as part of this as well. Which is definitely not a good idea. So two more namespaces were created.
  2. hivevar: To store user variables
  3. system: To store system variables.

And so if you are storing a variable as part of a query (i.e. date or product_number) you should use hivevar namespace and not hiveconf namespace.

And this is how it works.

hiveconf is still the default namespace, so if you don't provide any namespace it will store your variable in hiveconf namespace.

However, when it comes to referring a variable, it's not true. By default it refers to hivevar namespace. Confusing, right? It can become clearer with the following example.

If you do not provide namespace as mentioned below, variable var will be stored in hiveconf namespace.

set var="default_namespace";

So, to access this you need to specify hiveconf namespace

select ${hiveconf:var};

And if you do not provide namespace it will give you an error as mentioned below, reason being that by default if you try to access a variable it checks in hivevar namespace only. And in hivevar there is no variable named var

select ${var}; 

We have explicitly provided hivevar namespace

set hivevar:var="hivevar_namespace";

as we are providing the namespace this will work.

select ${hivevar:var}; 

And as default, workspace used during referring a variable is hivevar, the following will work too.

select ${var};