Styjun

I am very new to Hadoop and some how we managed to install it with apache distribution and Derby database.

My requirement is having multiple users to access hive at a single time. But right now we are only able to allow a single user at a time.

I searched some of the blogs but haven't found the solution.

Could some one help me with solution?

Derby only allows single connection (process) to access the database at a give time, hence only one user can access the Hive.

Upgrade your hive metastore to either MySQL, PostgreSQL to support multiple concurrent connections to Hive.

For upgrading your metastore from Derby to MySQL/PostgreSQL there are lot resources online here's some of them:

• From Cloudera
  


• From Apache Hive Wiki
  

There are many different ways to access metastore by multiple users concurrently.

1. Embedded metastore.(default metastore:derby)


2. Local metastore.


3. Remote metastore.

Let's see the usage of above mentioned metastore.

Embedded metastore :

This metastore is only using for Unit test. And it's limitation that, it allows only a user to access Hive at same (Multiple sessions are not allowed and it throws error).

Local metastore:(By using MySql or Oracle DB)

To overcome the default metastore limitation the Local metastore is used, this can allow multiple user in same JVM (It allows multiple session on same machine). To setup this mode see below of this answer.

Remote Metastore(This metastore is using in production)

In a same project multiple hive users need to worked on it, and they can use hive concurrently on different machine but the metadata should be stored on centralized by using MySql or Oracle, ect,. Here, hive are running on each users JVM, If users are are processing, then they want to communicate with metastore which is centralized, for communicating we are going with Thrift Network APIs. To setup this mode see below of this answer.

METASTORE SETUP FOR MULTIPLE USER:

Step 1 : Download and install mysql server

sudo apt-get install mysql-server

Step 2 : Download and install JDBC driver.

sudo apt-get install libmysql-java

Step 3 : We need to copy the downloaded JDBC driver to hive/lib/ or link the JDBC location to hive/lib.

-Goto to the $HIVE_HOME/lib folder and create a link to the MySQL JDBC library.

ln -s /usr/share/java/mysql-connector-java.jar

Step 4 : Create users on metastore to access remotly and locally.

mysql -u root -p <Give password while installing DB>
mysql> CREATE USER 'user1'@'%' IDENTIFIED BY 'user1pass';
mysql> GRANT ALL PRIVILEGES ON *.* TO 'hduserdb'@'%' WITH GRANT OPTION;
mysql> flush privileges;

• IF you want multiple user to access do repeat the step 4 by giving user name, password.

Step 5 :: Goto hive/conf/hive-site.xml (If it's not available create it.)

<configuration>

<property> 
 <name>javax.jdo.option.ConnectionURL</name>
 <value>jdbc:mysql://localhost:3306/metastore?createDatabaseIfNotExist=true</value>
 <description>replace -master- with your database hostname</description>
</property>

<property>
 <name>javax.jdo.option.ConnectionDriverName</name>
 <value>com.mysql.jdbc.Driver</value>
 <description>MySQL JDBC driver class</description>
</property>

<property>
 <name>javax.jdo.option.ConnectionUserName</name>
 <value>user1</value>
 <description>user name for connecting to mysql server</description>
</property>

<property>
 <name>javax.jdo.option.ConnectionPassword</name>
 <value>user1pass</value>
 <description>password for connecting to mysql server</description>
</property>

<property>
 <name>hive.metastore.uris</name>
 <value>thrift://slave2:9083</value>
 <description>Here use your metasore host name to access from different machine</description>
</property>

</configuration>

• Do repeat only Step 5 on all users machine and change user name and password according.

Step 6 : From hive-2.. onwards we must give this comment.

slave@ubuntu~$: schematool -initSchema -dbType mysql 

Step 7 : To start hive metastore server

~$: hive --service metastore &

Now, check hive with different user concurrently from different machine.