Styjun

I have a Hadoop cluster, with one master and 3 slaves. Now, I want to add Apache Impala functionality over this cluster. I've downloaded the tarball from here. I want to build Impala, but am not sure what are the prerequisites. There are two different sources:

1. 
   This, from the Docs, which says the requirements are: MySQL (or PostgreSQL), Hive metastore, and Java dependencies (obviously).


2. 
   

   The README.md file inside the apache-impala directory created after untarring the tar ball. Quoting it:

   
   
   

   
   

   Impala can be built with pre-built components, downloaded from S3, or
   can be built with an in-place toolchain located in the thirdparty
   directory (not recommended). The components needed to build Impala are
   Apache Hadoop, Hive, HBase, and Sentry.

   
   

   
   

I am confused regarding both the sources. What should I do? A clear set of dependencies for Apache Impala would be great!

If you carefully read the Impala Requirements you will see that Hadoop support is implied while the Sentry requirement is buried in the Impala Security link near the bottom of the page.

Under the Java Dependencies section it says:

All Java dependencies are packaged in the impala-dependencies.jar file, which is located at /usr/lib/impala/lib/. These map to everything that is built under fe/target/dependency.

Looking at the corresponding pom.xml you will see all the dependencies. Grepping artifactId shows the following:

$ grep artifactId fe/pom.xml 
 <artifactId>impala-parent</artifactId>
 <artifactId>impala-frontend</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>impala-data-source-api</artifactId>
 <artifactId>hadoop-hdfs</artifactId>
 <artifactId>hadoop-common</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>hadoop-auth</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>hadoop-aws</artifactId>
 <artifactId>hadoop-azure-datalake</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>sentry-core-common</artifactId>
 <artifactId>yarn-extras</artifactId>
 <artifactId>sentry-core-model-db</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>sentry-provider-common</artifactId>
 <artifactId>sentry-provider-db</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>sentry-provider-file</artifactId>
 <artifactId>sentry-provider-cache</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>sentry-policy-common</artifactId>
 <artifactId>sentry-binding-hive</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>sentry-policy-engine</artifactId>
 <artifactId>sentry-service-api</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>parquet-hadoop-bundle</artifactId>
 <artifactId>hbase-client</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>hbase-common</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>hbase-protocol</artifactId>
 <artifactId>commons-lang</artifactId>
 <artifactId>java-cup</artifactId>
 <artifactId>libthrift</artifactId>
 <artifactId>hive-service</artifactId>
 <artifactId>hive-llap-server</artifactId>
 <artifactId>json-smart</artifactId>
 <artifactId>hive-serde</artifactId>

So the README.md is correct in stating you need Hadoop, Hive, HBase, and Sentry to build Impala.