Hadoop Tutorial

Merge option in Hive with Example

Wow! Merge in Hive ? Yes , after the successful release of hive 2.2.X merge is also possible in hive now. Today I will walk you through one simple example that will clear merge concept in hive. What is Merge option in hive:- With Merge option we can perform record level insert,update and delete in h..

Parse XML File using HIVE

In this post , we will learn how to parse XML file using hive. I am using below xml file for this example. jmdbks@hadoop:~$ cat test.xml <test><name>Sumit Kumar</name><properties><age>29</age><sex>male</sex></properties></test> <test>..

Important built-in function in Hive

(I)explode() and posexplode():- explode() takes in an array (or a map) as an input and outputs the elements of the array (map) as separate rows. Below example will help to understand explode() better. 1)Create example data set that having only one column  as Array<int>. beauty2955@hadoop:~$ ca..

Date function in hive

  1)from_unixtime: This function converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a STRING that represents the TIMESTAMP of that moment in the current system time zone in the format of “1970-01-01 00:00:00”. The following example returns the current date including the time..

Hive Installation

Hive installation: 1.) search for apache hive-2.2.0 bin in google and download zar file (latest bin.tar.gz file) http://www-eu.apache.org/dist/hive/hive-2.2.0/ e.g. :- apache-hive-2.2.0-bin.tar.gz or download hive from linux command as below:-- wget http://www-eu.apache.org/dist/hive/hive-2.2.0/apac..

Big Data: Frequently used HDFS commands in Real Time

HDFS commands 1)mkdir (Create a directory) hadoop fs –mkdir  /data 2)copyFromLocal(Copy a file or directory from Local to HDFS) If we want to copy file1 from local to HDFS inside directory /data then we have to use below command hadoop fs –copyFromLocal file1  /data/ Note: Can be used for copying mu..

Planet Polaris

Website Support
website support at Planet Polaris