Tags

,

Hi all

HDFS commands examples

View the usage of hadoop fs

# hadoop fs

Enter the following -ls command to view the contents of the user’s root directory in HDFS, which is /user/root:

# hadoop fs -ls

View the contents of the /user directory in HDFS:

# hadoop fs -ls /user

-ls command again, but this time specify the root HDFS folder:

# hadoop fs -ls / 

The output should look like:

Found 6 items

drwxrwxrwt – yarn hdfs 0 2013-08-20 13:59 /app-logs

drwxr-xr-x – hdfs hdfs 0 2013-08-20 13:53 /apps

Switch to  user:

# su – hdfs

Make a new directory in HDFS named /user/root:

$ hadoop fs -mkdir /user/root

Change the permissions to make root the owner of the directory:

$ hadoop fs -chown root /user/root

Switch back to the root user:

$ exit

Create a directory named sample in HDFS:

hadoop fs -mkdir sample

Verify the folder was created successfully:

# hadoop fs -ls Found 1 items

Create a couple of subdirectories of sample:

# hadoop fs -mkdir sample/sample1

Delete the sample2 folder (and recursively its subcontents)

# hadoop fs -rm -R sample/sample2

run the ls -R command:

# hadoop fs -ls -R 

The directory structure of the output

.Trash

.Trash/Current

.Trash/Current/user

.Trash/Current/user/root

Put a file into the sample folder. Change directories to

/var/log/hadoop/hdfs:

# cd /var/log/hadoop/hdfs 

Notice this folder contains a file named hdfs-audit.log:

# tail hdfs-audit.log 

Run the following -put command to copy hdfs-audit.log into the sample folder in HDFS:

# hadoop fs -put hdfs-audit.log sample/

Verify the file is in HDFS by listing the contents of sample:

# hadoop fs -ls sample

Copy the hdfs-audit.log file in sample to another folder in HDFS:

# hadoop fs -cp sample/hdfs-audit.log sample/sample1/copy.log

# hadoop fs -getmerge sample /tmp/merged.txt