In Hadoop 2.0,
hdfs dfs -copyToLocal <hdfs_input_file_path> <output_path>
where,
hdfs_input_file_path
maybe obtained from http://<<name_node_ip>>:50070/explorer.html
output_path
is the local path of the file, where the file is to be copied to.
you may also use get
in place of copyToLocal
.
you can accomplish in both these ways.
1.hadoop fs -get <HDFS file path> <Local system directory path>
2.hadoop fs -copyToLocal <HDFS file path> <Local system directory path>
Ex:
My files are located in /sourcedata/mydata.txt I want to copy file to Local file system in this path /user/ravi/mydata
hadoop fs -get /sourcedata/mydata.txt /user/ravi/mydata/
if you are using docker you have to do the following steps:
copy the file from hdfs to namenode (hadoop fs -get output/part-r-00000 /out_text). "/out_text" will be stored on the namenode.
copy the file from namenode to local disk by (docker cp namenode:/out_text output.txt)
output.txt will be there on your current working directory
If your source "file" is split up among multiple files (maybe as the result of map-reduce) that live in the same directory tree, you can copy that to a local file with:
hadoop fs -getmerge /hdfs/source/dir_root/ local/destination
In order to copy files from HDFS to the local file system the following command could be run:
hadoop dfs -copyToLocal <input> <output>
<input>
: the HDFS directory path (e.g /mydata) that you want to copy<output>
: the destination directory path (e.g. ~/Documents)1.- Remember the name you gave to the file and instead of using hdfs dfs -put. Use 'get' instead. See below.
$hdfs dfs -get /output-fileFolderName-In-hdfs
This worked for me on my VM instance of Ubuntu.
hdfs dfs -copyToLocal [hadoop directory] [local directory]
bin/hadoop fs -put /localfs/destination/path /hdfs/source/path
Source: Stackoverflow.com