[linux] How to append contents of multiple files into one file

I want to copy the contents of five files to one file as is. I tried doing it using cp for each file. But that overwrites the contents copied from the previous file. I also tried

paste -d "\n" 1.txt 0.txt

and it did not work.

I want my script to add the newline at the end of each text file.

eg. Files 1.txt, 2.txt, 3.txt. Put contents of 1,2,3 in 0.txt

How do I do it ?

This question is related to linux bash unix

The answer is


If all your files are in single directory you can simply do

cat * > 0.txt

Files 1.txt,2.txt, .. will go into 0.txt


If you want to append contents of 3 files into one file, then the following command will be a good choice:

cat file1 file2 file3 | tee -a file4 > /dev/null

It will combine the contents of all files into file4, throwing console output to /dev/null.


if you have a certain output type then do something like this

cat /path/to/files/*.txt >> finalout.txt

if your files contain headers and you want remove them in the output file, you can use:

for f in `ls *.txt`; do sed '2,$!d' $f >> 0.out; done

If all your files are named similarly you could simply do:

cat *.log >> output.log

for i in {1..3}; do cat "$i.txt" >> 0.txt; done

I found this page because I needed to join 952 files together into one. I found this to work much better if you have many files. This will do a loop for however many numbers you need and cat each one using >> to append onto the end of 0.txt.

Edit:

as brought up in the comments:

cat {1..3}.txt >> 0.txt

or

cat {0..3}.txt >> all.txt

Another option, for those of you who still stumble upon this post like I did, is to use find -exec:

find . -type f -name '*.txt' -exec cat {} + >> output.file

In my case, I needed a more robust option that would look through multiple subdirectories so I chose to use find. Breaking it down:

find .

Look within the current working directory.

-type f

Only interested in files, not directories, etc.

-name '*.txt'

Whittle down the result set by name

-exec cat {} +

Execute the cat command for each result. "+" means only 1 instance of cat is spawned (thx @gniourf_gniourf)

 >> output.file

As explained in other answers, append the cat-ed contents to the end of an output file.


If the original file contains non-printable characters, they will be lost when using the cat command. Using 'cat -v', the non-printables will be converted to visible character strings, but the output file would still not contain the actual non-printables characters in the original file. With a small number of files, an alternative might be to open the first file in an editor (e.g. vim) that handles non-printing characters. Then maneuver to the bottom of the file and enter ":r second_file_name". That will pull in the second file, including non-printing characters. The same could be done for additional files. When all files have been read in, enter ":w". The end result is that the first file will now contain what it did originally, plus the content of the files that were read in.


All of the (text-) files into one

find . | xargs cat > outfile

xargs makes the output-lines of find . the arguments of cat.

find has many options, like -name '*.txt' or -type.

you should check them out if you want to use it in your pipeline


Another option is sed:

sed r 1.txt 2.txt 3.txt > merge.txt 

Or...

sed h 1.txt 2.txt 3.txt > merge.txt 

Or...

sed -n p 1.txt 2.txt 3.txt > merge.txt # -n is mandatory here

Or without redirection ...

sed wmerge.txt 1.txt 2.txt 3.txt

Note that last line write also merge.txt (not wmerge.txt!). You can use w"merge.txt" to avoid confusion with the file name, and -n for silent output.

Of course, you can also shorten the file list with wildcards. For instance, in case of numbered files as in the above examples, you can specify the range with braces in this way:

sed -n w"merge.txt" {1..3}.txt

Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to unix

Docker CE on RHEL - Requires: container-selinux >= 2.9 What does `set -x` do? How to find files modified in last x minutes (find -mmin does not work as expected) sudo: npm: command not found How to sort a file in-place How to read a .properties file which contains keys that have a period character using Shell script gpg decryption fails with no secret key error Loop through a comma-separated shell variable Best way to find os name and version in Unix/Linux platform Resource u'tokenizers/punkt/english.pickle' not found