[linux] Insert multiple lines into a file after specified pattern using shell script

I want to insert multiple lines into a file using shell script. Let us consider my input file contents are: input.txt:

abcd
accd
cdef
line
web

Now I have to insert four lines after the line 'cdef' in the input.txt file. After inserting my file should change like this:

abcd
accd
cdef
line1
line2
line3
line4
line
web

The above insertion I should do using the shell script. Can any one help me?

This question is related to linux bash shell sed awk

The answer is


sed '/^cdef$/r'<(
    echo "line1"
    echo "line2"
    echo "line3"
    echo "line4"
) -i -- input.txt

This answer is easy to understand

  • Copy before the pattern
  • Add your lines
  • Copy after the pattern
  • Replace original file

    FILENAME='app/Providers/AuthServiceProvider.php'

STEP 1 copy until the pattern

sed '/THEPATTERNYOUARELOOKINGFOR/Q' $FILENAME >>${FILENAME}_temp

STEP 2 add your lines

cat << 'EOL' >> ${FILENAME}_temp

HERE YOU COPY AND
PASTE MULTIPLE
LINES, ALSO YOU CAN
//WRITE COMMENTS

AND NEW LINES
AND SPECIAL CHARS LIKE $THISONE

EOL

STEP 3 add the rest of the file

grep -A 9999 'THEPATTERNYOUARELOOKINGFOR' $FILENAME >>${FILENAME}_temp

REPLACE original file

mv ${FILENAME}_temp $FILENAME

if you need variables, in step 2 replace 'EOL' with EOL

cat << EOL >> ${FILENAME}_temp

this variable will expand: $variable1

EOL

Here is a more generic solution based on @rindeal solution which does not work on MacOS/BSD (/r expects a file):

cat << DOC > input.txt
abc
cdef
line
DOC
$ cat << EOF | sed '/^cdef$/ r /dev/stdin' input.txt
line 1
line 2
EOF

# outputs:
abc
cdef
line 1
line 2
line

This can be used to pipe anything into the file at the given position:

$ date | sed '/^cdef$/ r /dev/stdin' input.txt

# outputs
abc
cdef
Tue Mar 17 10:50:15 CET 2020
line

Also, you could add multiple commands which allows deleting the marker line cdef:

$ date | sed '/^cdef$/ {
  r /dev/stdin
  d
}' input.txt

# outputs
abc
Tue Mar 17 10:53:53 CET 2020
line

I needed to template a few files with minimal tooling and for me the issue with above sed -e '/../r file.txt is that it only appends the file after it prints out the rest of the match, it doesn't replace it.

This doesn't do it (all matches are replaced and pattern matching continues from same point)

#!/bin/bash

TEMPDIR=$(mktemp -d "${TMPDIR:-/tmp/}$(basename $0).XXXXXXXXXXXX")
# remove on exit
trap "rm -rf $TEMPDIR" EXIT

DCTEMPLATE=$TEMPDIR/dctemplate.txt
DCTEMPFILE=$TEMPDIR/dctempfile.txt

# template that will replace
printf "0replacement
1${SHELL} data
2anotherlinenoEOL" > $DCTEMPLATE

# test data
echo -e "xxy \n987 \nxx xx\n yz yxxyy" > $DCTEMPFILE

# print original for debug
echo "---8<--- $DCTEMPFILE"
cat $DCTEMPFILE
echo "---8<--- $DCTEMPLATE"
cat $DCTEMPLATE
echo "---8<---"

# replace 'xx' -> contents of $DCTEMPFILE
perl -e "our \$fname = '${DCTEMPLATE}';" -pe 's/xx/`cat $fname`/eg' ${DCTEMPFILE}

Using GNU sed:

sed "/cdef/aline1\nline2\nline3\nline4" input.txt

If you started with:

abcd
accd
cdef
line
web

this would produce:

abcd
accd
cdef
line1
line2
line3
line4
line
web

If you want to save the changes to the file in-place, say:

sed -i "/cdef/aline1\nline2\nline3\nline4" input.txt

if you want to do that with a bash script, that's useful.

echo $password | echo 'net.ipv4.ping_group_range=0 2147483647' |  sudo -S tee -a /etc/sysctl.conf

You can use awk for inserting output of some command in the middle of input.txt.
The lines to be inserted can be the output of a cat otherfile, ls -l or 4 lines with a number generated by printf.

awk 'NR==FNR {a[NR]=$0;next}
    {print}
    /cdef/ {for (i=1; i <= length(a); i++) { print a[i] }}'
    <(printf "%s\n" line{1..4}) input.txt

Using awk:

awk '/cdef/{print $0 RS "line1" RS "line2" RS "line3" RS "line4";next}1' input.txt 

Explanation:

  • You find the line you want to insert from using /.../
  • You print the current line using print $0
  • RS is built-in awk variable that is by default set to new-line.
  • You add new lines separated by this variable
  • 1 at the end results in printing of every other lines. Using next before it allows us to prevent the current line since you have already printed it using print $0.

$ awk '/cdef/{print $0 RS "line1" RS "line2" RS "line3" RS "line4";next}1' input.txt
abcd
accd
cdef
line1
line2
line3
line4
line
web

To make changes to the file you can do:

awk '...' input.txt > tmp && mv tmp input.txt

Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to sed

Retrieve last 100 lines logs How to replace multiple patterns at once with sed? Insert multiple lines into a file after specified pattern using shell script Linux bash script to extract IP address Ansible playbook shell output remove white space from the end of line in linux bash, extract string before a colon invalid command code ., despite escaping periods, using sed RE error: illegal byte sequence on Mac OS X How to use variables in a command in sed?

Examples related to awk

What are NR and FNR and what does "NR==FNR" imply? awk - concatenate two string variable and assign to a third Printing column separated by comma using Awk command line Insert multiple lines into a file after specified pattern using shell script cut or awk command to print first field of first row How to run an awk commands in Windows? Linux bash script to extract IP address Print line numbers starting at zero using awk Trim leading and trailing spaces from a string in awk Use awk to find average of a column