[linux] How to print third column to last column?

I'm trying to remove the first two columns (of which I'm not interested in) from a DbgView log file. I can't seem to find an example that prints from column 3 onwards until the end of the line. Note that each line has variable number of columns.

This question is related to linux shell unix multiple-columns

The answer is


awk '{print ""}{for(i=3;i<=NF;++i)printf $i" "}'

The following awk command prints the last N fields of each line and at the end of the line prints a new line character:

awk '{for( i=6; i<=NF; i++ ){printf( "%s ", $i )}; printf( "\n"); }'

Find below an example that lists the content of the /usr/bin directory and then holds the last 3 lines and then prints the last 4 columns of each line using awk:

$ ls -ltr /usr/bin/ | tail -3
-rwxr-xr-x 1 root root       14736 Jan 14  2014 bcomps
-rwxr-xr-x 1 root root       10480 Jan 14  2014 acyclic
-rwxr-xr-x 1 root root    35868448 May 22  2014 skype

$ ls -ltr /usr/bin/ | tail -3 | awk '{for( i=6; i<=NF; i++ ){printf( "%s ", $i )}; printf( "\n"); }'
Jan 14 2014 bcomps 
Jan 14 2014 acyclic 
May 22 2014 skype

awk '{for(i=3;i<=NF;++i)print $i}' 

Jonathan Feinberg's answer prints each field on a separate line. You could use printf to rebuild the record for output on the same line, but you can also just move the fields a jump to the left.

awk '{for (i=1; i<=NF-2; i++) $i = $(i+2); NF-=2; print}' logfile

awk '{$1=$2=$3=""}1' file

NB: this method will leave "blanks" in 1,2,3 fields but not a problem if you just want to look at output.



If you want to print the columns after the 3rd for example in the same line, you can use:

awk '{for(i=3; i<=NF; ++i) printf "%s ", $i; print ""}'

For example:

Mar 09:39 20180301_123131.jpg
Mar 13:28 20180301_124304.jpg
Mar 13:35 20180301_124358.jpg
Feb 09:45 Cisco_WebEx_Add-On.dmg
Feb 12:49 Docker.dmg
Feb 09:04 Grammarly.dmg
Feb 09:20 Payslip 10459 %2828-02-2018%29.pdf

It will print:

20180301_123131.jpg
20180301_124304.jpg
20180301_124358.jpg
Cisco_WebEx_Add-On.dmg
Docker.dmg
Grammarly.dmg
Payslip 10459 %2828-02-2018%29.pdf

As we can see, the payslip even with space, shows in the correct line.


awk '{a=match($0, $3); print substr($0,a)}'

First you find the position of the start of the third column. With substr you will print the whole line ($0) starting at the position(in this case a) to the end of the line.


In AWK columns are called fields, hence NF is the key

all rows:

awk -F '<column separator>' '{print $(NF-2)}' <filename>

first row only:

awk -F '<column separator>' 'NR<=1{print $(NF-2)}' <filename>

In Bash you can use the following syntax with positional parameters:

while read -a cols; do echo ${cols[@]:2}; done < file.txt

Learn more: Handling positional parameters at Bash Hackers Wiki


awk '{$1=$2=""}1' FILENAME | sed 's/\s\+//g'

First two columns are cleared, sed removes leading spaces.


Perl solution:

perl -lane 'splice @F,0,2; print join " ",@F' file

These command-line options are used:

  • -n loop around every line of the input file, do not automatically print every line

  • -l removes newlines before processing, and adds them back in afterwards

  • -a autosplit mode – split input lines into the @F array. Defaults to splitting on whitespace

  • -e execute the perl code

splice @F,0,2 cleanly removes columns 0 and 1 from the @F array

join " ",@F joins the elements of the @F array, using a space in-between each element

If your input file is comma-delimited, rather than space-delimited, use -F, -lane


Python solution:

python -c "import sys;[sys.stdout.write(' '.join(line.split()[2:]) + '\n') for line in sys.stdin]" < file


awk '{for (i=4; i<=NF; i++)printf("%c", $i); printf("\n");}'

prints records starting from the 4th field to the last field in the same order they were in the original file


awk -v m="\x0a" -v N="3" '{$N=m$N ;print substr($0, index($0,m)+1)}'

This chops what is before the given field nr., N, and prints all the rest of the line, including field nr.N and maintaining the original spacing (it does not reformat). It doesn't mater if the string of the field appears also somewhere else in the line, which is the problem with daisaa's answer.

Define a function:

fromField () { 
awk -v m="\x0a" -v N="$1" '{$N=m$N; print substr($0,index($0,m)+1)}'
}

And use it like this:

$ echo "  bat   bi       iru   lau bost   " | fromField 3
iru   lau bost   
$ echo "  bat   bi       iru   lau bost   " | fromField 2
bi       iru   lau bost 

Output maintains everything, including trailing spaces

Works well for files where '/n' is the record separator so you don't have that new-line char inside the lines. If you want to use it with other record separators then use:

awk -v m="\x01" -v N="3" '{$N=m$N ;print substr($0, index($0,m)+1)}'

for example. Works well with almost all files as long as they don't use hexadecimal char nr. 1 inside the lines.


What about following line:

awk '{$1=$2=$3=""; print}' file

Based on @ghostdog74 suggestion. Mine should behave better when you filter lines, i.e.:

awk '/^exim4-config/ {$1=""; print }' file

If its only about ignoring the first two fields and if you don't want a space when masking those fields (like some of the answers above do) :

awk '{gsub($1" "$2" ",""); print;}' file

A bit late here, but none of the above seemed to work. Try this, using printf, inserts spaces between each. I chose to not have newline at the end.

awk '{for(i=3;i<=NF;++i) printf("%s ",  $i) }'

Well, you can easily accomplish the same effect using a regular expression. Assuming the separator is a space, it would look like:

awk '{ sub(/[^ ]+ +[^ ]+ +/, ""); print }'

Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to shell

Comparing a variable with a string python not working when redirecting from bash script Get first line of a shell command's output How to run shell script file using nodejs? Run bash command on jenkins pipeline Way to create multiline comments in Bash? How to do multiline shell script in Ansible How to check if a file exists in a shell script How to check if an environment variable exists and get its value? Curl to return http status code along with the response docker entrypoint running bash script gets "permission denied"

Examples related to unix

Docker CE on RHEL - Requires: container-selinux >= 2.9 What does `set -x` do? How to find files modified in last x minutes (find -mmin does not work as expected) sudo: npm: command not found How to sort a file in-place How to read a .properties file which contains keys that have a period character using Shell script gpg decryption fails with no secret key error Loop through a comma-separated shell variable Best way to find os name and version in Unix/Linux platform Resource u'tokenizers/punkt/english.pickle' not found

Examples related to multiple-columns

How to make div same height as parent (displayed as table-cell) Scrolling a flexbox with overflowing content How do I change Bootstrap 3 column order on mobile layout? Bootstrap 3 Multi-column within a single ul not floating properly Multiple input box excel VBA Unique Key constraints for multiple columns in Entity Framework Combine two or more columns in a dataframe into a new column with a new name Apply pandas function to column to create multiple new columns? mysqli_fetch_array while loop columns MySQL ORDER BY multiple column ASC and DESC