[linux] Print a file, skipping the first X lines, in Bash

I have a very long file which I want to print, skipping the first 1,000,000 lines, for example.

I looked into the cat man page, but I did not see any option to do this. I am looking for a command to do this or a simple Bash program.

This question is related to linux bash printing skip

The answer is


You can do this using the head and tail commands:

head -n <num> | tail -n <lines to print>

where num is 1e6 + the number of lines you want to print.


A less verbose version with AWK:

awk 'NR > 1e6' myfile.txt

But I would recommend using integer numbers.


Use the sed delete command with a range address. For example:

sed 1,100d file.txt # Print file.txt omitting lines 1-100.

Alternatively, if you want to only print a known range, use the print command with the -n flag:

sed -n 201,300p file.txt # Print lines 201-300 from file.txt

This solution should work reliably on all Unix systems, regardless of the presence of GNU utilities.


Easiest way I found to remove the first ten lines of a file:

$ sed 1,10d file.txt

In the general case (where X is the number of initial lines to delete, credit to commenters and editors for this):

$ sed 1,Xd file.txt

If you want to skip first two line:

tail -n +3 <filename>

If you want to skip first x line:

tail -n +$((x+1)) <filename>

I needed to do the same and found this thread.

I tried "tail -n +, but it just printed everything.

The more +lines worked nicely on the prompt, but it turned out it behaved totally different when run in headless mode (cronjob).

I finally wrote this myself:

skip=5
FILE="/tmp/filetoprint"
tail -n$((`cat "${FILE}" | wc -l` - skip)) "${FILE}"

This shell script works fine for me:

#!/bin/bash
awk -v initial_line=$1 -v end_line=$2 '{
    if (NR >= initial_line && NR <= end_line) 
    print $0
}' $3

Used with this sample file (file.txt):

one
two
three
four
five
six

The command (it will extract from second to fourth line in the file):

edu@debian5:~$./script.sh 2 4 file.txt

Output of this command:

two
three
four

Of course, you can improve it, for example by testing that all argument values are the expected :-)


Use:

sed -n '1d;p'

This command will delete the first line and print the rest.


If you want to see the first 10 lines you can use sed as below:

sed -n '1,10 p' myFile.txt

Or if you want to see lines from 20 to 30 you can use:

sed -n '20,30 p' myFile.txt

Just to propose a sed alternative. :) To skip first one million lines, try |sed '1,1000000d'.

Example:

$ perl -wle 'print for (1..1_000_005)'|sed '1,1000000d'
1000001
1000002
1000003
1000004
1000005

cat < File > | awk '{if(NR > 6) print $0}'

If you have GNU tail available on your system, you can do the following:

tail -n +1000001 huge-file.log

It's the + character that does what you want. To quote from the man page:

If the first character of K (the number of bytes or lines) is a `+', print beginning with the Kth item from the start of each file.

Thus, as noted in the comment, putting +1000001 starts printing with the first item after the first 1,000,000 lines.


Examples related to linux

grep's at sign caught as whitespace How to prevent Google Colab from disconnecting? "E: Unable to locate package python-pip" on Ubuntu 18.04 How to upgrade Python version to 3.7? Install Qt on Ubuntu Get first line of a shell command's output Cannot connect to the Docker daemon at unix:/var/run/docker.sock. Is the docker daemon running? Run bash command on jenkins pipeline How to uninstall an older PHP version from centOS7 How to update-alternatives to Python 3 without breaking apt?

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to printing

How do I print colored output with Python 3? Print a div content using Jquery Python 3 print without parenthesis How to find integer array size in java Differences Between vbLf, vbCrLf & vbCr Constants Printing variables in Python 3.4 Show DataFrame as table in iPython Notebook Removing display of row names from data frame Javascript window.print() in chrome, closing new window or tab instead of cancelling print leaves javascript blocked in parent window Print a div using javascript in angularJS single page application

Examples related to skip

PHP array value passes to next row Skip first couple of lines while reading lines in Python file Print a file, skipping the first X lines, in Bash