We all know how to use <ctrl>-R to reverse search through history, but did you know you can use <ctrl>-S to forward search if you set stty stop ""? Also, have you ever tried running bind -p to see all of your keyboard shortcuts listed? There are over 455 on Mac OS X by default.
What is your single most favorite obscure trick, keyboard shortcut or shopt configuration using bash?
This question is related to
bash
command-line
When navigating between two separate directories and copying files back and forth, I do this:
cd /some/where/long
src=`pwd`
cd /other/where/long
dest=`pwd`
cp $src/foo $dest
command completion will work by expanding the variable, so you can use tab completion to specify a file you're working with.
I like a splash of colour in my prompts:
export PS1="\[\033[07;31m\] \h \[\033[47;30m\] \W \[\033[00;31m\] \$ \[\e[m\]"
I'm afraid I don't have a screenshot for what that looks like, but it's supposed to be something like (all on one line):
[RED BACK WHITE TEXT] Computer name
[BLACK BACK WHITE TEXT] Working Directory
[WHITE BACK RED TEXT] $
Customise as per what you like to see :)
pbcopy
This copies to the Mac system clipboard. You can pipe commands to it...try:
pwd | pbcopy
<anything> | sort | uniq -c | sort -n
will give you a count of all the different occurrences of <anything>.
Often, awk, sed, or cut help with the parsing of data in <anything>.
I'm a fan of the !$, !^ and !* expandos, returning, from the most recent submitted command line: the last item, first non-command item, and all non-command items. To wit (Note that the shell prints out the command first):
$ echo foo bar baz
foo bar baz
$ echo bang-dollar: !$ bang-hat: !^ bang-star: !*
echo bang-dollar: baz bang-hat: foo bang-star: foo bar baz
bang-dollar: baz bang-hat: foo bang-star: foo bar baz
This comes in handy when you, say ls filea fileb, and want to edit one of them: vi !$ or both of them: vimdiff !*. It can also be generalized to "the nth argument" like so:
$ echo foo bar baz
$ echo !:2
echo bar
bar
Finally, with pathnames, you can get at parts of the path by appending :h and :t to any of the above expandos:
$ ls /usr/bin/id
/usr/bin/id
$ echo Head: !$:h Tail: !$:t
echo Head: /usr/bin Tail: id
Head: /usr/bin Tail: id
Using alias can be time-saving
alias myDir = "cd /this/is/a/long/directory; pwd"
Well, this may be a bit off topic, but if you are an Emacs user, I would say "emacs" is the most powerful trick... before you downvote this, try out "M-x shell" within an emacs instance... you get a shell inside emacs, and have all the power of emacs along with the power of a shell (there are some limitations, such as opening another emacs within it, but in most cases it is a lot more powerful than a vanilla bash prompt).
I like to construct commands with echo and pipe them to the shell:
$ find dir -name \*~ | xargs echo rm
...
$ find dir -name \*~ | xargs echo rm | ksh -s
Why? Because it allows me to look at what's going to be done before I do it. That way if I have a horrible error (like removing my home directory), I can catch it before it happens. Obviously, this is most important for destructive or irrevocable actions.
One preferred way of navigating when I'm using multiple directories in widely separate places in a tree hierarchy is to use acf_func.sh (listed below). Once defined, you can do
cd --
to see a list of recent directories, with a numerical menu
cd -2
to go to the second-most recent directory.
Very easy to use, very handy.
Here's the code:
# do ". acd_func.sh"
# acd_func 1.0.5, 10-nov-2004
# petar marinov, http:/geocities.com/h2428, this is public domain
cd_func ()
{
local x2 the_new_dir adir index
local -i cnt
if [[ $1 == "--" ]]; then
dirs -v
return 0
fi
the_new_dir=$1
[[ -z $1 ]] && the_new_dir=$HOME
if [[ ${the_new_dir:0:1} == '-' ]]; then
#
# Extract dir N from dirs
index=${the_new_dir:1}
[[ -z $index ]] && index=1
adir=$(dirs +$index)
[[ -z $adir ]] && return 1
the_new_dir=$adir
fi
#
# '~' has to be substituted by ${HOME}
[[ ${the_new_dir:0:1} == '~' ]] && the_new_dir="${HOME}${the_new_dir:1}"
#
# Now change to the new dir and add to the top of the stack
pushd "${the_new_dir}" > /dev/null
[[ $? -ne 0 ]] && return 1
the_new_dir=$(pwd)
#
# Trim down everything beyond 11th entry
popd -n +11 2>/dev/null 1>/dev/null
#
# Remove any other occurence of this dir, skipping the top of the stack
for ((cnt=1; cnt <= 10; cnt++)); do
x2=$(dirs +${cnt} 2>/dev/null)
[[ $? -ne 0 ]] && return 0
[[ ${x2:0:1} == '~' ]] && x2="${HOME}${x2:1}"
if [[ "${x2}" == "${the_new_dir}" ]]; then
popd -n +$cnt 2>/dev/null 1>/dev/null
cnt=cnt-1
fi
done
return 0
}
alias cd=cd_func
if [[ $BASH_VERSION > "2.05a" ]]; then
# ctrl+w shows the menu
bind -x "\"\C-w\":cd_func -- ;"
fi
Not really obscure, but one of the features I absolutely love is tab completion.
Really useful when you are navigating trough an entire subtree structure, or when you are using some obscure, or long command!
Want to get the last few lines of a log file?
tail /var/log/syslog
Want to keep an eye on a log file for when it changes?
tail -f /var/log/syslog
Want to quickly read over a file from the start?
more /var/log/syslog
Want to quickly find if a file contains some text?
grep "find this text" /var/log/syslog
If I am searching for something in a directory, but I am not sure of the file, then I just grep the files in the directory by:
find . -exec grep whatIWantToFind {} \;
The FIGNORE environment variable is nice when you want TAB completion to ignore files or folders with certain suffixes, e.g.:
export FIGNORE="CVS:.svn:~"
Use the IFS environment variable when you want to define an item separator other than space, e.g.:
export IFS="
"
This will make you able to loop through files and folders with spaces in them without performing any magic, like this:
$ touch "with spaces" withoutspaces
$ for i in `ls *`; do echo $i; done
with
spaces
withoutspaces
$ IFS="
"
$ for i in `ls *`; do echo $i; done
with spaces
withoutspaces
A simple thing to do when you realize you just typed the wrong line is hit Ctrl+C; if you want to keep the line, but need to execute something else first, begin a new line with a back slash - \, then Ctrl+C. The line will remain in your history.
I have various typographical error corrections in aliases
alias mkae=make
alias mroe=less
More of a novelty, but it's clever...
Top 10 commands used:
$ history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -nr | head
Sample output:
242 git
83 rake
43 cd
33 ss
24 ls
15 rsg
11 cap
10 dig
9 ping
3 vi
One preferred way of navigating when I'm using multiple directories in widely separate places in a tree hierarchy is to use acf_func.sh (listed below). Once defined, you can do
cd --
to see a list of recent directories, with a numerical menu
cd -2
to go to the second-most recent directory.
Very easy to use, very handy.
Here's the code:
# do ". acd_func.sh"
# acd_func 1.0.5, 10-nov-2004
# petar marinov, http:/geocities.com/h2428, this is public domain
cd_func ()
{
local x2 the_new_dir adir index
local -i cnt
if [[ $1 == "--" ]]; then
dirs -v
return 0
fi
the_new_dir=$1
[[ -z $1 ]] && the_new_dir=$HOME
if [[ ${the_new_dir:0:1} == '-' ]]; then
#
# Extract dir N from dirs
index=${the_new_dir:1}
[[ -z $index ]] && index=1
adir=$(dirs +$index)
[[ -z $adir ]] && return 1
the_new_dir=$adir
fi
#
# '~' has to be substituted by ${HOME}
[[ ${the_new_dir:0:1} == '~' ]] && the_new_dir="${HOME}${the_new_dir:1}"
#
# Now change to the new dir and add to the top of the stack
pushd "${the_new_dir}" > /dev/null
[[ $? -ne 0 ]] && return 1
the_new_dir=$(pwd)
#
# Trim down everything beyond 11th entry
popd -n +11 2>/dev/null 1>/dev/null
#
# Remove any other occurence of this dir, skipping the top of the stack
for ((cnt=1; cnt <= 10; cnt++)); do
x2=$(dirs +${cnt} 2>/dev/null)
[[ $? -ne 0 ]] && return 0
[[ ${x2:0:1} == '~' ]] && x2="${HOME}${x2:1}"
if [[ "${x2}" == "${the_new_dir}" ]]; then
popd -n +$cnt 2>/dev/null 1>/dev/null
cnt=cnt-1
fi
done
return 0
}
alias cd=cd_func
if [[ $BASH_VERSION > "2.05a" ]]; then
# ctrl+w shows the menu
bind -x "\"\C-w\":cd_func -- ;"
fi
I have various typographical error corrections in aliases
alias mkae=make
alias mroe=less
alias mycommand = 'verylongcommand -with -a -lot -of -parameters'
alias grep='grep --color'
find more than one word with grep :
netstat -c |grep 'msn\|skype\|icq'
I often have aliases for vi, ls, etc. but sometimes you want to escape the alias. Just add a back slash to the command in front:
Eg:
$ alias vi=vim
$ # To escape the alias for vi:
$ \vi # This doesn't open VIM
Cool, isn't it?
Using 'set -o vi' from the command line, or better, in .bashrc, puts you in vi editing mode on the command line. You start in 'insert' mode so you can type and backspace as normal, but if you make a 'large' mistake you can hit the esc key and then use 'b' and 'f' to move around as you do in vi. cw to change a word. Particularly useful after you've brought up a history command that you want to change.
Curly-Brace Expansion:
Really comes in handy when running a ./configure with a lot of options:
./configure --{prefix=/usr,mandir=/usr/man,{,sh}libdir=/usr/lib64,\
enable-{gpl,pthreads,bzlib,lib{faad{,bin},mp3lame,schroedinger,speex,theora,vorbis,xvid,x264},\
pic,shared,postproc,avfilter{-lavf,}},disable-static}
This is quite literally my configure settings for ffmpeg. Without the braces it's 409 characters.
Or, even better:
echo "I can count to a thousand!" ...{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}...
ESC.
Inserts the last arguments from your last bash command. It comes in handy more than you think.
cp file /to/some/long/path
cd ESC.
And this one is key for me actually:
set -o vi
/Allan
Shell-fu is a place for storing, moderating and propagating command line tips and tricks. A bit like StackOverflow, but solely for shell. You'll find plenty of answers to this question there.
pbcopy
This copies to the Mac system clipboard. You can pipe commands to it...try:
pwd | pbcopy
When downloading a large file I quite often do:
while ls -la <filename>; do sleep 5; done
And then just ctrl+c when I'm done (or if ls returns non-zero). It's similar to the watch program but it uses the shell instead, so it works on platforms without watch.
Another useful tool is netcat, or nc. If you do:
nc -l -p 9100 > printjob.prn
Then you can set up a printer on another computer but instead use the IP address of the computer running netcat. When the print job is sent, it is received by the computer running netcat and dumped into printjob.prn.
sudo !!
Runs the last command with administrator privileges.
^R reverse search. Hit ^R, type a fragment of a previous command you want to match, and hit ^R until you find the one you want. Then I don't have to remember recently used commands that are still in my history. Not exclusively bash, but also: ^E for end of line, ^A for beginning of line, ^U and ^K to delete before and after the cursor, respectively.
You changed to a new directory and want to move a file from the new directory to the old one. In one move: mv file $OLDPWD
Apropos history -- using cryptic carets, etc. is not entirely intuitive. To print all history items containing a given string:
function histgrep { fc -l -$((HISTSIZE-1)) | egrep "$@" ;}
When navigating between two separate directories and copying files back and forth, I do this:
cd /some/where/long
src=`pwd`
cd /other/where/long
dest=`pwd`
cp $src/foo $dest
command completion will work by expanding the variable, so you can use tab completion to specify a file you're working with.
Some useful mencoder commands I found out about when looking for some audio and video editing tools:
from .xxx to .avi
mencoder movie.wmv -o movie.avi -ovc lavc -oac lavc
Dump sound from a video:
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile fileout.mp3 filein.avi
My favorite is '^string^string2' which takes the last command, replaces string with string2 and executes it
$ ehco foo bar baz
bash: ehco: command not found
$ ^ehco^echo
foo bar baz
Custom Tab Completion (compgen and complete bash builtins)
Tab Completion is nice, but being able to apply it to more than just filenames is great. I have used it to create custom functions to expand arguments to commands I use all the time. For example, lets say you often need to add the FQDN as an argument to a command (e.g. ping blah.really.long.domain.name.foo.com). You can use compgen and complete to create a bash function that reads your /etc/hosts file for results so all you have to type then is:
ping blah.<tab>
and it will display all your current match options.
So basically anything that can return a word list can be used as a function.
CTRL+D quits the shell.
My favorite is '^string^string2' which takes the last command, replaces string with string2 and executes it
$ ehco foo bar baz
bash: ehco: command not found
$ ^ehco^echo
foo bar baz
rename
Example:
$ ls
this_has_text_to_find_1.txt
this_has_text_to_find_2.txt
this_has_text_to_find_3.txt
this_has_text_to_find_4.txt
$ rename 's/text_to_find/been_renamed/' *.txt
$ ls
this_has_been_renamed_1.txt
this_has_been_renamed_2.txt
this_has_been_renamed_3.txt
this_has_been_renamed_4.txt
So useful
When running a command with lots of output (like a big "make") I want to not only save the output, but also see it:
make install 2>&1 | tee E.make
This prevents less (less is more) from clearing the screen at the end of a file:
export LESS="-X"
You changed to a new directory and want to move a file from the new directory to the old one. In one move: mv file $OLDPWD
I have various typographical error corrections in aliases
alias mkae=make
alias mroe=less
Ctrl + L will usually clear the screen. Works from the Bash prompt (obviously) and in GDB, and a lot of other prompts.
!<first few characters of the command> will execute the last command which matches.
Example:
!b will run "build whatever -O -p -t -i -on"
!. will run ./a.out
It works best with long and repetitive commands, like compile, build, execute, etc. It saved me sooo much time when coding and testing.
I've always liked this one. Add this to your /etc/inputrc or ~/.inputrc
"\e[A":history-search-backward "\e[B":history-search-forward
When you type ls <up-arrow> it will be replaced with the last command starting with "ls " or whatever else you put in.
As a quick calculator, say to compute a percentage:
$ date
Thu Sep 18 12:55:33 EDT 2008
$ answers=60
$ curl "http://stackoverflow.com/questions/68372/what-are-some-of-your-favorite-command-line-tricks-using-bash" > tmp.html
$ words=`awk '/class="post-text"/ {s = s $0} \
> /<\/div>/ { gsub("<[^>]*>", "", s); print s; s = ""} \
> length(s) > 0 {s = s $0}' tmp.html \
> | awk '{n = n + NF} END {print n}'`
$ answers=`awk '/([0-9]+) Answers/ {sub("<h2>", "", $1); print $1}' tmp.html`
and finally:
$ echo $words words, $answers answers, $((words / $answers)) words per answer
4126 words, 60 answers, 68 words per answer
$
Not that division is truncated, not rounded. But often that's good enough for a quick calculation.
type -a PROG
in order to find all the places where PROG is available, usually somewhere in ~/bin rather than the one in /usr/bin/PROG that might have been expected.
I've always been partial to:
ctrl-E # move cursor to end of line
ctrl-A # move cursor to beginning of line
I also use shopt -s cdable_vars, then you can create bash variables to common directories. So, for my company's source tree, I create a bunch of variables like:
export Dcentmain="/var/localdata/p4ws/centaur/main/apps/core"
then I can change to that directory by cd Dcentmain.
bash can redirect to and from TCP/IP sockets. /dev/tcp/ and /dev/udp.
Some people think it's a security issue, but that's what OS level security like Solaris X's jail is for.
As Will Robertson notes, change prompt to do stuff... print the command # for !nn Set the Xterm terminal name. If it's an old Xterm that doesn't sniff traffic to set it's title.
A few years ago, I discovered the p* commands or get information about processes: ptree, pgrep, pkill, and pfiles. Of course, the mother of them all is ps, but you need to pipe the output into less, grep and/or awk to make sense of the output under heavy load. top (and variants) help too.
I use the following a lot:
The :p modifier to print a history result. E.g.
!!:p
Will print the last command so you can check that it's correct before running it again. Just enter !! to execute it.
In a similar vein:
!?foo?:p
Will search your history for the most recent command that contained the string 'foo' and print it.
If you don't need to print,
!?foo
does the search and executes it straight away.
Apropos history -- using cryptic carets, etc. is not entirely intuitive. To print all history items containing a given string:
function histgrep { fc -l -$((HISTSIZE-1)) | egrep "$@" ;}
find -iregex '.*\.py$\|.*\.xml$' | xargs egrep -niH 'a.search.pattern' | vi -R -
Searches a pattern in all Python files and all XML files and pipes the result in a readonly Vim session.
Curly-Brace Expansion:
Really comes in handy when running a ./configure with a lot of options:
./configure --{prefix=/usr,mandir=/usr/man,{,sh}libdir=/usr/lib64,\
enable-{gpl,pthreads,bzlib,lib{faad{,bin},mp3lame,schroedinger,speex,theora,vorbis,xvid,x264},\
pic,shared,postproc,avfilter{-lavf,}},disable-static}
This is quite literally my configure settings for ffmpeg. Without the braces it's 409 characters.
Or, even better:
echo "I can count to a thousand!" ...{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}{0,1,2,3,4,5,6,7,8,9}...
Signal trapping:
You can trap signals sent to the shell process and have them silently run commands in their respective environment as if typed on the command line:
# TERM or QUIT probably means the system is shutting down; make sure history is
# saved to $HISTFILE (does not do this by default)
trap 'logout' TERM QUIT
# save history when signalled by cron(1) script with USR1
trap 'history -a && history -n' USR1
You changed to a new directory and want to move a file from the new directory to the old one. In one move: mv file $OLDPWD
I'm a fan of the !$, !^ and !* expandos, returning, from the most recent submitted command line: the last item, first non-command item, and all non-command items. To wit (Note that the shell prints out the command first):
$ echo foo bar baz
foo bar baz
$ echo bang-dollar: !$ bang-hat: !^ bang-star: !*
echo bang-dollar: baz bang-hat: foo bang-star: foo bar baz
bang-dollar: baz bang-hat: foo bang-star: foo bar baz
This comes in handy when you, say ls filea fileb, and want to edit one of them: vi !$ or both of them: vimdiff !*. It can also be generalized to "the nth argument" like so:
$ echo foo bar baz
$ echo !:2
echo bar
bar
Finally, with pathnames, you can get at parts of the path by appending :h and :t to any of the above expandos:
$ ls /usr/bin/id
/usr/bin/id
$ echo Head: !$:h Tail: !$:t
echo Head: /usr/bin Tail: id
Head: /usr/bin Tail: id
I have various typographical error corrections in aliases
alias mkae=make
alias mroe=less
Not really obscure, but one of the features I absolutely love is tab completion.
Really useful when you are navigating trough an entire subtree structure, or when you are using some obscure, or long command!
You should be able to paste the following into a bash terminal window.
Display ANSI colour palette:
e="\033["
for f in 0 7 `seq 6`; do
no="" bo=""
for b in n 7 0 `seq 6`; do
co="3$f"; p=" "
[ $b = n ] || { co="$co;4$b";p=""; }
no="${no}${e}${co}m ${p}${co} ${e}0m"
bo="${bo}${e}1;${co}m ${p}1;${co} ${e}0m"
done
echo -e "$no\n$bo"
done
256 colour demo:
yes "$(seq 232 255;seq 254 -1 233)" |
while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done
cd -
It's the command-line equivalent of the back button (takes you to the previous directory you were in).
You can use the watch command in conjunction with another command to look for changes. An example of this was when I was testing my router, and I wanted to get up-to-date numbers on stuff like signal-to-noise ratio, etc.
watch --interval=10 lynx -dump http://dslrouter/stats.html
Custom Tab Completion (compgen and complete bash builtins)
Tab Completion is nice, but being able to apply it to more than just filenames is great. I have used it to create custom functions to expand arguments to commands I use all the time. For example, lets say you often need to add the FQDN as an argument to a command (e.g. ping blah.really.long.domain.name.foo.com). You can use compgen and complete to create a bash function that reads your /etc/hosts file for results so all you have to type then is:
ping blah.<tab>
and it will display all your current match options.
So basically anything that can return a word list can be used as a function.
I use the following a lot:
The :p modifier to print a history result. E.g.
!!:p
Will print the last command so you can check that it's correct before running it again. Just enter !! to execute it.
In a similar vein:
!?foo?:p
Will search your history for the most recent command that contained the string 'foo' and print it.
If you don't need to print,
!?foo
does the search and executes it straight away.
Want to get the last few lines of a log file?
tail /var/log/syslog
Want to keep an eye on a log file for when it changes?
tail -f /var/log/syslog
Want to quickly read over a file from the start?
more /var/log/syslog
Want to quickly find if a file contains some text?
grep "find this text" /var/log/syslog
!<first few characters of the command> will execute the last command which matches.
Example:
!b will run "build whatever -O -p -t -i -on"
!. will run ./a.out
It works best with long and repetitive commands, like compile, build, execute, etc. It saved me sooo much time when coding and testing.
If I am searching for something in a directory, but I am not sure of the file, then I just grep the files in the directory by:
find . -exec grep whatIWantToFind {} \;
This prevents less (less is more) from clearing the screen at the end of a file:
export LESS="-X"
I often have aliases for vi, ls, etc. but sometimes you want to escape the alias. Just add a back slash to the command in front:
Eg:
$ alias vi=vim
$ # To escape the alias for vi:
$ \vi # This doesn't open VIM
Cool, isn't it?
More of a novelty, but it's clever...
Top 10 commands used:
$ history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -nr | head
Sample output:
242 git
83 rake
43 cd
33 ss
24 ls
15 rsg
11 cap
10 dig
9 ping
3 vi
Mac only. This is simple, but MAN do I wish I had known about this years ago.
open ./
Opens the current directory in Finder. You can also use it to open any file with it's default application. Can also be used for URLs, but only if you prefix the URL with http://, which limits it's utility for opening the occasional random site.
To be able to quickly edit a shell script you know is in your $PATH (do not try with ls...):
function viscr { vi $(which $*); }
How to find which files match text, using find | grep -H In this example, which ruby file contains the jump string -
find . -name '*.rb' -exec grep -H jump {} \;
When navigating between two separate directories and copying files back and forth, I do this:
cd /some/where/long
src=`pwd`
cd /other/where/long
dest=`pwd`
cp $src/foo $dest
command completion will work by expanding the variable, so you can use tab completion to specify a file you're working with.
When running a command with lots of output (like a big "make") I want to not only save the output, but also see it:
make install 2>&1 | tee E.make
Good for making an exact recursive copy/backup of a directory including symlinks (rather than following them or ignoring them like cp):
$ mkdir new_dir
$ cd old_dir
$ tar cf - . | ( cd ../old_dir; tar xf - )
$ echo !$ !-2^ * Alt+Ctrl+e
$ echo aword someotherword * Ctrl+_
$ echo !$ !-2^ * Ctrl+x, *
$ echo !$ !-2^ LOG Makefile bar.c foo.h
&c.
Two of my favorites are:
1) Make tab-completion case insensitive (e.g. "cd /home/User " converts your command line to: "cd /home/user" if the latter exists and the former doesn't. You can turn it on with "set completion-ignore-case on" at the prompt, or add it permanently by adding "set completion-ignore-case on" to your .inputrc file.
2) The built-in 'type' command is like "which" but aware of aliases also. For example
$ type cdhome
cdhome is aliased to 'cd ~'
$ type bash
bash is /bin/bash
$ touch {1,2}.txt
$ ls [12].txt
1.txt 2.txt
$ rm !:1
rm [12].txt
$ history | tail -10
...
10007 touch {1,2}.txt
...
$ !10007
touch {1,2}.txt
$ for f in *.txt; do mv $f ${f/txt/doc}; done
Here's a couple of configuration tweaks:
~/.inputrc:
"\C-[[A": history-search-backward
"\C-[[B": history-search-forward
This works the same as ^R but using the arrow keys instead. This means I can type (e.g.) cd /media/ then hit up-arrow to go to the last thing I cd'd to inside the /media/ folder.
(I use Gnome Terminal, you may need to change the escape codes for other terminal emulators.)
Bash completion is also incredibly useful, but it's a far more subtle addition. In ~/.bashrc:
if [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
This will enable per-program tab-completion (e.g. attempting tab completion when the command line starts with evince will only show files that evince can open, and it will also tab-complete command line options).
Works nicely with this also in ~/.inputrc:
set completion-ignore-case on
set show-all-if-ambiguous on
set show-all-if-unmodified on
bash can redirect to and from TCP/IP sockets. /dev/tcp/ and /dev/udp.
Some people think it's a security issue, but that's what OS level security like Solaris X's jail is for.
As Will Robertson notes, change prompt to do stuff... print the command # for !nn Set the Xterm terminal name. If it's an old Xterm that doesn't sniff traffic to set it's title.
I often have aliases for vi, ls, etc. but sometimes you want to escape the alias. Just add a back slash to the command in front:
Eg:
$ alias vi=vim
$ # To escape the alias for vi:
$ \vi # This doesn't open VIM
Cool, isn't it?
$ touch {1,2}.txt
$ ls [12].txt
1.txt 2.txt
$ rm !:1
rm [12].txt
$ history | tail -10
...
10007 touch {1,2}.txt
...
$ !10007
touch {1,2}.txt
$ for f in *.txt; do mv $f ${f/txt/doc}; done
On Mac OS X,
ESC .
will cycle through recent arguments in place. That's: press and release ESC, then press and release . (period key). On Ubuntu, I think it's ALT+..
You can do that more than once, to go back through all your recent arguments. It's kind of like CTRL + R, but for arguments only. It's also much safer than !! or $!, since you see what you're going to get before you actually run the command.
sudo !!
Runs the last command with administrator privileges.
Quick Text
I use these sequences of text all too often, so I put shortcuts to them in by .inputrc:
# redirection short cuts
"\ew": "2>&1"
"\eq": "&>/dev/null &"
"\e\C-q": "2>/dev/null"
"\eg": "&>~/.garbage.out &"
"\e\C-g": "2>~/.garbage.out"
$if term=xterm
"\M-w": "2>&1"
"\M-q": "&>/dev/null &"
"\M-\C-q": "2>/dev/null"
"\M-g": "&>~/.garbage.out &"
"\M-\C-g": "2>~/.garbage.out"
$endif
Ctrl + L will usually clear the screen. Works from the Bash prompt (obviously) and in GDB, and a lot of other prompts.
One of my favorites tricks with bash is the "tar pipe". When you have a monstrous quantity of files to copy from one directory to another, doing "cp * /an/other/dir" doesn't work if the number of files is too high and explode the bash globber, so, the tar pipe :
(cd /path/to/source/dir/ ; tar cf - * ) | (cd /path/to/destination/ ; tar xf - )
...and if you have netcat, you can even do the "netcat tar pipe" through the network !!
Similar to many above, my current favorite is the keystroke [alt]. (Alt and "." keys together) this is the same as $! (Inserts the last argument from the previous command) except that it's immediate and for me easier to type. (Just can't be used in scripts)
eg:
mkdir -p /tmp/test/blah/oops/something
cd [alt].
If I am searching for something in a directory, but I am not sure of the file, then I just grep the files in the directory by:
find . -exec grep whatIWantToFind {} \;
du -a | sort -n | tail -99
to find the big files (or directories of files) to clean up to free up disk space.
pushd and popd almost always come in handy
I like a splash of colour in my prompts:
export PS1="\[\033[07;31m\] \h \[\033[47;30m\] \W \[\033[00;31m\] \$ \[\e[m\]"
I'm afraid I don't have a screenshot for what that looks like, but it's supposed to be something like (all on one line):
[RED BACK WHITE TEXT] Computer name
[BLACK BACK WHITE TEXT] Working Directory
[WHITE BACK RED TEXT] $
Customise as per what you like to see :)
Insert preceding lines final parameter
ALT-. the most useful key combination ever, try it and see, for some reason no one knows about this one.
Press it again and again to select older last parameters.
Great when you want to do something else to something you used just a moment ago.
Sure, you can "diff file1.txt file2.txt", but Bash supports process substitution, which allows you to diff the output of commands.
For example, let's say I want to make sure my script gives me the output I expect. I can just wrap my script in <( ) and feed it to diff to get a quick and dirty unit test:
$ cat myscript.sh
#!/bin/sh
echo -e "one\nthree"
$
$ ./myscript.sh
one
three
$
$ cat expected_output.txt
one
two
three
$
$ diff <(./myscript.sh) expected_output.txt
1a2
> two
$
As another example, let's say I want to check if two servers have the same list of RPMs installed. Rather than sshing to each server, writing each list of RPMs to separate files, and doing a diff on those files, I can just do the diff from my workstation:
$ diff <(ssh server1 'rpm -qa | sort') <(ssh server2 'rpm -qa | sort')
241c240
< kernel-2.6.18-92.1.6.el5
---
> kernel-2.6.18-92.el5
317d315
< libsmi-0.4.5-2.el5
727,728d724
< wireshark-0.99.7-1.el5
< wireshark-gnome-0.99.7-1.el5
$
There are more examples in the Advanced Bash-Scripting Guide at http://tldp.org/LDP/abs/html/process-sub.html.
bash can redirect to and from TCP/IP sockets. /dev/tcp/ and /dev/udp.
Some people think it's a security issue, but that's what OS level security like Solaris X's jail is for.
As Will Robertson notes, change prompt to do stuff... print the command # for !nn Set the Xterm terminal name. If it's an old Xterm that doesn't sniff traffic to set it's title.
Here's a couple of configuration tweaks:
~/.inputrc:
"\C-[[A": history-search-backward
"\C-[[B": history-search-forward
This works the same as ^R but using the arrow keys instead. This means I can type (e.g.) cd /media/ then hit up-arrow to go to the last thing I cd'd to inside the /media/ folder.
(I use Gnome Terminal, you may need to change the escape codes for other terminal emulators.)
Bash completion is also incredibly useful, but it's a far more subtle addition. In ~/.bashrc:
if [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
This will enable per-program tab-completion (e.g. attempting tab completion when the command line starts with evince will only show files that evince can open, and it will also tab-complete command line options).
Works nicely with this also in ~/.inputrc:
set completion-ignore-case on
set show-all-if-ambiguous on
set show-all-if-unmodified on
The easiest keystrokes for me for "last argument of the last command" is !$
echo what the heck?
what the heck?
echo !$
heck?
sudo !!
Runs the last command with administrator privileges.
More of a novelty, but it's clever...
Top 10 commands used:
$ history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -nr | head
Sample output:
242 git
83 rake
43 cd
33 ss
24 ls
15 rsg
11 cap
10 dig
9 ping
3 vi
while IFS= read -r line; do
echo "$line"
done < somefile.txt
This is a good way to process a file line by line. Clearing IFS is needed to get whitespace characters at the front or end of the line. The "-r" is needed to get all raw characters, including backslashes.
Not really obscure, but one of the features I absolutely love is tab completion.
Really useful when you are navigating trough an entire subtree structure, or when you are using some obscure, or long command!
I've always been partial to:
ctrl-E # move cursor to end of line
ctrl-A # move cursor to beginning of line
I also use shopt -s cdable_vars, then you can create bash variables to common directories. So, for my company's source tree, I create a bunch of variables like:
export Dcentmain="/var/localdata/p4ws/centaur/main/apps/core"
then I can change to that directory by cd Dcentmain.
I want to mention how we can redirect top command output to file using its batch mode (-b)
$ top -b -n 1 > top.out.$(date +%s)
By default, top is invoked using interactive mode in which top runs indefinitely and accepts keypress to redefine how top works.
A post I wrote can be found here
I prefer reading man pages in vi, so I have the following in my .profile or .bashrc file
man () {
sought=$*
/usr/bin/man $sought | col -b | vim -R -c "set nonumber" -c "set syntax=man" -
}
I like to construct commands with echo and pipe them to the shell:
$ find dir -name \*~ | xargs echo rm
...
$ find dir -name \*~ | xargs echo rm | ksh -s
Why? Because it allows me to look at what's going to be done before I do it. That way if I have a horrible error (like removing my home directory), I can catch it before it happens. Obviously, this is most important for destructive or irrevocable actions.
$_ (dollar underscore): the last word from the previous command. Similar to !$ except it doesn't put its substitution in your history like !$ does.
I have a really stupid, but extremely helpful one when navigating deep tree structures. Put this in .bashrc (or similar):
alias cd6="cd ../../../../../.."
alias cd5="cd ../../../../.."
alias cd4="cd ../../../.."
alias cd3="cd ../../.."
alias cd2="cd ../.."
$_ (dollar underscore): the last word from the previous command. Similar to !$ except it doesn't put its substitution in your history like !$ does.
ESC.
Inserts the last arguments from your last bash command. It comes in handy more than you think.
cp file /to/some/long/path
cd ESC.
I have plenty of directories which I want to access quickly, CDPATH variable is solution
that speed up my work-flow enormously:
export CDPATH=.:/home/gadolin/sth:/home/gadolin/dir1/importantDir
now with cd I can jump to any of sub directories of /home/gadolin/sth or /home/gadolin/dir1/importantDir without providing the full path. And also <tab> works here just like I would be there!
So if there are directories /home/gadolin/sth/1 /home/gadolin/sth/2, I type cd 1 wherever, and I am there.
Renaming/moving files with suffixes quickly:
cp /home/foo/realllylongname.cpp{,-old}
This expands to:
cp /home/foo/realllylongname.cpp /home/foo/realllylongname.cpp-old
This prevents less (less is more) from clearing the screen at the end of a file:
export LESS="-X"
String multiple commands together using the && command:
./run.sh && tail -f log.txt
or
kill -9 1111 && ./start.sh
du -a | sort -n | tail -99
to find the big files (or directories of files) to clean up to free up disk space.
You changed to a new directory and want to move a file from the new directory to the old one. In one move: mv file $OLDPWD
Similar to many above, my current favorite is the keystroke [alt]. (Alt and "." keys together) this is the same as $! (Inserts the last argument from the previous command) except that it's immediate and for me easier to type. (Just can't be used in scripts)
eg:
mkdir -p /tmp/test/blah/oops/something
cd [alt].
alias mycommand = 'verylongcommand -with -a -lot -of -parameters'
alias grep='grep --color'
find more than one word with grep :
netstat -c |grep 'msn\|skype\|icq'
Another favorite:
!!
Repeats your last command. Most useful in the form:
sudo !!
I have got a secret weapon : shell-fu.
There are thousand of smart tips, cool tricks and efficient recipes that most of the time fit on a single line.
One that I love (but I cheat a bit since I use the fact that Python is installed on most Unix system now) :
alias webshare='python -m SimpleHTTPServer'
Now everytime you type "webshare", the current directory will be available through the port 8000. Really nice when you want to share files with friends on a local network without usb key or remote dir. Streaming video and music will work too.
And of course the classic fork bomb that is completely useless but still a lot of fun :
$ :(){ :|:& };:
Don't try that in a production server...
<anything> | sort | uniq -c | sort -n
will give you a count of all the different occurrences of <anything>.
Often, awk, sed, or cut help with the parsing of data in <anything>.
Mac only. This is simple, but MAN do I wish I had known about this years ago.
open ./
Opens the current directory in Finder. You can also use it to open any file with it's default application. Can also be used for URLs, but only if you prefix the URL with http://, which limits it's utility for opening the occasional random site.
The FIGNORE environment variable is nice when you want TAB completion to ignore files or folders with certain suffixes, e.g.:
export FIGNORE="CVS:.svn:~"
Use the IFS environment variable when you want to define an item separator other than space, e.g.:
export IFS="
"
This will make you able to loop through files and folders with spaces in them without performing any magic, like this:
$ touch "with spaces" withoutspaces
$ for i in `ls *`; do echo $i; done
with
spaces
withoutspaces
$ IFS="
"
$ for i in `ls *`; do echo $i; done
with spaces
withoutspaces
CTRL+D quits the shell.
On Mac OS X,
ESC .
will cycle through recent arguments in place. That's: press and release ESC, then press and release . (period key). On Ubuntu, I think it's ALT+..
You can do that more than once, to go back through all your recent arguments. It's kind of like CTRL + R, but for arguments only. It's also much safer than !! or $!, since you see what you're going to get before you actually run the command.
Two of my favorites are:
1) Make tab-completion case insensitive (e.g. "cd /home/User " converts your command line to: "cd /home/user" if the latter exists and the former doesn't. You can turn it on with "set completion-ignore-case on" at the prompt, or add it permanently by adding "set completion-ignore-case on" to your .inputrc file.
2) The built-in 'type' command is like "which" but aware of aliases also. For example
$ type cdhome
cdhome is aliased to 'cd ~'
$ type bash
bash is /bin/bash
Here's a couple of configuration tweaks:
~/.inputrc:
"\C-[[A": history-search-backward
"\C-[[B": history-search-forward
This works the same as ^R but using the arrow keys instead. This means I can type (e.g.) cd /media/ then hit up-arrow to go to the last thing I cd'd to inside the /media/ folder.
(I use Gnome Terminal, you may need to change the escape codes for other terminal emulators.)
Bash completion is also incredibly useful, but it's a far more subtle addition. In ~/.bashrc:
if [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
This will enable per-program tab-completion (e.g. attempting tab completion when the command line starts with evince will only show files that evince can open, and it will also tab-complete command line options).
Works nicely with this also in ~/.inputrc:
set completion-ignore-case on
set show-all-if-ambiguous on
set show-all-if-unmodified on
I use the following a lot:
The :p modifier to print a history result. E.g.
!!:p
Will print the last command so you can check that it's correct before running it again. Just enter !! to execute it.
In a similar vein:
!?foo?:p
Will search your history for the most recent command that contained the string 'foo' and print it.
If you don't need to print,
!?foo
does the search and executes it straight away.
<anything> | sort | uniq -c | sort -n
will give you a count of all the different occurrences of <anything>.
Often, awk, sed, or cut help with the parsing of data in <anything>.
Apropos history -- using cryptic carets, etc. is not entirely intuitive. To print all history items containing a given string:
function histgrep { fc -l -$((HISTSIZE-1)) | egrep "$@" ;}
Not really interactive shell tricks, but valid nonetheless as tricks for writing good scripts.
getopts, shift, $OPTIND, $OPTARG:
I love making customizable scripts:
while getopts 'vo:' flag; do
case "$flag" in
'v')
VERBOSE=1
;;
'o')
OUT="$OPTARG"
;;
esac
done
shift "$((OPTIND-1))"
xargs(1):
I have a triple-core processor and like to run scripts that perform compression, or some other CPU-intensive serial operation on a set of files. I like to speed it up using xargs as a job queue.
if [ "$#" -gt 1 ]; then
# schedule using xargs
(for file; do
echo -n "$file"
echo -ne '\0'
done) |xargs -0 -n 1 -P "$NUM_JOBS" -- "$0"
else
# do the actual processing
fi
This acts a lot like make -j [NUM_JOBS].
Programmable Completion:
Nothing fancy. I always disable it when I'm using Knoppix because it gets in the way too often. Just some basic ones:
shopt -s progcomp
complete -A stopped -P '%' bg
complete -A job -P '%' fg jobs disown wait
complete -A variable readonly export
complete -A variable -A function unset
complete -A setopt set
complete -A shopt shopt
complete -A helptopic help
complete -A alias alias unalias
complete -A binding bind
complete -A command type which \
killall pidof
complete -A builtin builtin
complete -A disabled enable
Eliminate duplicate lines from a file
#sort -u filename > filename.new
List all lines that do not match a condition
#grep -v ajsk filename
These are not necessarily Bash specific (but hey neither is ls -thor :) )
Some other useful cmds:
prtdiag, psrinfo, prtconf - more info here and here (posts on my blog).
rename
Example:
$ ls
this_has_text_to_find_1.txt
this_has_text_to_find_2.txt
this_has_text_to_find_3.txt
this_has_text_to_find_4.txt
$ rename 's/text_to_find/been_renamed/' *.txt
$ ls
this_has_been_renamed_1.txt
this_has_been_renamed_2.txt
this_has_been_renamed_3.txt
this_has_been_renamed_4.txt
So useful
Want to get the last few lines of a log file?
tail /var/log/syslog
Want to keep an eye on a log file for when it changes?
tail -f /var/log/syslog
Want to quickly read over a file from the start?
more /var/log/syslog
Want to quickly find if a file contains some text?
grep "find this text" /var/log/syslog
SSH tunnel:
ssh -fNR 1234:localhost:22 [email protected]
Sure, you can "diff file1.txt file2.txt", but Bash supports process substitution, which allows you to diff the output of commands.
For example, let's say I want to make sure my script gives me the output I expect. I can just wrap my script in <( ) and feed it to diff to get a quick and dirty unit test:
$ cat myscript.sh
#!/bin/sh
echo -e "one\nthree"
$
$ ./myscript.sh
one
three
$
$ cat expected_output.txt
one
two
three
$
$ diff <(./myscript.sh) expected_output.txt
1a2
> two
$
As another example, let's say I want to check if two servers have the same list of RPMs installed. Rather than sshing to each server, writing each list of RPMs to separate files, and doing a diff on those files, I can just do the diff from my workstation:
$ diff <(ssh server1 'rpm -qa | sort') <(ssh server2 'rpm -qa | sort')
241c240
< kernel-2.6.18-92.1.6.el5
---
> kernel-2.6.18-92.el5
317d315
< libsmi-0.4.5-2.el5
727,728d724
< wireshark-0.99.7-1.el5
< wireshark-gnome-0.99.7-1.el5
$
There are more examples in the Advanced Bash-Scripting Guide at http://tldp.org/LDP/abs/html/process-sub.html.
To be able to quickly edit a shell script you know is in your $PATH (do not try with ls...):
function viscr { vi $(which $*); }
How to list only subdirectories in the current one ?
ls -d */
It's a simple trick, but you wouldn't know how much time I needed to find that one !
Using 'set -o vi' from the command line, or better, in .bashrc, puts you in vi editing mode on the command line. You start in 'insert' mode so you can type and backspace as normal, but if you make a 'large' mistake you can hit the esc key and then use 'b' and 'f' to move around as you do in vi. cw to change a word. Particularly useful after you've brought up a history command that you want to change.
I've always liked this one. Add this to your /etc/inputrc or ~/.inputrc
"\e[A":history-search-backward "\e[B":history-search-forward
When you type ls <up-arrow> it will be replaced with the last command starting with "ls " or whatever else you put in.
How to list only subdirectories in the current one ?
ls -d */
It's a simple trick, but you wouldn't know how much time I needed to find that one !
Renaming/moving files with suffixes quickly:
cp /home/foo/realllylongname.cpp{,-old}
This expands to:
cp /home/foo/realllylongname.cpp /home/foo/realllylongname.cpp-old
Two of my favorites are:
1) Make tab-completion case insensitive (e.g. "cd /home/User " converts your command line to: "cd /home/user" if the latter exists and the former doesn't. You can turn it on with "set completion-ignore-case on" at the prompt, or add it permanently by adding "set completion-ignore-case on" to your .inputrc file.
2) The built-in 'type' command is like "which" but aware of aliases also. For example
$ type cdhome
cdhome is aliased to 'cd ~'
$ type bash
bash is /bin/bash
My favorite is '^string^string2' which takes the last command, replaces string with string2 and executes it
$ ehco foo bar baz
bash: ehco: command not found
$ ^ehco^echo
foo bar baz
find -iregex '.*\.py$\|.*\.xml$' | xargs egrep -niH 'a.search.pattern' | vi -R -
Searches a pattern in all Python files and all XML files and pipes the result in a readonly Vim session.
Using alias can be time-saving
alias myDir = "cd /this/is/a/long/directory; pwd"
type -a PROG
in order to find all the places where PROG is available, usually somewhere in ~/bin rather than the one in /usr/bin/PROG that might have been expected.
Good for making an exact recursive copy/backup of a directory including symlinks (rather than following them or ignoring them like cp):
$ mkdir new_dir
$ cd old_dir
$ tar cf - . | ( cd ../old_dir; tar xf - )
More of a novelty, but it's clever...
Top 10 commands used:
$ history | awk '{print $2}' | awk 'BEGIN {FS="|"}{print $1}' | sort | uniq -c | sort -nr | head
Sample output:
242 git
83 rake
43 cd
33 ss
24 ls
15 rsg
11 cap
10 dig
9 ping
3 vi
I often have aliases for vi, ls, etc. but sometimes you want to escape the alias. Just add a back slash to the command in front:
Eg:
$ alias vi=vim
$ # To escape the alias for vi:
$ \vi # This doesn't open VIM
Cool, isn't it?
type -a PROG
in order to find all the places where PROG is available, usually somewhere in ~/bin rather than the one in /usr/bin/PROG that might have been expected.
Want to get the last few lines of a log file?
tail /var/log/syslog
Want to keep an eye on a log file for when it changes?
tail -f /var/log/syslog
Want to quickly read over a file from the start?
more /var/log/syslog
Want to quickly find if a file contains some text?
grep "find this text" /var/log/syslog
rename
Example:
$ ls
this_has_text_to_find_1.txt
this_has_text_to_find_2.txt
this_has_text_to_find_3.txt
this_has_text_to_find_4.txt
$ rename 's/text_to_find/been_renamed/' *.txt
$ ls
this_has_been_renamed_1.txt
this_has_been_renamed_2.txt
this_has_been_renamed_3.txt
this_has_been_renamed_4.txt
So useful
You can use the watch command in conjunction with another command to look for changes. An example of this was when I was testing my router, and I wanted to get up-to-date numbers on stuff like signal-to-noise ratio, etc.
watch --interval=10 lynx -dump http://dslrouter/stats.html
A few years ago, I discovered the p* commands or get information about processes: ptree, pgrep, pkill, and pfiles. Of course, the mother of them all is ps, but you need to pipe the output into less, grep and/or awk to make sense of the output under heavy load. top (and variants) help too.
Signal trapping:
You can trap signals sent to the shell process and have them silently run commands in their respective environment as if typed on the command line:
# TERM or QUIT probably means the system is shutting down; make sure history is
# saved to $HISTFILE (does not do this by default)
trap 'logout' TERM QUIT
# save history when signalled by cron(1) script with USR1
trap 'history -a && history -n' USR1
I want to mention how we can redirect top command output to file using its batch mode (-b)
$ top -b -n 1 > top.out.$(date +%s)
By default, top is invoked using interactive mode in which top runs indefinitely and accepts keypress to redefine how top works.
A post I wrote can be found here
One of my favorites tricks with bash is the "tar pipe". When you have a monstrous quantity of files to copy from one directory to another, doing "cp * /an/other/dir" doesn't work if the number of files is too high and explode the bash globber, so, the tar pipe :
(cd /path/to/source/dir/ ; tar cf - * ) | (cd /path/to/destination/ ; tar xf - )
...and if you have netcat, you can even do the "netcat tar pipe" through the network !!
pbcopy and pbpaste aliases for GNU/Linux
alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'
alias -- ddt='ls -trFld'
dt () { ddt --color "$@" | tail -n 30; }
Gives you the most recent files in the current directory. I use it all the time...
I'm a fan of the !$, !^ and !* expandos, returning, from the most recent submitted command line: the last item, first non-command item, and all non-command items. To wit (Note that the shell prints out the command first):
$ echo foo bar baz
foo bar baz
$ echo bang-dollar: !$ bang-hat: !^ bang-star: !*
echo bang-dollar: baz bang-hat: foo bang-star: foo bar baz
bang-dollar: baz bang-hat: foo bang-star: foo bar baz
This comes in handy when you, say ls filea fileb, and want to edit one of them: vi !$ or both of them: vimdiff !*. It can also be generalized to "the nth argument" like so:
$ echo foo bar baz
$ echo !:2
echo bar
bar
Finally, with pathnames, you can get at parts of the path by appending :h and :t to any of the above expandos:
$ ls /usr/bin/id
/usr/bin/id
$ echo Head: !$:h Tail: !$:t
echo Head: /usr/bin Tail: id
Head: /usr/bin Tail: id
My favorite command is "ls -thor"
It summons the power of the gods to list the most recently modified files in a conveniently readable format.
alias -- ddt='ls -trFld'
dt () { ddt --color "$@" | tail -n 30; }
Gives you the most recent files in the current directory. I use it all the time...
Using history substiution characters !# to access the current command line, in combination with ^, $, etc.
E.g. move a file out of the way with an "old-" prefix:
mv file-with-long-name-typed-with-tab-completion.txt old-!#^
If I am searching for something in a directory, but I am not sure of the file, then I just grep the files in the directory by:
find . -exec grep whatIWantToFind {} \;
How to list only subdirectories in the current one ?
ls -d */
It's a simple trick, but you wouldn't know how much time I needed to find that one !
while IFS= read -r line; do
echo "$line"
done < somefile.txt
This is a good way to process a file line by line. Clearing IFS is needed to get whitespace characters at the front or end of the line. The "-r" is needed to get all raw characters, including backslashes.
Top 10 commands again (like ctcherry's post, only shorter):
history | awk '{ print $2 }' | sort | uniq -c |sort -rn | head
<anything> | sort | uniq -c | sort -n
will give you a count of all the different occurrences of <anything>.
Often, awk, sed, or cut help with the parsing of data in <anything>.
Not really interactive shell tricks, but valid nonetheless as tricks for writing good scripts.
getopts, shift, $OPTIND, $OPTARG:
I love making customizable scripts:
while getopts 'vo:' flag; do
case "$flag" in
'v')
VERBOSE=1
;;
'o')
OUT="$OPTARG"
;;
esac
done
shift "$((OPTIND-1))"
xargs(1):
I have a triple-core processor and like to run scripts that perform compression, or some other CPU-intensive serial operation on a set of files. I like to speed it up using xargs as a job queue.
if [ "$#" -gt 1 ]; then
# schedule using xargs
(for file; do
echo -n "$file"
echo -ne '\0'
done) |xargs -0 -n 1 -P "$NUM_JOBS" -- "$0"
else
# do the actual processing
fi
This acts a lot like make -j [NUM_JOBS].
$ echo !$ !-2^ * Alt+Ctrl+e
$ echo aword someotherword * Ctrl+_
$ echo !$ !-2^ * Ctrl+x, *
$ echo !$ !-2^ LOG Makefile bar.c foo.h
&c.
And this one is key for me actually:
set -o vi
/Allan
type -a PROG
in order to find all the places where PROG is available, usually somewhere in ~/bin rather than the one in /usr/bin/PROG that might have been expected.
pbcopy and pbpaste aliases for GNU/Linux
alias pbcopy='xclip -selection clipboard'
alias pbpaste='xclip -selection clipboard -o'
A simple thing to do when you realize you just typed the wrong line is hit Ctrl+C; if you want to keep the line, but need to execute something else first, begin a new line with a back slash - \, then Ctrl+C. The line will remain in your history.
Eliminate duplicate lines from a file
#sort -u filename > filename.new
List all lines that do not match a condition
#grep -v ajsk filename
These are not necessarily Bash specific (but hey neither is ls -thor :) )
Some other useful cmds:
prtdiag, psrinfo, prtconf - more info here and here (posts on my blog).
A few years ago, I discovered the p* commands or get information about processes: ptree, pgrep, pkill, and pfiles. Of course, the mother of them all is ps, but you need to pipe the output into less, grep and/or awk to make sense of the output under heavy load. top (and variants) help too.
My favorite is '^string^string2' which takes the last command, replaces string with string2 and executes it
$ ehco foo bar baz
bash: ehco: command not found
$ ^ehco^echo
foo bar baz
pushd and popd almost always come in handy
To be able to quickly edit a shell script you know is in your $PATH (do not try with ls...):
function viscr { vi $(which $*); }
Using history substiution characters !# to access the current command line, in combination with ^, $, etc.
E.g. move a file out of the way with an "old-" prefix:
mv file-with-long-name-typed-with-tab-completion.txt old-!#^
A simple thing to do when you realize you just typed the wrong line is hit Ctrl+C; if you want to keep the line, but need to execute something else first, begin a new line with a back slash - \, then Ctrl+C. The line will remain in your history.
How to find which files match text, using find | grep -H In this example, which ruby file contains the jump string -
find . -name '*.rb' -exec grep -H jump {} \;
You can use the watch command in conjunction with another command to look for changes. An example of this was when I was testing my router, and I wanted to get up-to-date numbers on stuff like signal-to-noise ratio, etc.
watch --interval=10 lynx -dump http://dslrouter/stats.html
Another favorite:
!!
Repeats your last command. Most useful in the form:
sudo !!
String multiple commands together using the && command:
./run.sh && tail -f log.txt
or
kill -9 1111 && ./start.sh
For the sheer humor factor, create an empty file "myself" and then: $ touch myself
I'm a fan of the !$, !^ and !* expandos, returning, from the most recent submitted command line: the last item, first non-command item, and all non-command items. To wit (Note that the shell prints out the command first):
$ echo foo bar baz
foo bar baz
$ echo bang-dollar: !$ bang-hat: !^ bang-star: !*
echo bang-dollar: baz bang-hat: foo bang-star: foo bar baz
bang-dollar: baz bang-hat: foo bang-star: foo bar baz
This comes in handy when you, say ls filea fileb, and want to edit one of them: vi !$ or both of them: vimdiff !*. It can also be generalized to "the nth argument" like so:
$ echo foo bar baz
$ echo !:2
echo bar
bar
Finally, with pathnames, you can get at parts of the path by appending :h and :t to any of the above expandos:
$ ls /usr/bin/id
/usr/bin/id
$ echo Head: !$:h Tail: !$:t
echo Head: /usr/bin Tail: id
Head: /usr/bin Tail: id
You can use the watch command in conjunction with another command to look for changes. An example of this was when I was testing my router, and I wanted to get up-to-date numbers on stuff like signal-to-noise ratio, etc.
watch --interval=10 lynx -dump http://dslrouter/stats.html
Duplicate file finder
This will run checksums recursively from the current directory, and give back the filenames of all identical checksum results:
find ./ -type f -print0 | xargs -0 -n1 md5sum | sort -k 1,32 | uniq -w 32 -d --all-repeated=separate | sed -e 's/^[0-9a-f]*\ *//;'
You can, of course, change the path around.
Maybe put it into a function or alias, and pass in the target path as a parameter.
When downloading a large file I quite often do:
while ls -la <filename>; do sleep 5; done
And then just ctrl+c when I'm done (or if ls returns non-zero). It's similar to the watch program but it uses the shell instead, so it works on platforms without watch.
Another useful tool is netcat, or nc. If you do:
nc -l -p 9100 > printjob.prn
Then you can set up a printer on another computer but instead use the IP address of the computer running netcat. When the print job is sent, it is received by the computer running netcat and dumped into printjob.prn.
alias mycommand = 'verylongcommand -with -a -lot -of -parameters'
alias grep='grep --color'
find more than one word with grep :
netstat -c |grep 'msn\|skype\|icq'
For the sheer humor factor, create an empty file "myself" and then: $ touch myself
And this one is key for me actually:
set -o vi
/Allan
I like a splash of colour in my prompts:
export PS1="\[\033[07;31m\] \h \[\033[47;30m\] \W \[\033[00;31m\] \$ \[\e[m\]"
I'm afraid I don't have a screenshot for what that looks like, but it's supposed to be something like (all on one line):
[RED BACK WHITE TEXT] Computer name
[BLACK BACK WHITE TEXT] Working Directory
[WHITE BACK RED TEXT] $
Customise as per what you like to see :)
You should be able to paste the following into a bash terminal window.
Display ANSI colour palette:
e="\033["
for f in 0 7 `seq 6`; do
no="" bo=""
for b in n 7 0 `seq 6`; do
co="3$f"; p=" "
[ $b = n ] || { co="$co;4$b";p=""; }
no="${no}${e}${co}m ${p}${co} ${e}0m"
bo="${bo}${e}1;${co}m ${p}1;${co} ${e}0m"
done
echo -e "$no\n$bo"
done
256 colour demo:
yes "$(seq 232 255;seq 254 -1 233)" |
while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done
My favorite command is "ls -thor"
It summons the power of the gods to list the most recently modified files in a conveniently readable format.
Since I always need the for i in $(ls) statement I made a shortcut:
fea(){
if test -z ${2:0:1}; then action=echo; else action=$2; fi
for i in $(ls $1);
do $action $i ;
done;
}
Another one is:
echo ${!B*}
It will print a list of all defined variables that start with 'B'.
ESC.
Inserts the last arguments from your last bash command. It comes in handy more than you think.
cp file /to/some/long/path
cd ESC.
The FIGNORE environment variable is nice when you want TAB completion to ignore files or folders with certain suffixes, e.g.:
export FIGNORE="CVS:.svn:~"
Use the IFS environment variable when you want to define an item separator other than space, e.g.:
export IFS="
"
This will make you able to loop through files and folders with spaces in them without performing any magic, like this:
$ touch "with spaces" withoutspaces
$ for i in `ls *`; do echo $i; done
with
spaces
withoutspaces
$ IFS="
"
$ for i in `ls *`; do echo $i; done
with spaces
withoutspaces
!<first few characters of the command> will execute the last command which matches.
Example:
!b will run "build whatever -O -p -t -i -on"
!. will run ./a.out
It works best with long and repetitive commands, like compile, build, execute, etc. It saved me sooo much time when coding and testing.
Delete everything except important-file:
# shopt -s extglob
# rm -rf !(important-file)
The same in zsh:
# rm -rf *~important-file
Bevore I knew that I had to move the important fiels to an other dictionary, delete everything and move the important back again.
Top 10 commands again (like ctcherry's post, only shorter):
history | awk '{ print $2 }' | sort | uniq -c |sort -rn | head
Not really obscure, but one of the features I absolutely love is tab completion.
Really useful when you are navigating trough an entire subtree structure, or when you are using some obscure, or long command!
Custom Tab Completion (compgen and complete bash builtins)
Tab Completion is nice, but being able to apply it to more than just filenames is great. I have used it to create custom functions to expand arguments to commands I use all the time. For example, lets say you often need to add the FQDN as an argument to a command (e.g. ping blah.really.long.domain.name.foo.com). You can use compgen and complete to create a bash function that reads your /etc/hosts file for results so all you have to type then is:
ping blah.<tab>
and it will display all your current match options.
So basically anything that can return a word list can be used as a function.
I prefer reading man pages in vi, so I have the following in my .profile or .bashrc file
man () {
sought=$*
/usr/bin/man $sought | col -b | vim -R -c "set nonumber" -c "set syntax=man" -
}
I like to set a prompt which shows the current directory in the window title of an xterm. It also shows the time and current directory. In addition, if bash wants to report that a background job has finished, it is reported in a different colour using ANSI escape sequences. I use a black-on-light console so my colours may not be right for you if you favour light-on-black.
PROMPT_COMMAND='echo -e "\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~}\007\033[1;31m${PWD/#$HOME/~}\033[1;34m"'
PS1='\[\e[1;31m\]\t \$ \[\e[0m\]'
Make sure you understand how to use \[ and \] correctly in your PS1 string so that bash knows how long your prompt-string actually renders on screen. This is so it can redraw your command-line correctly when you move beyond a single line command.
Some useful mencoder commands I found out about when looking for some audio and video editing tools:
from .xxx to .avi
mencoder movie.wmv -o movie.avi -ovc lavc -oac lavc
Dump sound from a video:
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile fileout.mp3 filein.avi
^R reverse search. Hit ^R, type a fragment of a previous command you want to match, and hit ^R until you find the one you want. Then I don't have to remember recently used commands that are still in my history. Not exclusively bash, but also: ^E for end of line, ^A for beginning of line, ^U and ^K to delete before and after the cursor, respectively.
Some Bash nuggets also here:
Someone else recommended "M-x shell RET" in Emacs. I think "M-x eshell RET" is even better.
Delete everything except important-file:
# shopt -s extglob
# rm -rf !(important-file)
The same in zsh:
# rm -rf *~important-file
Bevore I knew that I had to move the important fiels to an other dictionary, delete everything and move the important back again.
# Batch extension renamer (usage: renamer txt mkd)
renamer() {
local fn
for fn in *."$1"; do
mv "$fn" "${fn%.*}"."$2"
done
}
Some Bash nuggets also here:
$_ (dollar underscore): the last word from the previous command. Similar to !$ except it doesn't put its substitution in your history like !$ does.
The easiest keystrokes for me for "last argument of the last command" is !$
echo what the heck?
what the heck?
echo !$
heck?
When downloading a large file I quite often do:
while ls -la <filename>; do sleep 5; done
And then just ctrl+c when I'm done (or if ls returns non-zero). It's similar to the watch program but it uses the shell instead, so it works on platforms without watch.
Another useful tool is netcat, or nc. If you do:
nc -l -p 9100 > printjob.prn
Then you can set up a printer on another computer but instead use the IP address of the computer running netcat. When the print job is sent, it is received by the computer running netcat and dumped into printjob.prn.
To be able to quickly edit a shell script you know is in your $PATH (do not try with ls...):
function viscr { vi $(which $*); }
When running a command with lots of output (like a big "make") I want to not only save the output, but also see it:
make install 2>&1 | tee E.make
Ctrl + L will usually clear the screen. Works from the Bash prompt (obviously) and in GDB, and a lot of other prompts.
Top 10 commands again (like ctcherry's post, only shorter):
history | awk '{ print $2 }' | sort | uniq -c |sort -rn | head
I'm a big fan of Bash job control, mainly the use of Control-Z and fg, especially if I'm doing development in a terminal. If I've got emacs open and need to compile, deploy, etc. I just Control-Z to suspend emacs, do what I need, and fg to bring it back. This keeps all of the emacs buffers intact and makes things much easier than re-launching whatever I'm doing.
cd -
It's the command-line equivalent of the back button (takes you to the previous directory you were in).
I like to construct commands with echo and pipe them to the shell:
$ find dir -name \*~ | xargs echo rm
...
$ find dir -name \*~ | xargs echo rm | ksh -s
Why? Because it allows me to look at what's going to be done before I do it. That way if I have a horrible error (like removing my home directory), I can catch it before it happens. Obviously, this is most important for destructive or irrevocable actions.
String multiple commands together using the && command:
./run.sh && tail -f log.txt
or
kill -9 1111 && ./start.sh
I've always been partial to:
ctrl-E # move cursor to end of line
ctrl-A # move cursor to beginning of line
I also use shopt -s cdable_vars, then you can create bash variables to common directories. So, for my company's source tree, I create a bunch of variables like:
export Dcentmain="/var/localdata/p4ws/centaur/main/apps/core"
then I can change to that directory by cd Dcentmain.
Renaming/moving files with suffixes quickly:
cp /home/foo/realllylongname.cpp{,-old}
This expands to:
cp /home/foo/realllylongname.cpp /home/foo/realllylongname.cpp-old
I've always liked this one. Add this to your /etc/inputrc or ~/.inputrc
"\e[A":history-search-backward "\e[B":history-search-forward
When you type ls <up-arrow> it will be replaced with the last command starting with "ls " or whatever else you put in.
Using 'set -o vi' from the command line, or better, in .bashrc, puts you in vi editing mode on the command line. You start in 'insert' mode so you can type and backspace as normal, but if you make a 'large' mistake you can hit the esc key and then use 'b' and 'f' to move around as you do in vi. cw to change a word. Particularly useful after you've brought up a history command that you want to change.
alias -- ddt='ls -trFld'
dt () { ddt --color "$@" | tail -n 30; }
Gives you the most recent files in the current directory. I use it all the time...
Apropos history -- using cryptic carets, etc. is not entirely intuitive. To print all history items containing a given string:
function histgrep { fc -l -$((HISTSIZE-1)) | egrep "$@" ;}
ctrl-u delete all written stuff
$_ (dollar underscore): the last word from the previous command. Similar to !$ except it doesn't put its substitution in your history like !$ does.
Using alias can be time-saving
alias myDir = "cd /this/is/a/long/directory; pwd"
As a quick calculator, say to compute a percentage:
$ date
Thu Sep 18 12:55:33 EDT 2008
$ answers=60
$ curl "http://stackoverflow.com/questions/68372/what-are-some-of-your-favorite-command-line-tricks-using-bash" > tmp.html
$ words=`awk '/class="post-text"/ {s = s $0} \
> /<\/div>/ { gsub("<[^>]*>", "", s); print s; s = ""} \
> length(s) > 0 {s = s $0}' tmp.html \
> | awk '{n = n + NF} END {print n}'`
$ answers=`awk '/([0-9]+) Answers/ {sub("<h2>", "", $1); print $1}' tmp.html`
and finally:
$ echo $words words, $answers answers, $((words / $answers)) words per answer
4126 words, 60 answers, 68 words per answer
$
Not that division is truncated, not rounded. But often that's good enough for a quick calculation.
My favorite command is "ls -thor"
It summons the power of the gods to list the most recently modified files in a conveniently readable format.
A few years ago, I discovered the p* commands or get information about processes: ptree, pgrep, pkill, and pfiles. Of course, the mother of them all is ps, but you need to pipe the output into less, grep and/or awk to make sense of the output under heavy load. top (and variants) help too.
I always set my default prompt to "username@hostname:/current/path/name/in/full> "
PS1='\u@\h:\w> '
export PS1
Saves lots of confusion when you're dealing with lots of different machines.
cd -
It's the command-line equivalent of the back button (takes you to the previous directory you were in).
find -iregex '.*\.py$\|.*\.xml$' | xargs egrep -niH 'a.search.pattern' | vi -R -
Searches a pattern in all Python files and all XML files and pipes the result in a readonly Vim session.
http://www.commandlinefu.com is also a great site.
I learned quite useful things there like:
sudo !!
or
mount | column -t
Eliminate duplicate lines from a file
#sort -u filename > filename.new
List all lines that do not match a condition
#grep -v ajsk filename
These are not necessarily Bash specific (but hey neither is ls -thor :) )
Some other useful cmds:
prtdiag, psrinfo, prtconf - more info here and here (posts on my blog).
Here's a couple of configuration tweaks:
~/.inputrc:
"\C-[[A": history-search-backward
"\C-[[B": history-search-forward
This works the same as ^R but using the arrow keys instead. This means I can type (e.g.) cd /media/ then hit up-arrow to go to the last thing I cd'd to inside the /media/ folder.
(I use Gnome Terminal, you may need to change the escape codes for other terminal emulators.)
Bash completion is also incredibly useful, but it's a far more subtle addition. In ~/.bashrc:
if [ -f /etc/bash_completion ]; then
. /etc/bash_completion
fi
This will enable per-program tab-completion (e.g. attempting tab completion when the command line starts with evince will only show files that evince can open, and it will also tab-complete command line options).
Works nicely with this also in ~/.inputrc:
set completion-ignore-case on
set show-all-if-ambiguous on
set show-all-if-unmodified on
Custom Tab Completion (compgen and complete bash builtins)
Tab Completion is nice, but being able to apply it to more than just filenames is great. I have used it to create custom functions to expand arguments to commands I use all the time. For example, lets say you often need to add the FQDN as an argument to a command (e.g. ping blah.really.long.domain.name.foo.com). You can use compgen and complete to create a bash function that reads your /etc/hosts file for results so all you have to type then is:
ping blah.<tab>
and it will display all your current match options.
So basically anything that can return a word list can be used as a function.
extended globbing:
rm !(foo|bar)
expands like * without foo or bar:
$ ls
foo
bar
foobar
FOO
$ echo !(foo|bar)
foobar FOO
I've always been partial to:
ctrl-E # move cursor to end of line
ctrl-A # move cursor to beginning of line
I also use shopt -s cdable_vars, then you can create bash variables to common directories. So, for my company's source tree, I create a bunch of variables like:
export Dcentmain="/var/localdata/p4ws/centaur/main/apps/core"
then I can change to that directory by cd Dcentmain.
I prefer reading man pages in vi, so I have the following in my .profile or .bashrc file
man () {
sought=$*
/usr/bin/man $sought | col -b | vim -R -c "set nonumber" -c "set syntax=man" -
}
bash can redirect to and from TCP/IP sockets. /dev/tcp/ and /dev/udp.
Some people think it's a security issue, but that's what OS level security like Solaris X's jail is for.
As Will Robertson notes, change prompt to do stuff... print the command # for !nn Set the Xterm terminal name. If it's an old Xterm that doesn't sniff traffic to set it's title.
The easiest keystrokes for me for "last argument of the last command" is !$
echo what the heck?
what the heck?
echo !$
heck?
Using alias can be time-saving
alias myDir = "cd /this/is/a/long/directory; pwd"
pushd and popd almost always come in handy
./mylittlealgorithm < input.txt > output.txt
When running a command with lots of output (like a big "make") I want to not only save the output, but also see it:
make install 2>&1 | tee E.make
Using 'set -o vi' from the command line, or better, in .bashrc, puts you in vi editing mode on the command line. You start in 'insert' mode so you can type and backspace as normal, but if you make a 'large' mistake you can hit the esc key and then use 'b' and 'f' to move around as you do in vi. cw to change a word. Particularly useful after you've brought up a history command that you want to change.
extended globbing:
rm !(foo|bar)
expands like * without foo or bar:
$ ls
foo
bar
foobar
FOO
$ echo !(foo|bar)
foobar FOO
CTRL+D quits the shell.
Top 10 commands again (like ctcherry's post, only shorter):
history | awk '{ print $2 }' | sort | uniq -c |sort -rn | head
As an extension to CTRL-r to search backwards, you can auto-complete your current input with your history if you bind 'history-search-backward'. I typically bind it to the same key that it is in tcsh: ESC-p. You can do this by putting the following line in your .inputrc file:
"\M-p": history-search-backward
E.g. if you have previously executed 'make some_really_painfully_long_target' you can type:
> make <ESC p>
and it will give you
> make some_really_painfully_long_target
I'm a big fan of Bash job control, mainly the use of Control-Z and fg, especially if I'm doing development in a terminal. If I've got emacs open and need to compile, deploy, etc. I just Control-Z to suspend emacs, do what I need, and fg to bring it back. This keeps all of the emacs buffers intact and makes things much easier than re-launching whatever I'm doing.
alias ..='cd ..'
So when navigating back up a directory just use ..<Enter>
^R reverse search. Hit ^R, type a fragment of a previous command you want to match, and hit ^R until you find the one you want. Then I don't have to remember recently used commands that are still in my history. Not exclusively bash, but also: ^E for end of line, ^A for beginning of line, ^U and ^K to delete before and after the cursor, respectively.
As an extension to CTRL-r to search backwards, you can auto-complete your current input with your history if you bind 'history-search-backward'. I typically bind it to the same key that it is in tcsh: ESC-p. You can do this by putting the following line in your .inputrc file:
"\M-p": history-search-backward
E.g. if you have previously executed 'make some_really_painfully_long_target' you can type:
> make <ESC p>
and it will give you
> make some_really_painfully_long_target
I like to set a prompt which shows the current directory in the window title of an xterm. It also shows the time and current directory. In addition, if bash wants to report that a background job has finished, it is reported in a different colour using ANSI escape sequences. I use a black-on-light console so my colours may not be right for you if you favour light-on-black.
PROMPT_COMMAND='echo -e "\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~}\007\033[1;31m${PWD/#$HOME/~}\033[1;34m"'
PS1='\[\e[1;31m\]\t \$ \[\e[0m\]'
Make sure you understand how to use \[ and \] correctly in your PS1 string so that bash knows how long your prompt-string actually renders on screen. This is so it can redraw your command-line correctly when you move beyond a single line command.
I use the following a lot:
The :p modifier to print a history result. E.g.
!!:p
Will print the last command so you can check that it's correct before running it again. Just enter !! to execute it.
In a similar vein:
!?foo?:p
Will search your history for the most recent command that contained the string 'foo' and print it.
If you don't need to print,
!?foo
does the search and executes it straight away.
du -a | sort -n | tail -99
to find the big files (or directories of files) to clean up to free up disk space.
As an extension to CTRL-r to search backwards, you can auto-complete your current input with your history if you bind 'history-search-backward'. I typically bind it to the same key that it is in tcsh: ESC-p. You can do this by putting the following line in your .inputrc file:
"\M-p": history-search-backward
E.g. if you have previously executed 'make some_really_painfully_long_target' you can type:
> make <ESC p>
and it will give you
> make some_really_painfully_long_target
I like to set a prompt which shows the current directory in the window title of an xterm. It also shows the time and current directory. In addition, if bash wants to report that a background job has finished, it is reported in a different colour using ANSI escape sequences. I use a black-on-light console so my colours may not be right for you if you favour light-on-black.
PROMPT_COMMAND='echo -e "\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~}\007\033[1;31m${PWD/#$HOME/~}\033[1;34m"'
PS1='\[\e[1;31m\]\t \$ \[\e[0m\]'
Make sure you understand how to use \[ and \] correctly in your PS1 string so that bash knows how long your prompt-string actually renders on screen. This is so it can redraw your command-line correctly when you move beyond a single line command.
sudo !!
Runs the last command with administrator privileges.
rename
Example:
$ ls
this_has_text_to_find_1.txt
this_has_text_to_find_2.txt
this_has_text_to_find_3.txt
this_has_text_to_find_4.txt
$ rename 's/text_to_find/been_renamed/' *.txt
$ ls
this_has_been_renamed_1.txt
this_has_been_renamed_2.txt
this_has_been_renamed_3.txt
this_has_been_renamed_4.txt
So useful
Insert preceding lines final parameter
ALT-. the most useful key combination ever, try it and see, for some reason no one knows about this one.
Press it again and again to select older last parameters.
Great when you want to do something else to something you used just a moment ago.
I have a really stupid, but extremely helpful one when navigating deep tree structures. Put this in .bashrc (or similar):
alias cd6="cd ../../../../../.."
alias cd5="cd ../../../../.."
alias cd4="cd ../../../.."
alias cd3="cd ../../.."
alias cd2="cd ../.."
Good for making an exact recursive copy/backup of a directory including symlinks (rather than following them or ignoring them like cp):
$ mkdir new_dir
$ cd old_dir
$ tar cf - . | ( cd ../old_dir; tar xf - )
I like a splash of colour in my prompts:
export PS1="\[\033[07;31m\] \h \[\033[47;30m\] \W \[\033[00;31m\] \$ \[\e[m\]"
I'm afraid I don't have a screenshot for what that looks like, but it's supposed to be something like (all on one line):
[RED BACK WHITE TEXT] Computer name
[BLACK BACK WHITE TEXT] Working Directory
[WHITE BACK RED TEXT] $
Customise as per what you like to see :)
find -iregex '.*\.py$\|.*\.xml$' | xargs egrep -niH 'a.search.pattern' | vi -R -
Searches a pattern in all Python files and all XML files and pipes the result in a readonly Vim session.
One preferred way of navigating when I'm using multiple directories in widely separate places in a tree hierarchy is to use acf_func.sh (listed below). Once defined, you can do
cd --
to see a list of recent directories, with a numerical menu
cd -2
to go to the second-most recent directory.
Very easy to use, very handy.
Here's the code:
# do ". acd_func.sh"
# acd_func 1.0.5, 10-nov-2004
# petar marinov, http:/geocities.com/h2428, this is public domain
cd_func ()
{
local x2 the_new_dir adir index
local -i cnt
if [[ $1 == "--" ]]; then
dirs -v
return 0
fi
the_new_dir=$1
[[ -z $1 ]] && the_new_dir=$HOME
if [[ ${the_new_dir:0:1} == '-' ]]; then
#
# Extract dir N from dirs
index=${the_new_dir:1}
[[ -z $index ]] && index=1
adir=$(dirs +$index)
[[ -z $adir ]] && return 1
the_new_dir=$adir
fi
#
# '~' has to be substituted by ${HOME}
[[ ${the_new_dir:0:1} == '~' ]] && the_new_dir="${HOME}${the_new_dir:1}"
#
# Now change to the new dir and add to the top of the stack
pushd "${the_new_dir}" > /dev/null
[[ $? -ne 0 ]] && return 1
the_new_dir=$(pwd)
#
# Trim down everything beyond 11th entry
popd -n +11 2>/dev/null 1>/dev/null
#
# Remove any other occurence of this dir, skipping the top of the stack
for ((cnt=1; cnt <= 10; cnt++)); do
x2=$(dirs +${cnt} 2>/dev/null)
[[ $? -ne 0 ]] && return 0
[[ ${x2:0:1} == '~' ]] && x2="${HOME}${x2:1}"
if [[ "${x2}" == "${the_new_dir}" ]]; then
popd -n +$cnt 2>/dev/null 1>/dev/null
cnt=cnt-1
fi
done
return 0
}
alias cd=cd_func
if [[ $BASH_VERSION > "2.05a" ]]; then
# ctrl+w shows the menu
bind -x "\"\C-w\":cd_func -- ;"
fi
Well, this may be a bit off topic, but if you are an Emacs user, I would say "emacs" is the most powerful trick... before you downvote this, try out "M-x shell" within an emacs instance... you get a shell inside emacs, and have all the power of emacs along with the power of a shell (there are some limitations, such as opening another emacs within it, but in most cases it is a lot more powerful than a vanilla bash prompt).
String multiple commands together using the && command:
./run.sh && tail -f log.txt
or
kill -9 1111 && ./start.sh
# Batch extension renamer (usage: renamer txt mkd)
renamer() {
local fn
for fn in *."$1"; do
mv "$fn" "${fn%.*}"."$2"
done
}
http://www.commandlinefu.com is also a great site.
I learned quite useful things there like:
sudo !!
or
mount | column -t
ESC.
Inserts the last arguments from your last bash command. It comes in handy more than you think.
cp file /to/some/long/path
cd ESC.
Another favorite:
!!
Repeats your last command. Most useful in the form:
sudo !!
Well, this may be a bit off topic, but if you are an Emacs user, I would say "emacs" is the most powerful trick... before you downvote this, try out "M-x shell" within an emacs instance... you get a shell inside emacs, and have all the power of emacs along with the power of a shell (there are some limitations, such as opening another emacs within it, but in most cases it is a lot more powerful than a vanilla bash prompt).
As a quick calculator, say to compute a percentage:
$ date
Thu Sep 18 12:55:33 EDT 2008
$ answers=60
$ curl "http://stackoverflow.com/questions/68372/what-are-some-of-your-favorite-command-line-tricks-using-bash" > tmp.html
$ words=`awk '/class="post-text"/ {s = s $0} \
> /<\/div>/ { gsub("<[^>]*>", "", s); print s; s = ""} \
> length(s) > 0 {s = s $0}' tmp.html \
> | awk '{n = n + NF} END {print n}'`
$ answers=`awk '/([0-9]+) Answers/ {sub("<h2>", "", $1); print $1}' tmp.html`
and finally:
$ echo $words words, $answers answers, $((words / $answers)) words per answer
4126 words, 60 answers, 68 words per answer
$
Not that division is truncated, not rounded. But often that's good enough for a quick calculation.
$ touch {1,2}.txt
$ ls [12].txt
1.txt 2.txt
$ rm !:1
rm [12].txt
$ history | tail -10
...
10007 touch {1,2}.txt
...
$ !10007
touch {1,2}.txt
$ for f in *.txt; do mv $f ${f/txt/doc}; done
Programmable Completion:
Nothing fancy. I always disable it when I'm using Knoppix because it gets in the way too often. Just some basic ones:
shopt -s progcomp
complete -A stopped -P '%' bg
complete -A job -P '%' fg jobs disown wait
complete -A variable readonly export
complete -A variable -A function unset
complete -A setopt set
complete -A shopt shopt
complete -A helptopic help
complete -A alias alias unalias
complete -A binding bind
complete -A command type which \
killall pidof
complete -A builtin builtin
complete -A disabled enable
I always set my default prompt to "username@hostname:/current/path/name/in/full> "
PS1='\u@\h:\w> '
export PS1
Saves lots of confusion when you're dealing with lots of different machines.
Quick Text
I use these sequences of text all too often, so I put shortcuts to them in by .inputrc:
# redirection short cuts
"\ew": "2>&1"
"\eq": "&>/dev/null &"
"\e\C-q": "2>/dev/null"
"\eg": "&>~/.garbage.out &"
"\e\C-g": "2>~/.garbage.out"
$if term=xterm
"\M-w": "2>&1"
"\M-q": "&>/dev/null &"
"\M-\C-q": "2>/dev/null"
"\M-g": "&>~/.garbage.out &"
"\M-\C-g": "2>~/.garbage.out"
$endif
I have got a secret weapon : shell-fu.
There are thousand of smart tips, cool tricks and efficient recipes that most of the time fit on a single line.
One that I love (but I cheat a bit since I use the fact that Python is installed on most Unix system now) :
alias webshare='python -m SimpleHTTPServer'
Now everytime you type "webshare", the current directory will be available through the port 8000. Really nice when you want to share files with friends on a local network without usb key or remote dir. Streaming video and music will work too.
And of course the classic fork bomb that is completely useless but still a lot of fun :
$ :(){ :|:& };:
Don't try that in a production server...
I prefer reading man pages in vi, so I have the following in my .profile or .bashrc file
man () {
sought=$*
/usr/bin/man $sought | col -b | vim -R -c "set nonumber" -c "set syntax=man" -
}
Since I always need the for i in $(ls) statement I made a shortcut:
fea(){
if test -z ${2:0:1}; then action=echo; else action=$2; fi
for i in $(ls $1);
do $action $i ;
done;
}
Another one is:
echo ${!B*}
It will print a list of all defined variables that start with 'B'.
Another favorite:
!!
Repeats your last command. Most useful in the form:
sudo !!
SSH tunnel:
ssh -fNR 1234:localhost:22 [email protected]
cd -
It's the command-line equivalent of the back button (takes you to the previous directory you were in).
Shell-fu is a place for storing, moderating and propagating command line tips and tricks. A bit like StackOverflow, but solely for shell. You'll find plenty of answers to this question there.
alias -- ddt='ls -trFld'
dt () { ddt --color "$@" | tail -n 30; }
Gives you the most recent files in the current directory. I use it all the time...
Duplicate file finder
This will run checksums recursively from the current directory, and give back the filenames of all identical checksum results:
find ./ -type f -print0 | xargs -0 -n1 md5sum | sort -k 1,32 | uniq -w 32 -d --all-repeated=separate | sed -e 's/^[0-9a-f]*\ *//;'
You can, of course, change the path around.
Maybe put it into a function or alias, and pass in the target path as a parameter.
You should be able to paste the following into a bash terminal window.
Display ANSI colour palette:
e="\033["
for f in 0 7 `seq 6`; do
no="" bo=""
for b in n 7 0 `seq 6`; do
co="3$f"; p=" "
[ $b = n ] || { co="$co;4$b";p=""; }
no="${no}${e}${co}m ${p}${co} ${e}0m"
bo="${bo}${e}1;${co}m ${p}1;${co} ${e}0m"
done
echo -e "$no\n$bo"
done
256 colour demo:
yes "$(seq 232 255;seq 254 -1 233)" |
while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done
alias ..='cd ..'
So when navigating back up a directory just use ..<Enter>
When navigating between two separate directories and copying files back and forth, I do this:
cd /some/where/long
src=`pwd`
cd /other/where/long
dest=`pwd`
cp $src/foo $dest
command completion will work by expanding the variable, so you can use tab completion to specify a file you're working with.
Similar to many above, my current favorite is the keystroke [alt]. (Alt and "." keys together) this is the same as $! (Inserts the last argument from the previous command) except that it's immediate and for me easier to type. (Just can't be used in scripts)
eg:
mkdir -p /tmp/test/blah/oops/something
cd [alt].
Ctrl + L will usually clear the screen. Works from the Bash prompt (obviously) and in GDB, and a lot of other prompts.
I want to mention how we can redirect top command output to file using its batch mode (-b)
$ top -b -n 1 > top.out.$(date +%s)
By default, top is invoked using interactive mode in which top runs indefinitely and accepts keypress to redefine how top works.
A post I wrote can be found here
Some useful mencoder commands I found out about when looking for some audio and video editing tools:
from .xxx to .avi
mencoder movie.wmv -o movie.avi -ovc lavc -oac lavc
Dump sound from a video:
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile fileout.mp3 filein.avi
Someone else recommended "M-x shell RET" in Emacs. I think "M-x eshell RET" is even better.
./mylittlealgorithm < input.txt > output.txt
CTRL+D quits the shell.
Renaming/moving files with suffixes quickly:
cp /home/foo/realllylongname.cpp{,-old}
This expands to:
cp /home/foo/realllylongname.cpp /home/foo/realllylongname.cpp-old
My favorite command is "ls -thor"
It summons the power of the gods to list the most recently modified files in a conveniently readable format.
As a quick calculator, say to compute a percentage:
$ date
Thu Sep 18 12:55:33 EDT 2008
$ answers=60
$ curl "http://stackoverflow.com/questions/68372/what-are-some-of-your-favorite-command-line-tricks-using-bash" > tmp.html
$ words=`awk '/class="post-text"/ {s = s $0} \
> /<\/div>/ { gsub("<[^>]*>", "", s); print s; s = ""} \
> length(s) > 0 {s = s $0}' tmp.html \
> | awk '{n = n + NF} END {print n}'`
$ answers=`awk '/([0-9]+) Answers/ {sub("<h2>", "", $1); print $1}' tmp.html`
and finally:
$ echo $words words, $answers answers, $((words / $answers)) words per answer
4126 words, 60 answers, 68 words per answer
$
Not that division is truncated, not rounded. But often that's good enough for a quick calculation.
!<first few characters of the command> will execute the last command which matches.
Example:
!b will run "build whatever -O -p -t -i -on"
!. will run ./a.out
It works best with long and repetitive commands, like compile, build, execute, etc. It saved me sooo much time when coding and testing.
A simple thing to do when you realize you just typed the wrong line is hit Ctrl+C; if you want to keep the line, but need to execute something else first, begin a new line with a back slash - \, then Ctrl+C. The line will remain in your history.
The easiest keystrokes for me for "last argument of the last command" is !$
echo what the heck?
what the heck?
echo !$
heck?
I've always liked this one. Add this to your /etc/inputrc or ~/.inputrc
"\e[A":history-search-backward "\e[B":history-search-forward
When you type ls <up-arrow> it will be replaced with the last command starting with "ls " or whatever else you put in.
$ touch {1,2}.txt
$ ls [12].txt
1.txt 2.txt
$ rm !:1
rm [12].txt
$ history | tail -10
...
10007 touch {1,2}.txt
...
$ !10007
touch {1,2}.txt
$ for f in *.txt; do mv $f ${f/txt/doc}; done
Well, this may be a bit off topic, but if you are an Emacs user, I would say "emacs" is the most powerful trick... before you downvote this, try out "M-x shell" within an emacs instance... you get a shell inside emacs, and have all the power of emacs along with the power of a shell (there are some limitations, such as opening another emacs within it, but in most cases it is a lot more powerful than a vanilla bash prompt).
You should be able to paste the following into a bash terminal window.
Display ANSI colour palette:
e="\033["
for f in 0 7 `seq 6`; do
no="" bo=""
for b in n 7 0 `seq 6`; do
co="3$f"; p=" "
[ $b = n ] || { co="$co;4$b";p=""; }
no="${no}${e}${co}m ${p}${co} ${e}0m"
bo="${bo}${e}1;${co}m ${p}1;${co} ${e}0m"
done
echo -e "$no\n$bo"
done
256 colour demo:
yes "$(seq 232 255;seq 254 -1 233)" |
while read i; do printf "\x1b[48;5;${i}m\n"; sleep .01; done
Eliminate duplicate lines from a file
#sort -u filename > filename.new
List all lines that do not match a condition
#grep -v ajsk filename
These are not necessarily Bash specific (but hey neither is ls -thor :) )
Some other useful cmds:
prtdiag, psrinfo, prtconf - more info here and here (posts on my blog).
Not my favorite, by very helpful if you're trying any of the other answers using copy and paste:
function $
{
"$@"
}
Now you can paste examples that include a $ prompt at the start of each line.
And this one is key for me actually:
set -o vi
/Allan
du -a | sort -n | tail -99
to find the big files (or directories of files) to clean up to free up disk space.
Similar to many above, my current favorite is the keystroke [alt]. (Alt and "." keys together) this is the same as $! (Inserts the last argument from the previous command) except that it's immediate and for me easier to type. (Just can't be used in scripts)
eg:
mkdir -p /tmp/test/blah/oops/something
cd [alt].
alias mycommand = 'verylongcommand -with -a -lot -of -parameters'
alias grep='grep --color'
find more than one word with grep :
netstat -c |grep 'msn\|skype\|icq'
I have plenty of directories which I want to access quickly, CDPATH variable is solution
that speed up my work-flow enormously:
export CDPATH=.:/home/gadolin/sth:/home/gadolin/dir1/importantDir
now with cd I can jump to any of sub directories of /home/gadolin/sth or /home/gadolin/dir1/importantDir without providing the full path. And also <tab> works here just like I would be there!
So if there are directories /home/gadolin/sth/1 /home/gadolin/sth/2, I type cd 1 wherever, and I am there.
One preferred way of navigating when I'm using multiple directories in widely separate places in a tree hierarchy is to use acf_func.sh (listed below). Once defined, you can do
cd --
to see a list of recent directories, with a numerical menu
cd -2
to go to the second-most recent directory.
Very easy to use, very handy.
Here's the code:
# do ". acd_func.sh"
# acd_func 1.0.5, 10-nov-2004
# petar marinov, http:/geocities.com/h2428, this is public domain
cd_func ()
{
local x2 the_new_dir adir index
local -i cnt
if [[ $1 == "--" ]]; then
dirs -v
return 0
fi
the_new_dir=$1
[[ -z $1 ]] && the_new_dir=$HOME
if [[ ${the_new_dir:0:1} == '-' ]]; then
#
# Extract dir N from dirs
index=${the_new_dir:1}
[[ -z $index ]] && index=1
adir=$(dirs +$index)
[[ -z $adir ]] && return 1
the_new_dir=$adir
fi
#
# '~' has to be substituted by ${HOME}
[[ ${the_new_dir:0:1} == '~' ]] && the_new_dir="${HOME}${the_new_dir:1}"
#
# Now change to the new dir and add to the top of the stack
pushd "${the_new_dir}" > /dev/null
[[ $? -ne 0 ]] && return 1
the_new_dir=$(pwd)
#
# Trim down everything beyond 11th entry
popd -n +11 2>/dev/null 1>/dev/null
#
# Remove any other occurence of this dir, skipping the top of the stack
for ((cnt=1; cnt <= 10; cnt++)); do
x2=$(dirs +${cnt} 2>/dev/null)
[[ $? -ne 0 ]] && return 0
[[ ${x2:0:1} == '~' ]] && x2="${HOME}${x2:1}"
if [[ "${x2}" == "${the_new_dir}" ]]; then
popd -n +$cnt 2>/dev/null 1>/dev/null
cnt=cnt-1
fi
done
return 0
}
alias cd=cd_func
if [[ $BASH_VERSION > "2.05a" ]]; then
# ctrl+w shows the menu
bind -x "\"\C-w\":cd_func -- ;"
fi
I want to mention how we can redirect top command output to file using its batch mode (-b)
$ top -b -n 1 > top.out.$(date +%s)
By default, top is invoked using interactive mode in which top runs indefinitely and accepts keypress to redefine how top works.
A post I wrote can be found here
I'm new to programming on a mac, and I miss being able to launch gui programs from bash...so I have to create functions like this:
function macvim
{
/Applications/MacVim.app/Contents/MacOS/Vim "$@" -gp &
}
Some useful mencoder commands I found out about when looking for some audio and video editing tools:
from .xxx to .avi
mencoder movie.wmv -o movie.avi -ovc lavc -oac lavc
Dump sound from a video:
mplayer -ao pcm -vo null -vc dummy -dumpaudio -dumpfile fileout.mp3 filein.avi
I like to construct commands with echo and pipe them to the shell:
$ find dir -name \*~ | xargs echo rm
...
$ find dir -name \*~ | xargs echo rm | ksh -s
Why? Because it allows me to look at what's going to be done before I do it. That way if I have a horrible error (like removing my home directory), I can catch it before it happens. Obviously, this is most important for destructive or irrevocable actions.
I like to set a prompt which shows the current directory in the window title of an xterm. It also shows the time and current directory. In addition, if bash wants to report that a background job has finished, it is reported in a different colour using ANSI escape sequences. I use a black-on-light console so my colours may not be right for you if you favour light-on-black.
PROMPT_COMMAND='echo -e "\033]0;${USER}@${HOSTNAME%%.*}:${PWD/#$HOME/~}\007\033[1;31m${PWD/#$HOME/~}\033[1;34m"'
PS1='\[\e[1;31m\]\t \$ \[\e[0m\]'
Make sure you understand how to use \[ and \] correctly in your PS1 string so that bash knows how long your prompt-string actually renders on screen. This is so it can redraw your command-line correctly when you move beyond a single line command.
How to list only subdirectories in the current one ?
ls -d */
It's a simple trick, but you wouldn't know how much time I needed to find that one !
The FIGNORE environment variable is nice when you want TAB completion to ignore files or folders with certain suffixes, e.g.:
export FIGNORE="CVS:.svn:~"
Use the IFS environment variable when you want to define an item separator other than space, e.g.:
export IFS="
"
This will make you able to loop through files and folders with spaces in them without performing any magic, like this:
$ touch "with spaces" withoutspaces
$ for i in `ls *`; do echo $i; done
with
spaces
withoutspaces
$ IFS="
"
$ for i in `ls *`; do echo $i; done
with spaces
withoutspaces
pushd and popd almost always come in handy
Good for making an exact recursive copy/backup of a directory including symlinks (rather than following them or ignoring them like cp):
$ mkdir new_dir
$ cd old_dir
$ tar cf - . | ( cd ../old_dir; tar xf - )
This prevents less (less is more) from clearing the screen at the end of a file:
export LESS="-X"
When downloading a large file I quite often do:
while ls -la <filename>; do sleep 5; done
And then just ctrl+c when I'm done (or if ls returns non-zero). It's similar to the watch program but it uses the shell instead, so it works on platforms without watch.
Another useful tool is netcat, or nc. If you do:
nc -l -p 9100 > printjob.prn
Then you can set up a printer on another computer but instead use the IP address of the computer running netcat. When the print job is sent, it is received by the computer running netcat and dumped into printjob.prn.
Not my favorite, by very helpful if you're trying any of the other answers using copy and paste:
function $
{
"$@"
}
Now you can paste examples that include a $ prompt at the start of each line.
Two of my favorites are:
1) Make tab-completion case insensitive (e.g. "cd /home/User " converts your command line to: "cd /home/user" if the latter exists and the former doesn't. You can turn it on with "set completion-ignore-case on" at the prompt, or add it permanently by adding "set completion-ignore-case on" to your .inputrc file.
2) The built-in 'type' command is like "which" but aware of aliases also. For example
$ type cdhome
cdhome is aliased to 'cd ~'
$ type bash
bash is /bin/bash
I'm new to programming on a mac, and I miss being able to launch gui programs from bash...so I have to create functions like this:
function macvim
{
/Applications/MacVim.app/Contents/MacOS/Vim "$@" -gp &
}
As an extension to CTRL-r to search backwards, you can auto-complete your current input with your history if you bind 'history-search-backward'. I typically bind it to the same key that it is in tcsh: ESC-p. You can do this by putting the following line in your .inputrc file:
"\M-p": history-search-backward
E.g. if you have previously executed 'make some_really_painfully_long_target' you can type:
> make <ESC p>
and it will give you
> make some_really_painfully_long_target
ctrl-u delete all written stuff
Source: Stackoverflow.com