Personally I don't find the standard diagramming technique very helpful - the arrows always seem to point the wrong way for me. (They generally point towards the "parent" of each commit, which ends up being backwards in time, which is weird).
To explain it in words:
For reasons I don't understand, GUI tools for Git have never made much of an effort to present merge histories more cleanly, abstracting out the individual merges. So if you want a "clean history", you need to use rebase.
I seem to recall having read blog posts from programmers who only use rebase and others that never use rebase.
I'll try explaining this with a just-words example. Let's say other people on your project are working on the user interface, and you're writing documentation. Without rebase, your history might look something like:
Write tutorial
Merge remote-tracking branch 'origin/master' into fixdocs
Bigger buttons
Drop down list
Extend README
Merge remote-tracking branch 'origin/master' into fixdocs
Make window larger
Fix a mistake in howto.md
That is, merges and UI commits in the middle of your documentation commits.
If you rebased your code onto master instead of merging it, it would look like this:
Write tutorial
Extend README
Fix a mistake in howto.md
Bigger buttons
Drop down list
Make window larger
All of your commits are at the top (newest), followed by the rest of the master
branch.
(Disclaimer: I'm the author of the "10 things I hate about Git" post referred to in another answer)