I am using tcpdump to get HTTP data by executing the below command:
sudo tcpdump -A -s 1492 dst port 80
The result of above command:
GET /modules/mod_news_pro_gk1/cache/stories.ilbalad.ajayeb.strange-tractor.jpg. I need a more clear result, for example, readable request > response header > response body etc. How can I filter my results?
There are tcpdump filters for HTTP GET & HTTP POST (or for both plus message body):
Run man tcpdump | less -Ip examples to see some examples
Here’s a tcpdump filter for HTTP GET (GET = 0x47, 0x45, 0x54, 0x20):
sudo tcpdump -s 0 -A 'tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x47455420'
Here’s a tcpdump filter for HTTP POST (POST = 0x50, 0x4f, 0x53, 0x54):
sudo tcpdump -s 0 -A 'tcp dst port 80 and (tcp[((tcp[12:1] & 0xf0) >> 2):4] = 0x504f5354)'
Monitor HTTP traffic including request and response headers and message body (source):
tcpdump -A -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
tcpdump -X -s 0 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
For more information on the bit-twiddling in the TCP header see: String-Matching Capture Filter Generator (link to Sake Blok's explanation).
I would recommend using Wireshark, which has a "Follow TCP Stream" option that makes it very easy to see the full requests and responses for a particular TCP connection. If you would prefer to use the command line, you can try tcpflow, a tool dedicated to capturing and reconstructing the contents of TCP streams.
Other options would be using an HTTP debugging proxy, like Charles or Fiddler as EricLaw suggests. These have the advantage of having specific support for HTTP to make it easier to deal with various sorts of encodings, and other features like saving requests to replay them or editing requests.
You could also use a tool like Firebug (Firefox), Web Inspector (Safari, Chrome, and other WebKit-based browsers), or Opera Dragonfly, all of which provide some ability to view the request and response headers and bodies (though most of them don't allow you to see the exact byte stream, but instead how the browsers parsed the requests).
And finally, you can always construct requests by hand, using something like telnet, netcat, or socat to connect to port 80 and type the request in manually, or a tool like htty to help easily construct a request and inspect the response.
Here is another choice: Chaosreader
So I need to debug an application which posts xml to a 3rd party application. I found a brilliant little perl script which does all the hard work – you just chuck it a tcpdump output file, and it does all the manipulation and outputs everything you need...
The script is called chaosreader0.94. See http://www.darknet.org.uk/2007/11/chaosreader-trace-tcpudp-sessions-from-tcpdump/
It worked like a treat, I did the following:
tcpdump host www.blah.com -s 9000 -w outputfile; perl chaosreader0.94 outputfile
Source: Stackoverflow.com