While this may be doable with Wireshark, it is orders of magnitude easier with Bro.
Extracting URIs
Simply run it with your trace file:
bro -r <trace>
This invocation generates a bunch of log files in the current directory. The one you are interested in is http.log
. You can filter the output to obtain only the GET requests:
bro-cut id.orig_h id.resp_h method host uri < http.log | awk '$3 == "GET"'
Example output:
192.168.1.104 212.96.161.238 GET update.avg.com /softw/90/update/avg9infowin.ctf
192.168.1.104 77.67.44.206 GET backup.avg.cz /softw/90/update/u7avi1777u1705ff.bin
192.168.1.104 198.189.255.75 GET aa.avg.com /softw/90/update/u7iavi2511u2510ff.bin
192.168.1.104 77.67.44.206 GET backup.avg.cz /softw/90/update/x8xplsb2_118c8.bin
As you can see, the last two columns make up the full URL. To remove the space in-between, you could use awk to concatenate the last two fields.
Extracting Files
Note: the upcoming Bro 2.1 release will have major improvements for file extractions. Until then, you can extract all files from a HTTP stream by specifying the MIME type of the files to store:
bro -r <trace> 'HTTP::extract_file_type = /video/avi/'
Bro sniffs the MIME type of a HTTP body and if it matches the regular expression /video/avi/
, it creates a file with the prefix http-item
. You can change the prefix name by redefining the HTTP::extraction_prefix
variable.
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…