Tackling the Web with PeepingTom

In episode 291 of Security Weekly, Paul did a tech segment on solving the problem of managing large quantities of web servers discovered by Nessus or Nmap scans. The traditional approach is to open up each of the servers in a browser to see what’s there, then make a decision on which servers to attack, if any. This is highly inefficient, and in the case of a time constrained penetration test, not practical. So Paul, with a combination of tools and bash foo, presented a way to practically manage this data. It basically came down to using perl libraries to screenshot web pages, collect header data, and place them in an html formatted file for analysis. Awesome right? Well I was on a recent penetration test where I was confronted with 822 discovered web servers in the target’s IP range and was anxious to put this new capability to use… right up to the point where the web scraping perl script failed miserably. The main script behind what Paul was trying to accomplish, webscour.pl, had issues dealing with invalid certificates and certain web responses (404, 302, 500, etc.), and the output was unimpressive to say the least. It just wasn’t functional enough to be useful in my particular case, especially since, as with most internal networks, the discovered results were littered with invalid SSL certificates. Therefore, I set out to write an improved version of the script in python, and PeepingTom was born.
While other tools such as Web2PNG, webkit2png, and html2png exist, these tools are designed to take screenshots of one website at a time and do not handle invalid SSL certificates or present results in a meaningful way. What PeepingTom does, is not only take a static screenshot of multiple websites, but it effectively handles invalid SSL certificates and errors, follows redirects, reports redirects, and dynamically builds a HTML report which presents users with a screenshot of the webpage, the header information for the final destination of each page, and whether or not the headers and image were the result of a redirect. Also, if the something fails, i.e. network timeout, etc., the generated report has a section for failed requests with the reasons for failure and links to the failed URL for manual investigation.

PeepingTom uses several different methods for capturing the web screenshots depending on your platform and preference. PeepingTom has the ability to use PyQt4 libraries or the Phantomjs framework with the mode being selected as a script argument at runtime. Personally, I prefer the standalone capability that Phantonjs provides, and Phantomjs seems to be a bit faster than PyQt4 as well. PeepingTom can be found here. Please submit bugs, issues, feature requests, etc. Enjoy!
Join me for SEC542: Web App Penetration Testing and Ethical Hacking at SANS Monterey 2013!
Monterey, CA | Fri Mar 22 – Wed Mar 27, 2013