This review is part of the September 2021 assessment of network vulnerability scanners. If you haven’t read the category overview, you might want to check it out. It explains the category’s basics, use cases and the general value proposition. Our testing methodology explains both how we interact with vendors and how we tested these products.
Vulnerability scanners are the sensors of the vulnerability management process – they reach out and touch systems to determine if they are vulnerable to exploits and other threats. While there are vulnerability scanners for containers, infrastructure-as-code tooling, cloud management consoles, and web applications, this group review will focus on network vulnerability scanners.
Company background
Founded over 20 years ago, Rapid7 was an early commercial vulnerability scanner and the original Nexpose product lives on today as part of the InsightVM platform. Rapid7 acquired the popular offensive security tool Metasploit in 2009 and continued expanding its product portfolio throughout the 2010s through additional acquisitions and internal development. The company went public in 2015.
Today, in addition to vulnerability management, Rapid7 offers products that fit into the UEBA, SIEM, SOAR, WAF, and cloud security categories. The company also provides traditional consulting services along with managed services and training.
Product summary
Aside from some friction in getting InsightVM deployed and running, the product is a solid vulnerability management choice. We feel InsightVM should add a unified console option, so that scans can be managed from the SaaS interface. Retaining the local/offline option makes sense, as many organizations still aren’t comfortable putting their vulnerability data in the cloud, or need to run a vulnerability scanner in an environment without Internet access.
We were charmed by Rapid7’s broad availability of workflow orchestrations, automating away some of the VM tasks most likely to gather dust in organizations. Asset groups, sites, and role-based access control encourage deploying InsightVM as an IT-wide tool – not just something for security to interact with.
The ability to create remediation plans, goals, and SLAs is a welcome function to help keep teams focused on the most critical tasks. Ultimately, InsightVM is a product that benefits greatly from integration with the rest of the Rapid7 portfolio.
Target Market: Rapid7 targets a broad range of customers, from small businesses to very large enterprises.
Time-to-Value: Time to value will depend heavily on how many scanning sensors need to be deployed and where they need to be deployed (this is discussed in more detail in the Overview [AS1] document under Scanner Architecture). For our review, we’ll consider a small enterprise with ~2000 assets, and a few segmented networks. We’ll depend on agents for a series of smaller sales offices with 3-5 assets apiece.
Including time for planning meetings, we’re estimating the effort at around 40 hours of combined effort to get Rapid7 InsightVM deployed in this size environment. This isn’t to say that everything will be deployed 40 hours after license keys are received, but that the total effort should be around 40 hours of time spent (e.g., 4 people in a 1-hour meeting = 4 hours).
To be clear, we’re defining value in the vulnerability scanner segment as the moment a scan successfully completes and returns with results on the requested hosts or network ranges.
Maintaining Value: It could be argued that what got us to the initial state of “value” won’t keep us there. If the scans aren’t scheduled correctly and a second scan never runs, the value train drifts to a halt. Same thing happens if someone changes the password or SSH key to the service account used to perform authenticated scans. Perhaps management is also expecting to see metrics from these new vulnerability management tools. For these reasons, there are a few factors we’re considering necessary to maintain value:
- The work necessary to set up monitoring for scans and ensure they continue to run correctly
- The work necessary to build and pull regular reports for management, meetings, etc.
- Managing the underlying operating systems for each scan engine
Total Cost: Three general categories are considered when calculating total cost: labor cost, product cost, and infrastructure costs. There are a lot of assumptions in here, even within the parameters we’ve set, so feel free to play with the numbers to make these reflect your environment.
- Product cost: Rapid7’s pricing calculator stops at 1,000 assets and our example assumes twice that, so we’ll use the 1,000-asset price ($1.63 per month per asset) to be conservative. The total for 2,000 assets comes to $39,120 per year.
- Deployment cost (labor): As previously mentioned, we’re estimating labor for junior-level folks. Using the salary estimates you can check out in the methodology document; 40 hours of labor comes to $1,346. Add onto this four hours for the one-off task of making sure someone gets notified when scans break for an additional $134.60
- Deployment cost (infrastructure): If deploying to bare metal, plan on three modest 1U servers meeting Rapid7’s recommendations, for around $1,500 apiece ($4,500 total). Or use those old dusty ones in a corner. Or ask the vSphere admins nicely for some resources.
- Maintaining value (labor): this breaks down into a few categories
- Maintenance of the scan engine (e.g., tweaking scan configurations) and the underlying OS: 4 hours per scan engine per month at a junior rate comes to $4,845.60 per year.
- The work of building and distributing reports and metrics will vary widely depending on the organization, but we’ll say a middle-of-the-road estimate would come to 2 hours per week, for a total of $3,499.60 per year.
- The work of analyzing and validating vulnerabilities can be the real time killer for a lot of organizations. It’s also tough to estimate, as the workload is heaviest when scanning assets for the first time, and wanes over the life of the asset somewhat. Throw in compliance and regulatory requirements and that workload increases dramatically (e.g., PCI and the need to obtain quarterly clean scans for the QSA). Our estimate, for a non-regulated organization is going to be 80 hours of work in the first month, going down to 20 hours per month after that initial big push. That initial 80 hours will likely involve senior folks to help triage findings (say, a 50/50 split) and train junior folks on separating signal and noise. We’ll estimate a 25/75 split for the ongoing work, as senior folks continue to validate some of the vulnerabilities and mentor junior staff. The total comes to $13,292.70 per year.
- Finally, tracking down unknown assets and their owners can also eat a lot of time and has a similar workload curve that’s heavy on the front, but tapers off to a constant value over time. Assuming a split between senior and junior staff that mirrors the previous estimate, we can easily see 40 hours spent on this in the first month and 10 hours per month following. The total comes to $6,646.36
All told, we estimate running Rapid7’s InsightVM in a 2,000-asset enterprise would run around $73,250.25 per year.
Strengths: Rapid7 has deep integrations across its own product lines and with third parties. Opportunities to automate common vulnerability management tasks or use vulnerability data to make automated risk decisions make it a strong choice.
Weaknesses: Deployment process could be more streamlined. Fragmented management experience across multiple consoles. As with all current vulnerability scanning products, this product struggles to accurately identify a broad range of IoT devices, particularly those that are Linux-based.
Conclusion: Rapid7’s InsightVM has all the features we’d expect, but it’s the efforts toward reducing manual VM tasks that makes it most worth a look.
Deployment and configuration
Rapid7 makes it easy to initiate a trial of InsightVM without having to talk to someone in sales, which is always appreciated. Initiating a trial by talking to a human first isn’t necessarily a bad thing, but sometimes it can introduce some unnecessary delays and friction, especially when testing multiple products at the same time, as we are.
After registering for a trial, we’re prompted to download either a Windows or Linux installer (a virtual appliance is also an option). The installer offers two roles – local console, plus the scan engine, or just the scan engine alone (for cases where a distributed architecture is necessary). After logging into the local console and entering the license key, it prompts us to log in with an Insight portal (SaaS) account. After getting an account set up, we’re given a pairing key to connect the local console with the Insight portal.
All told, we’re up and running in less than an hour.
Usage
After deploying a local scan engine and pairing it with the Insight portal, we expected to be able to manage scans from the SaaS console. Unfortunately, it seems that all scan-related functions must be managed via the local console. The Insight portal appears to be more focused on analysis and tracking. Some functions in the local console redirect to the Insight portal, so it might take some time to learn which console manages each group of functions. Since scans are managed via the local console, considerations for placement and access to the console should be made in case remote scan management is necessary.
As with other vulnerability scanners, we start by creating what Rapid7 calls a ‘site’. The site mechanism makes it easy to separate out assets by department, asset owners, or subsidiaries. It includes asset owner contact information, custom tags, and a myriad of other details. The ability to document these details within the product makes it easier to deal with staff attrition or role changes. The question of “who do I pester about this horribly insecure server” is easily answered by the tool itself.
For organizations that assign vulnerability management responsibility to asset owners themselves, this approach also lends itself well to role-based access control. Each asset owner can log in and see only the risk associated with systems they care about. They’re able to see month-over-month progress easily without having to fiddle with filters on dashboards.
It’s possible to assign assets to a site using static lists, static groups, or dynamic groups. Dynamic asset groups make it possible to have a group that includes all systems running macOS, for example. Whenever discovery scans pick up new Macs, they automatically get added to the group (and the site). This feature may seem minor, but potentially saves hours of time each month that would otherwise be spent manually assigning new assets to their respective groups. More likely, assets would go unseen for long periods of time, representing an unknown risk to the organization.
Dynamic groups reduce management overhead
We’d recommend using dynamic groups as much as possible. In addition, it would be wise to create a dynamic group that’s the inverse of all other groups. In other words, a group that acts as a catch-all for assets that aren’t in any other group, to ensure nothing slips through the cracks.
One of the more welcome additions to the vulnerability management world in the last decade is the correlation of vulnerabilities with known exploits and active threats, moving away from using base CVSS scores alone to prioritize. After extensive research, we’ve found that the existence of a Metasploit module is one of the highest quality risk indicators available. Given that Metasploit is part of Rapid7, it wasn’t surprising to find module correlation built into asset and vulnerability reporting.
InsightVM correlates vulnerabilities with Metasploit and ExploitDB
Risk scoring was a bit baffling at first. One asset has a score of 180000, while another has a score of 1200? What’s going on here? After doing a bit of reading, our understanding is that each vulnerability has a potential score between 0-1000. That means a host with four vulnerabilities has a theoretical range of 0-4000 on Rapid7’s risk scale. Yeah, that host in our lab with a score of 180000 is (intentionally) four or five versions behind on its operating system.
InsightVM also includes the ability to define remediation projects, goals, and SLAs. Like asset groups, remediation projects can be based on static or dynamic assets. This looks like a great feature for organizations just starting out that have lots of work to do and need to break things up into manageable chunks. Remediation projects can then be assigned to asset owners and given a due date.
Goals and SLAs have some additional functionality. We see these somewhat like a policy enforcement tool. If there’s a policy that credentialed scans should be performed for all supported assets, this can be created as a ‘continuous’ (non-ending) goal. Goals, SLAs, and remediation projects all get reported in a monthly executive summary report that is automatically generated once a month.
Integrations
A number of products will happily import Rapid7 Nexpose-format scan output, but it’s Rapid7’s Extensions that really caught our eye. These fall into a few categories. Workflows enable alerts and management from within Microsoft Teams and Slack. Workflows largely focus on awareness (alerting), management (removing decommissioned assets), and asset lookups.
Integrations enable some interesting orchestrations with third party products. One of our favorites leverages CyberArk to provide unique credentials, eliminating the need for static service account credentials for each credentialed scan. Another enables Carbon Black to quarantine a device if its risk score goes to high (or take it back out of quarantine if it improves) – a very NAC-like workflow.
As with most products that discover vulnerabilities, Service Now and Jira integrations are available as well.
Support
Rapid7 has excellent documentation and responsive support. The company has always been very community-focused and is very supportive of open-source software. A free, open source version of Metasploit remains available, boding well for Rapid7’s recent acquisition of the popular open-source endpoint visibility tool, Velociraptor. Rapid7 has solid integration within its own SOAR and EDR products.
Claims
“Gain Clarity Into Risk”
Sure, this is a defensible claim – this is the core of what InsightVM does. Not only is the risk quantified, but each vulnerability writeup includes references for more information and remediation advice.
“Extend Security’s Influence”
“See Shared Progress”
These last two hit on a passionate topic – the importance of allowing groups of asset owners to also ‘own’ the responsibility for security. Rapid7 certainly enables this with the ability to create asset groups and sites with distinct owners. However, it takes more than a tool to enable a cultural shift – it’s something that starts with leadership. A tool that enables shared responsibility is certainly a step in the right direction though.
EULA Check
Note: we’re not lawyers and this should not be interpreted as or taken as legal advice.
A review of Rapid7’s EULA, as posted here (last updated April 2021), shows it to be non-restrictive when it comes to product reviews and testing. What does this mean? It means that customers and reviewers alike can test the company’s products and share their findings with others.
Security program fit
The core of network vulnerability scanners is identifying vulnerabilities in devices, occupying that upper left-hand corner of the Cyber Defense Matrix. All modern vulnerability scanners also have built-in web application scanners (DAST) as well, so each of these vendors occupy that second square as well.
Conclusion
Rapid7 is one of the three dominant vulnerability scanning vendors and the maturity of the product shows. With deep integrations, threat intelligence correlation, and remediation guidance, customers get a lot for the money. Our reviewers have long memories and note that InsightVM is actually less expensive and more feature rich than the standalone Nexpose product was over a decade ago (without even taking inflation into account). It’s worth a look, especially if Insight IDR, or Insight Orchestrator catch your eye, as there are advantages when investing in the broader suite of products. More than anything else, Rapid7’s focus on automating repetitive workflows is the feature most likely to move the needle for customers.
