This review is part of the September 2021 assessment of network vulnerability scanners. If you haven’t read the category overview, you might want to check it out — it explains the category’s basics, use cases and the general value proposition. Our testing methodology explains both how we interact with vendors and how we tested these products.
Vulnerability scanners are the sensors of the vulnerability management process — they reach out and touch systems to determine if they are vulnerable to exploits and other threats. While there are vulnerability scanners for containers, infrastructure-as-code tooling, cloud management consoles, and web applications, this group review will focus on network vulnerability scanners.
Company background
Digital Defense was acquired by HelpSystems in February 2021, joining the consolidated cybersecurity teams, services, and products of Courion, Cobalt Strike, Damballa, and Core Security. The security portfolio of HelpSystems includes identity governance, penetration testing, and threat detection tools in addition to Digital Defense’s vulnerability and threat management tools.
The Digital Defense brand is less recognizable than others in this market, but has been around since 1999 — the very early days of vulnerability management (and security vendors!). The San Antonio-based company was angel-investor backed and never took VC investment. While the name Digital Defense or Frontline might not be familiar, the company has had some significant deals to white-label the product over the years. As a result most defenders in the cybersecurity industry have likely used the patented Frontline VM technology at one time or another, even if they didn’t realize it.
Review summary
Frontline VM is fast and simple to deploy. The interface is SaaS-based, while data is collected from easily deployable scan appliances and agents. Physical scan appliances can be purchased from Digital Defense, though we assume most customers will opt to one of the many virtual appliance options, either on-prem or in the cloud. Agents are available for 32-bit and 64-bit Windows as well as macOS.
Anyone familiar with managing a vulnerability scanner in the past 20 years will be able to do it in their sleep with Frontline. All the options we needed were there and they were exactly where we expected to find them. Scans run quickly and results are easy to find. Dashboards are functional and useful, making it obvious when there are truly critical vulnerabilities that represent an imminent threat. Prioritization performed well in our testing, made possible by Digital Defense’s Threat Landscape functionality.
A plethora of reporting options are available — PDF output looks better than most. RBAC controls exist for different business groups and/or departments that need access to the product, reports or dashboards. Asset groups can be static or dynamic and enable more functional RBAC groups.
Target market: Frontline is appropriate for a wide range of markets and business sizes, from small businesses to large enterprises.
Time-to-value: Time to value will depend heavily on how many scanning sensors need to be deployed and where they need to be deployed (this is discussed in more detail in the Overview document under Scanner Architecture). For easy comparison across our reviews, we factor our estimates using a hypothetical small enterprise that has 2,000 assets, 1,000 employees and a few segmented networks. We’ll use agents to support smaller sales offices and remote workers.
Including time for deployment-related meetings, we’re estimating the effort at around 40 hours of combined effort to get Frontline deployed in our hypothetical environment. This isn’t to say that everything will be deployed 40 hours after license keys are received, but that the total effort should be around 40 hours of work-hours spent (e.g., 4 people in a 1-hour meeting = 4 hours).
To be clear, we’re defining value in the vulnerability scanner segment as the moment a scan successfully completes and returns with results on the requested hosts or network ranges. This value will typically increase as staff get increasingly more familiar with the tool, tune it over time. There’s also a lot of initial work that comes with initial baseline scans, identifying assets, identifying asset owners, and validating findings.
Maintaining value: It could be argued that what got us to the initial state of “value” won’t keep us there. If the scans aren’t scheduled correctly and a second scan never runs, the value train drifts to a halt. Same thing happens if someone changes the password or SSH key to the service account used to perform authenticated scans. Perhaps management is also expecting to see metrics from these new vulnerability management tools. For these reasons, there are a few factors we’re considering necessary to maintain value:
- The work necessary to set up monitoring for scans and ensure they continue to run correctly
- The work necessary to build and pull regular reports for management, meetings, etc.
- Managing the underlying operating systems for each scan engine
Total cost: Three general categories are considered when calculating total cost: labor cost, product cost, and infrastructure costs. There are a lot of assumptions in here, even within the parameters we’ve set, so feel free to play with the numbers to make these reflect your environment.
- Product cost: For 2,000 assets, the cost for a Frontline Advanced subscription is just under $25,000 per year.
- Deployment cost (labor): We think deploying scanner VMs and agents can be handled by junior-level folks. We’re estimating 40 hours of labor to get it deployed and an additional 4 hours to set up notifications in case scans break. For junior-level resources, the total cost for deployment comes to $1,480.60 (Check out the methodology document for details on how we calculate labor estimates)
- Deployment cost (infrastructure): While Digital Defense does offer hardware Frontline appliances, deploying Frontline VMs would probably be more economical. Given that existing VM infrastructure would likely already exist in an organization of this size, security teams can usually get away with deploying a few scanning engines without any direct hit to their budget. Worst case, three modest systems capable of running scanners might run around $4,500, if we’re being conservative and include labor to set them up.
- Maintaining value (labor): this breaks down into a few categories
- Maintenance of the scan engine (e.g., tweaking scan configurations) and the underlying OS: 4 hours per scan engine per month at a junior rate comes to $4,845.60 per year.
- The work of building and distributing reports and metrics will vary widely depending on the organization, but we’ll say a middle-of-the-road estimate would come to 2 hours per week, for a total of $3,499.60 per year.
- The work of analyzing and validating vulnerabilities can be the real time killer for a lot of organizations. It’s also tough to estimate, as the workload is heaviest when scanning assets for the first time, and wanes over the life of the asset somewhat. Throw in compliance and regulatory requirements and that workload increases dramatically (e.g., PCI and the need to obtain quarterly clean scans for the QSA). Our estimate, for a non-regulated organization is going to be 80 hours of work in the first month, going down to 20 hours per month after that initial big push. That initial 80 hours will likely involve senior folks to help triage findings (say, a 50/50 split) and train junior folks on separating signal and noise. We’ll estimate a 25/75 split for the ongoing work, as senior folks continue to validate some of the vulnerabilities and mentor junior staff. The total comes to $13,292.70 per year.
- Finally, tracking down unknown assets and their owners can also eat a lot of time and has a similar workload curve that’s heavy on the front, but tapers off to a constant value over time. Assuming a split between senior and junior staff that mirrors the previous estimate, we can easily see 40 hours spent on this in the first month and 10 hours per month following. The total comes to $6,646.36
All told, we estimate running Digital Defense Frontline in a 2,000-asset enterprise would run around $59,264.86 per year.
Strengths: Digital Defense claims a high number of existing customers (over 2,000) a very high retention rate (98%) and high NPS score (75+). All this should be confidence-inspiring, especially for those unfamiliar with the brand. We found the product both easy to use and full-featured — a difficult balance to pull off, and one that some competitors struggle with.
Weaknesses: As with all current vulnerability scanning products we tested, this product struggles to accurately identify a broad range of IoT devices, particularly those that are Linux-based.
Conclusion: Frontline is a full-featured, legitimate alternative to competing products from Qualys, Rapid7, and Tenable.
Deployment and configuration
Frontline offers the usual mix of scanning options we’d expect to see from a vendor that emerged with a vulnerability management product in the early 2000s: internal scanning (via the RNA virtual appliance), external scanning, web application scanning, agent-based scanning, container scanning and versions of the RNA scanning appliance in AWS, Oracle Cloud, and Azure marketplaces. This set of group reviews is focusing primarily on what has traditionally been the most time-consuming component for organizations: internal vulnerability scanning.
For on-premises deployments, the RNA appliance is available in both OVA and Hyper-V image downloads, as well as a pre-configured physical appliance. We went with OVA and the setup was quick and easy: download, run, activate with an appliance token from the Frontline.cloud console. We were done in well under an hour, including the time to download the OVA. The console shows the status and details of all paired RNA appliances. While it would be nice to be able to modify the network configuration from the SaaS console, reconfiguring it via the VM console is simple enough that we wouldn’t need to provide instructions to the vSphere admin.
Usage
Now that we’ve got scanners deployed, the next step is to create a scan. This is done by going to Scheduled Scans → New scan, but we also noticed that a persistent, light blue box at the top of the UI acts as a shortcut for creating new users, scans, reports, or asset groups, regardless of what screen we might be on. By default, scans are a one-time affair, but they can easily be configured to reoccur, as can be seen in our daily scan in the screenshot below.
Digital Defense provides many built-in policies. We went with the default (literally named Default). Others range from different discovery options, to scans for specific, highly popular vulnerabilities. An interesting option we haven’t seen elsewhere is the ability to run an Active Threat Scan as opposed to a Vuln Assessment. From what we can tell, the Active Threat Scan looks for signs of active malware infections or other malicious activity. There’s even a Malware Infections widget on the main dashboard — not something we’d typically expect to see in a vulnerability management product, but a welcome bonus feature.
With scans scheduled, we’re ready to look at what it’s like to live with Frontline long-term.
While we don’t mind vendors getting creative when it comes to interfaces, many have become downright confusing. What the Frontline VM interface might lack in "flair" or dark modes, it more than makes up by being straightforward and intuitive. Dashboards are useful, showing some basic trends and statistics. They’re also actionable, showing the riskiest assets and vulnerabilities.
Frontline is better than most scanner offerings when it comes to prioritization. The Threat Landscape mode can quickly hide anything that isn’t an active threat, making it easy to zero in on the most critical vulnerabilities. Like most other modern vulnerability management vendors, Frontline prioritizes using curated threat intelligence gathered from a dedicated internal threat research team, as well as outside sources.
So many security tools have an interactive diagram view, but we’ve rarely used them. In this case, however, the diagram view is probably the best place to quickly spot anomalies — assets with known exploits, active threats, malware, and disabled anti-virus, for example.
Digital Defense offers other subscriptions and the status of these can be seen within the product. Some subscriptions extend the Frontline product capabilities, while others are services performed by Digital Defense’s consulting team.
Notable integrations
Frontline has a multitude of integrations with vulnerability prioritization, patching, deception, NAC, SIEM, ticketing, and many other types of products. A full list can be found here. Frontline also has a comprehensive API that covers a wide range of use cases.
Support
Digital Defense’s knowledge base answered all our questions and we had no need to use any human support options. The Frontline support resources have clearly been built up over a long period of time.
Claims
“The industry’s most comprehensive, accurate, and easy-to-use SaaS vulnerability management solution”
Is it easy-to-use? Absolutely. Is it comprehensive? Well, we can’t really say, as our testing wasn’t comprehensive. Was it accurate? Except for IoT devices, yes, it seemed to be. Was it the most accurate in the industry? Impossible to say without a comprehensive performance assessment, which we did not perform.
EULA check
Note: We’re not lawyers and this should not be interpreted as legal advice.
After reviewing HelpSystems’ EULA (Digital Defense was acquired by HelpSystems earlier this year), we didn’t find any problematic language that would prevent software testing or sharing results of that testing (our reviews, for example).
For more information on why we do EULA checks, check out https://transparencyincyber.org. We also discuss it in the vulnerability scanning methodology.
Security program fit
The core of network vulnerability scanners is identifying vulnerabilities in devices, occupying that upper left-hand corner of the Cyber Defense Matrix. All modern vulnerability scanners also have built-in web application scanners (DAST) as well, so each of these vendors occupy that second square as well. It is worth mentioning, however, that there are often two levels of web application scanners — the one built into the network scanner tends to be basic in terms of capabilities and features. Then there’s often a separate, standalone product — much more capable and full-featured — that must be purchased separately.
Conclusion
Frontline is a full-featured, legitimate alternative to competing products from Qualys, Rapid7, and Tenable. For most use cases and organization sizes, Digital Defense’s offering should be included in any vulnerability bake-off, or proof-of-concept. Frontline is easy enough for IT (non-security) folks to set up and manage, but full-featured enough for a large enterprise with a dedicated vulnerability management group.
