There are more than 100 federal cybersecurity service providers (CSSPs), network operations centers (NOCs), security operating centers (SOCs), and cybersecurity integrity centers (CICs) that deliver a unique and distinctive layer of cyber defense as part of our nation’s defense-in-depth strategy. These cyber organizations are established, organized, certified, and accredited to provision one or more cybersecurity services to implement and protect U.S. information systems and networks.
While federal cyber service providers tailor cybersecurity services to meet the customer’s mission and operational priorities, cybersecurity services are first baselined according to the generally organized areas:
- Identify: asset management, governance, risk assessment, and risk management strategy;
- Protect: vulnerability assessment and analysis, vulnerability management, and data security;
- Detect: information security continuous monitoring, insider threat, warning intelligence, and attack sensing and warning;
- Respond: cyber incident handling;
- Recover: continuity of operations.
Since its establishment in 2001, the DoD CSSP Program has progressively and systematically matured to become one of the most critical components of the Defense Department’s cybersecurity strategy. Today, 27 DoD CSSPs are responsible for provisioning 24x7x365 cybersecurity services to implement and protect DoD information systems and networks protecting a cyber landscape that extends globally to more than 145 countries, 15,000 classified and unclassified networks, and 7.5 million computers and information technology devices worldwide.
CSSP evolution through evaluation
Every three years, DoD CSSPs conduct a rigorous formal evaluation using the DOD Cybersecurity Services: evaluator scoring metrics or ESM. The ESM contains the criteria for which all DoD 24x7x365 cybersecurity services are provisioned and CSSP evaluations are conducted. The criteria is built from CSSP stakeholders’ coordinated metrics and required cybersecurity functions of the DoDI 8530.01 and other executive, national, federal, and DoD cybersecurity requirements, which govern cyberspace operations in the DoD.
ESM v8. Released in 2011, ESM version 8 was organized into four functional areas: protect, detect, react, and restore – and 18 computer network defense (CND) services. The 18 CND services were further augmented by 117 compliance-based metrics. Each metric was also prioritized in four levels, I, II, III, and IV, with priority level I representing the highest level of criticality and level IV representing the lowest one. During a computer network defense service provider (CNDSP) evaluation, each metric was assessed and assigned a value of compliant, non-compliant, or partially-compliant. After an ESM evaluation, each CND provider received a level I, II, or III Authorization to Operate designation determined by the overall accumulation and percentage of compliant priority I, II, III, and IV metrics.
For example, to attain an ATO level 3 CNDSP certification, the provider was required to meet 100% of priority I metrics, 100% of priority II metrics, and no less than 90% of priority III and IV metrics. The ESM version 8 was in effect for four years; it was replaced in 2015 by ESM v9.
ESM 9.2. ESM version 9.2, released in 2015, was the result of a wave of updates to the Defense Department’s DoD 8500 policy series. ESM 9.2 was considered by many in the CSSP community as the best ESM version. In fact, the DHS CSP Program, a sister to the DOD CSSP Program, used the ESM 9.2 to establish its CSP mission in late 2019. Through a partnership with the DoD, the Department of Homeland Security (DHS) was able to quickly benefit from the adoption and minimum tailoring of the DOD CSSP ESM 9.2. Today, 14 DHS CSPs are responsible for provisioning 24x7x365 cybersecurity services to implement and protect DHS information systems and networks globally.
Changes in cybersecurity policy triggered the full revamp of ESM v8 into a new version that incorporated a maturity model and metrics to monitor and measure the progress and continuous improvement of provider and subscriber’s relationships; cooperation, collaboration, and coordination among the CSSP community; and innovation and sharing. With these updates, the program was also renamed from CNDSP Program to CSSP Program. The formal and final ESM v9 was organized into four functional areas: protect, detect, respond, and sustain – and 13 cybersecurity activities. The 13 cybersecurity activities were further augmented by 38 performance metrics. Each metric was also prioritized in five maturity levels: ML0 (incomplete), ML1 (performed), ML2 (advanced), ML3 (optimized), and ML4 (innovative).
During a CSSP evaluation, each maturity level was assessed and assigned a value of fully executed and fully documented, and each metric was assessed and assigned a value of achieved, not achieved, or not applicable. After an ESM evaluation, each CND provider received a CSSP designation determined by the overall accumulation and percentage of compliant performance metrics. The ESM version 9 was in effect for 4 years; it was replaced in 2019 by ESM v10.
ESM v10. Today’s ESM version, v10, was released in 2019 and driven by Executive Order 13800, Strengthening the Cybersecurity of Federal Networks and Critical Infrastructure, which mandated the adoption of the NIST Cybersecurity Framework (CSF). This ESM is organized into NIST CSF core functions, categories and sub-categories to ensure the ability for mission owners to identify, protect, detect, respond, and recover in support of defensive cyber operations (DCO), DCO internal defensive measures (IDM), and DODIN Operations.
The formal and current ESM v10 is organized in NIST CSF core functions – identify, protect, detect, respond, and recover – and 20 cybersecurity service categories and sub-categories. The 20 cybersecurity service categories were further augmented by 50 metric indicators, numbered for reference and categorized by NIST cybersecurity framework functions, and further divided into measures of performance (MOPs) and measures of effectiveness (MOEs).
MOPs are the ESM’s traditional review of relevant operational processes, doctrine, and artifacts in order to show a DoD Component has the processes and procedures in place to defend their portion of the DODIN. Additionally, interviews and demonstrations are performed to validate artifact review to ensure results are accurate.
MOEs inject cyber effects into the assessed mission space to measure the effectiveness of defensive cyber forces (DCF). MOEs are considered critical elements and are graded based on the component’s ability to identify, protect, detect, respond and recover from cyber threat emulation (CTE) simulated adversarial activities and follow established policies for handling cyber event and/or incidents IAW DOD or intelligence community (IC) guidance.
Many in the CSSP community consider ESM v10 a step back after the initial two steps forward achieved by ESM v8 and ESM 9.2. While the intention of ESM v10 was fundamentally sound, the approach fell short of the previous two versions.
Simply put, the CSF is really a framework and not a standard. Frameworks are not overlayed and forced to fit. NIST developed the CSF framework in conjunction with industry for customization so it could precisely meet the needs of wide-ranging organizations including CSSP programs. The CSF consists of three primary parts: core, implementation tiers, and profiles; each of which support and requires tailoring. The ESM v10 simply treated the CSF as a standard and failed to tailor the CSF to meet CSSP specific needs and operational CSSP requirements.
Cesar Pie, president; Clinton Hackney, vice president cyberspace operations, CSIOS Corporation
