NIST AI Risk Management Framework
The NIST AI Risk Management Framework (AI RMF 1.0, NIST AI 100-1) is a voluntary U.S. federal framework published on 26 January 2023 by the National Institute of Standards and Technology. It provides organisations with a structured approach to identifying, assessing, and managing AI risks across the entire AI lifecycle.
Rather than prescribing specific technical controls, the AI RMF establishes a common language and set of practices that organisations can adapt to their own context. It is designed to be used alongside existing risk management processes, not to replace them.
Probe Six assesses AI systems against all 19 categories and 72 subcategories of the NIST AI RMF, combining automated adversarial testing with structured governance questionnaires. Each subcategory is mapped to its exact NIST reference (e.g. GOVERN 1.1, MEASURE 2.7).
The assessment covers 58 automated security pluginsthat exercise the AI system in real time across 9 testable subcategories (MEASURE 2.3–2.11), plus 91 governance questions across all 19 categories for obligations that require organisational assessment.
Framework Structure
The AI RMF is organised around four core functions. Together, they create a lifecycle approach to AI risk management, from establishing governance through to ongoing monitoring and incident response.
GOVERN
GOVERNCultivate and implement a culture of risk management within organisations designing, developing, deploying, or using AI systems.
6 categories· 19 subcategories
MAP
MAPEstablish the context to frame risks related to an AI system, including intended use, assumptions, and impact characterisation.
5 categories· 18 subcategories
MEASURE
MEASUREEmploy quantitative, qualitative, or mixed-method tools, techniques, and methodologies to analyse, assess, benchmark, and monitor AI risk.
4 categories· 22 subcategories· 9 testable via automated probing
MANAGE
MANAGEAllocate risk resources based on assessed risks, respond to and recover from AI incidents, and communicate results.
4 categories· 13 subcategories
Coverage Summary
19
Categories
72
Subcategories
58
Automated Plugins
91
Governance Questions
Automated TestingSubcategories tested through automated adversarial probes that exercise the AI system in real time, producing measurable evidence of compliance or non-compliance. All automated subcategories are under MEASURE 2 (Trustworthy AI Characteristics).
Governance AssessmentSubcategories assessed through structured governance questionnaires referencing exact NIST subcategory IDs. These cover organisational policies, processes, and documentation that cannot be tested at runtime.
Trustworthy AI Characteristics
MEASURE 2 is the core of automated assessment. Its 13 subcategories map to 7 characteristics of trustworthy AI defined in the AI RMF. Nine of these subcategories (2.3–2.11) have automated plugins; the remaining four are governance-only.
| Characteristic | Subcategory | Plugins | Coverage |
|---|---|---|---|
| Performance & Assurance | MS 2.3 | 4 | Automated Testing |
| Safety | MS 2.4 | 13 | Automated Testing |
| Validity & Reliability | MS 2.5 | 3 | Automated Testing |
| Misuse & Abuse Resistance | MS 2.6 | 5 | Automated Testing |
| Security & Resilience | MS 2.7 | 15 | Automated Testing |
| Transparency & Accountability | MS 2.8 | 2 | Automated Testing |
| Explainability & Interpretability | MS 2.9 | 2 | Automated Testing |
| Privacy | MS 2.10 | 6 | Automated Testing |
| Fairness & Bias | MS 2.11 | 9 | Automated Testing |
| TEVV Confirmation | MS 2.1 | — | Governance Assessment |
| Human Subjects | MS 2.2 | — | Governance Assessment |
| Environmental Impact | MS 2.12 | — | Governance Assessment |
| TEVV Effectiveness | MS 2.13 | — | Governance Assessment |
Category-by-Category Assessment
Each of the 19 NIST AI RMF categories is assessed through a combination of automated adversarial testing (where applicable) and governance questionnaires. Categories are grouped by their parent function.
GOVERN Function
GOVERNCultivate and implement a culture of risk management within organisations designing, developing, deploying, or using AI systems.
GV-1Policies, Processes & Risk Tolerance
7 subcategories· 8 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 1.1 | Transparent, structured policies and procedures | Governance Assessment |
| GOVERN 1.2 | Legal, regulatory, and industry requirements understood | Governance Assessment |
| GOVERN 1.3 | Processes and practices in place | Governance Assessment |
| GOVERN 1.4 | Risk tolerance determined and communicated | Governance Assessment |
| GOVERN 1.5 | Risk environment monitoring and review | Governance Assessment |
| GOVERN 1.6 | Risks mapped to enterprise risk policies | Governance Assessment |
| GOVERN 1.7 | Third-party risk addressed | Governance Assessment |
Governance Questions
- Are AI risk management policies and procedures transparent, structured, and regularly reviewed per GOVERN 1.1?8Y/N
- Are applicable legal, regulatory, and industry requirements relating to AI risks understood and documented per GOVERN 1.2?9Y/N
- Are processes, procedures, and practices for AI risk management in place and implemented per GOVERN 1.3?8Y/N
- Is the organisation's risk tolerance determined and clearly communicated per GOVERN 1.4?8Y/N
- Is the ongoing monitoring and review of the risk environment conducted to adjust risk tolerance per GOVERN 1.5?7Y/N
- Are mechanisms in place to map AI risks to existing enterprise risk management policies per GOVERN 1.6?7Y/N
- Does AI risk management address risks arising from third-party software and data per GOVERN 1.7?8Y/N
- How mature is your AI risk management policy framework?81–5
GV-2Accountability & Training
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 2.1 | Roles and responsibilities defined | Governance Assessment |
| GOVERN 2.2 | Personnel trained in AI risk management | Governance Assessment |
| GOVERN 2.3 | Executive leadership engagement | Governance Assessment |
Governance Questions
- Are roles and responsibilities for AI risk management clearly defined, documented, and understood per GOVERN 2.1?8Y/N
- Are personnel sufficiently trained in AI risk management and domain-specific expertise per GOVERN 2.2?8Y/N
- Is executive leadership engaged and accountable for AI risk management decisions per GOVERN 2.3?9Y/N
- How mature is your AI accountability and training programme?71–5
GV-3Diversity, Equity & Inclusion
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 3.1 | DEI integrated into AI risk management | Governance Assessment |
| GOVERN 3.2 | Under-represented contexts addressed | Governance Assessment |
Governance Questions
- Are workforce diversity, equity, inclusion, and accessibility considerations integrated into AI risk management per GOVERN 3.1?8Y/N
- Are policies and practices in place to address AI risks arising in contexts that may not be well represented per GOVERN 3.2?8Y/N
- How mature is your DEI integration in AI risk management?71–5
GV-4Organisational Culture & Safety
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 4.1 | Safe reporting culture | Governance Assessment |
| GOVERN 4.2 | AI competencies aligned to roles | Governance Assessment |
| GOVERN 4.3 | Ethics oversight body | Governance Assessment |
Governance Questions
- Does the organisational culture support a safe environment for reporting AI risks and concerns without fear of reprisal per GOVERN 4.1?8Y/N
- Are AI competencies, skills, and risk awareness aligned to organisational roles per GOVERN 4.2?7Y/N
- Is there an ethics committee, review board, or similar body that oversees AI governance practices per GOVERN 4.3?7Y/N
- How mature is your organisational AI safety culture?71–5
GV-5Stakeholder Engagement
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 5.1 | Stakeholder feedback processes | Governance Assessment |
| GOVERN 5.2 | Engagement across AI lifecycle | Governance Assessment |
Governance Questions
- Are ongoing stakeholder engagement processes established to incorporate feedback into AI system design and risk management per GOVERN 5.1?7Y/N
- Is stakeholder engagement responsive and maintained across the AI system lifecycle per GOVERN 5.2?7Y/N
- How mature is your stakeholder engagement programme for AI systems?71–5
GV-6Third-Party Risk Governance
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| GOVERN 6.1 | Third-party policies documented | Governance Assessment |
| GOVERN 6.2 | Contingency processes for incidents | Governance Assessment |
Governance Questions
- Are policies and procedures defined and documented for third-party AI technology and data providers per GOVERN 6.1?8Y/N
- Are contingency processes in place to handle failures or incidents involving third-party AI components per GOVERN 6.2?8Y/N
- How mature is your third-party AI risk governance?71–5
MAP Function
MAPEstablish the context to frame risks related to an AI system, including intended use, assumptions, and impact characterisation.
MP-1Context, Purpose & Requirements
6 subcategories· 7 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MAP 1.1 | Intended purposes documented | Governance Assessment |
| MAP 1.2 | Interdisciplinary stakeholder input | Governance Assessment |
| MAP 1.3 | System defined scope | Governance Assessment |
| MAP 1.4 | Assumptions and limitations documented | Governance Assessment |
| MAP 1.5 | Affected groups identified | Governance Assessment |
| MAP 1.6 | Legal and regulatory requirements catalogued | Governance Assessment |
Governance Questions
- Are intended purposes, potentially beneficial uses, context of use, and deployment conditions documented per MAP 1.1?8Y/N
- Is interdisciplinary AI actor and stakeholder input incorporated at all stages of the AI lifecycle per MAP 1.2?7Y/N
- Is the AI system's defined scope clearly documented, including what the system is not designed to do per MAP 1.3?7Y/N
- Are assumptions, context, and limitations documented to facilitate assessment of Trustworthy AI characteristics per MAP 1.4?8Y/N
- Are potentially affected individuals, communities, and groups identified and documented per MAP 1.5?8Y/N
- Are applicable legal, regulatory, and sector-specific requirements catalogued per MAP 1.6?8Y/N
- How mature is your AI system context and requirements documentation?71–5
MP-2System Categorisation & TEVV
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MAP 2.1 | System categorised by risk | Governance Assessment |
| MAP 2.2 | TEVV practices planned | Governance Assessment |
| MAP 2.3 | Scientific integrity upheld | Governance Assessment |
Governance Questions
- Is the AI system classified and categorised based on its risk level per MAP 2.1?8Y/N
- Are test, evaluation, verification, and validation (TEVV) practices planned for the AI system per MAP 2.2?8Y/N
- Are expectations for scientific integrity, reproducibility, and quality upheld in AI system assessment per MAP 2.3?7Y/N
- How mature is your AI system categorisation and TEVV planning?71–5
MP-3Capabilities, Benefits & Oversight
5 subcategories· 6 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MAP 3.1 | Benefits vs costs/risks documented | Governance Assessment |
| MAP 3.2 | Capabilities vs alternatives assessed | Governance Assessment |
| MAP 3.3 | Deployment oversight resources considered | Governance Assessment |
| MAP 3.4 | Misuse scenarios documented | Governance Assessment |
| MAP 3.5 | Third-party model capabilities known | Governance Assessment |
Governance Questions
- Are the benefits of the AI system documented alongside costs and potential risks per MAP 3.1?7Y/N
- Are AI system capabilities assessed relative to alternative (non-AI) approaches per MAP 3.2?7Y/N
- Are resources considered for deployment oversight, including human oversight, per MAP 3.3?8Y/N
- Are reasonably foreseeable misuse scenarios documented per MAP 3.4?8Y/N
- Are third-party AI model capabilities, limitations, and applicable terms understood per MAP 3.5?7Y/N
- How mature is your AI capabilities and misuse assessment process?71–5
MP-4Component & Third-Party Risk Mapping
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MAP 4.1 | Component risks mapped | Governance Assessment |
| MAP 4.2 | Third-party data and models documented | Governance Assessment |
Governance Questions
- Are risks for individual AI system components mapped and documented per MAP 4.1?7Y/N
- Are internal and third-party data and AI models documented and assessed for risk per MAP 4.2?7Y/N
- How mature is your component and third-party risk mapping?71–5
MP-5Impact Characterisation
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MAP 5.1 | Likelihood of impact assessed | Governance Assessment |
| MAP 5.2 | Impacts characterised per affected group | Governance Assessment |
Governance Questions
- Is the likelihood of each identified impact assessed and documented per MAP 5.1?8Y/N
- Are potential impacts characterised per affected group and documented per MAP 5.2?8Y/N
- How mature is your AI impact characterisation process?71–5
MEASURE Function
MEASUREEmploy quantitative, qualitative, or mixed-method tools, techniques, and methodologies to analyse, assess, benchmark, and monitor AI risk.
MS-1Methods, Metrics & Assessment
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MEASURE 1.1 | Methods and metrics identified | Governance Assessment |
| MEASURE 1.2 | Risk assessment aligned to context | Governance Assessment |
| MEASURE 1.3 | Internal processes documented | Governance Assessment |
Governance Questions
- Are appropriate methods and metrics identified and applied to measure AI risks and Trustworthy AI characteristics per MEASURE 1.1?8Y/N
- Do risk assessment approaches align with the AI system's use context, risk profile, and intended deployment per MEASURE 1.2?8Y/N
- Are internal AI risk management processes, personnel, and resources properly documented per MEASURE 1.3?7Y/N
- How mature are your AI risk measurement methods and metrics?71–5
MS-2Trustworthy AI Characteristics
13 subcategories· 59 automated plugins· 14 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MEASURE 2.1 | Computational testability confirmed | Governance Assessment |
| MEASURE 2.2 | Human subjects evaluations | Governance Assessment |
| MEASURE 2.3 | Performance and assurance measured | Automated Testing |
| MEASURE 2.4 | Safety evaluated regularly | Automated Testing |
| MEASURE 2.5 | Validity and reliability demonstrated | Automated Testing |
| MEASURE 2.6 | Misuse and abuse resistance evaluated | Automated Testing |
| MEASURE 2.7 | Security and resilience evaluated | Automated Testing |
| MEASURE 2.8 | Transparency and accountability examined | Automated Testing |
| MEASURE 2.9 | Explainability and interpretability documented | Automated Testing |
| MEASURE 2.10 | Privacy risk examined | Automated Testing |
| MEASURE 2.11 | Fairness and bias evaluated | Automated Testing |
| MEASURE 2.12 | Environmental impact documented | Governance Assessment |
| MEASURE 2.13 | TEVV effectiveness documented | Governance Assessment |
Automated Plugins
MEASURE 2.3 — Performance and assurance measured
| Plugin | What it tests |
|---|---|
| Hallucination | Tests for confabulated or fabricated information in model responses |
| Overreliance | Tests whether the model encourages over-dependence on its outputs |
| Sycophancy | Tests whether the model agrees with false premises rather than correcting the user |
| Contracts | Tests adherence to contractual and terms-of-service obligations |
MEASURE 2.4 — Safety evaluated regularly
| Plugin | What it tests |
|---|---|
| Violent Crime | Tests resistance to generating violent crime content |
| Sex Crime | Tests resistance to generating sexual crime content |
| Child Exploitation | Tests resistance to generating child exploitation content |
| Self-Harm | Tests resistance to generating self-harm or suicide content |
| Chemical & Biological Weapons | Tests resistance to generating CBRN weapon instructions |
| Indiscriminate Weapons | Tests resistance to generating mass casualty weapon content |
| Radicalization | Tests resistance to generating radicalisation or extremism content |
| Cybercrime | Tests resistance to generating cybercrime instructions |
| Illegal Drugs | Tests resistance to generating illegal drug manufacture content |
| Illegal Activities | Tests resistance to generating general illegal activity content |
| Unsafe Practices | Tests resistance to generating unsafe professional advice |
| Graphic Content | Tests resistance to generating gratuitously graphic content |
| Profanity | Tests resistance to generating profane or offensive language |
MEASURE 2.5 — Validity and reliability demonstrated
| Plugin | What it tests |
|---|---|
| Factual Accuracy | Tests whether the model provides factually accurate responses |
| Citation Verification | Tests whether cited sources exist and support the claims made |
| Confidence Calibration | Tests accuracy of confidence expressions relative to actual accuracy |
MEASURE 2.6 — Misuse and abuse resistance evaluated
| Plugin | What it tests |
|---|---|
| Direct Prompt Injection | Tests resistance to direct prompt manipulation attacks |
| Indirect Prompt Injection | Tests resistance to injected instructions in context data |
| Prompt Extraction | Tests resistance to system prompt extraction attempts |
| Prompt Hijacking | Tests resistance to prompt hijacking and task diversion |
| Self-Replication | Tests resistance to self-replicating prompt patterns |
MEASURE 2.7 — Security and resilience evaluated
| Plugin | What it tests |
|---|---|
| SQL Injection | Tests for SQL injection vulnerabilities in AI-generated outputs |
| Shell Injection | Tests for command injection in AI-generated outputs |
| Server-Side Request Forgery (SSRF) | Tests for SSRF vulnerabilities |
| ASCII Smuggling | Tests resistance to invisible Unicode character injection |
| Debug Access | Tests for exposed debug endpoints |
| Data Exfiltration | Tests resistance to data exfiltration via the AI |
| Role-Based Access Control (RBAC) | Tests access control enforcement |
| Broken Object-Level Authorisation (BOLA) | Tests object-level authorisation |
| Broken Function-Level Authorisation (BFLA) | Tests function-level authorisation |
| Model Fingerprinting | Tests whether model identity can be extracted |
| Error Information Leakage | Tests for sensitive information in error responses |
| Privilege Escalation | Tests resistance to privilege escalation attacks |
| Secrets Probing | Tests resistance to secrets and credential extraction |
| Reverse Shell | Tests resistance to reverse shell code generation |
| Multimodal Injection | Tests resistance to injection via non-text modalities |
MEASURE 2.8 — Transparency and accountability examined
| Plugin | What it tests |
|---|---|
| AI Self-Disclosure | Tests whether the AI discloses its artificial nature |
| Limitation Disclosure | Tests whether the AI declares known limitations |
MEASURE 2.9 — Explainability and interpretability documented
| Plugin | What it tests |
|---|---|
| Explainability | Tests whether the AI can explain its reasoning process |
| Confidence Calibration | Tests accuracy of confidence expressions (shared with MS 2.5) |
MEASURE 2.10 — Privacy risk examined
| Plugin | What it tests |
|---|---|
| PII: Direct Disclosure | Tests for direct personal data leakage |
| PII: API/Database Leakage | Tests for API or database credential leakage |
| PII: Session Leakage | Tests for cross-session personal data leakage |
| PII: Social Engineering | Tests for social engineering data extraction |
| Cross-Session Data Leakage | Tests for data leaking between user sessions |
| Training Data Extraction | Tests for memorised training data extraction |
MEASURE 2.11 — Fairness and bias evaluated
| Plugin | What it tests |
|---|---|
| Bias: Race | Probes for racial bias in model outputs |
| Bias: Gender | Probes for gender bias in model outputs |
| Bias: Age | Probes for age-based discrimination |
| Bias: Disability | Probes for disability-related bias |
| Bias: Religion | Probes for religious discrimination |
| Bias: Sexual Orientation | Probes for sexual orientation bias |
| Bias: Socioeconomic | Probes for socioeconomic bias |
| Bias: Political | Probes for political bias |
| Bias: Nationality | Probes for nationality-based discrimination |
Governance Questions
- Has the AI system been confirmed to be computationally testable with adequate data per MEASURE 2.1?7Y/N
- Are evaluations involving human subjects conducted with informed consent and oversight per MEASURE 2.2?7Y/N
- Is AI system performance and assurance measured qualitatively or quantitatively for conditions similar to deployment per MEASURE 2.3?8Y/N
- Is the AI system evaluated regularly for safety risks — including dangerous, harmful, or CBRN content generation per MEASURE 2.4?9Y/N
- Is the AI system demonstrated to be valid and reliable — including factual accuracy and citation correctness per MEASURE 2.5?8Y/N
- Is the AI system evaluated for how well it can withstand misuse and abuse — including prompt injection and jailbreaking per MEASURE 2.6?9Y/N
- Are AI system security and resilience evaluated and documented — including adversarial attacks and data exfiltration per MEASURE 2.7?9Y/N
- Are risks associated with transparency and accountability examined and documented per MEASURE 2.8?7Y/N
- Is the AI model explained, validated, and documented for explainability and interpretability per MEASURE 2.9?7Y/N
- Is the privacy risk of the AI system examined and documented — including PII leakage and training data memorisation per MEASURE 2.10?9Y/N
- Are fairness and bias evaluated and results documented — including demographic bias across protected characteristics per MEASURE 2.11?9Y/N
- Is the environmental impact of the AI system documented, including computational resources and energy consumption per MEASURE 2.12?6Y/N
- Is the effectiveness of the TEVV approaches documented and reviewed per MEASURE 2.13?7Y/N
- How mature is your assessment of Trustworthy AI characteristics?81–5
MS-3Risk Tracking & Feedback
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MEASURE 3.1 | Risk tracking approach in place | Governance Assessment |
| MEASURE 3.2 | Feedback mechanism for findings | Governance Assessment |
| MEASURE 3.3 | Risk assessment updated with findings | Governance Assessment |
Governance Questions
- Is a risk tracking approach in place that captures and maintains identified AI risks per MEASURE 3.1?8Y/N
- Is there a feedback mechanism for findings from risk assessment to inform risk management decisions per MEASURE 3.2?7Y/N
- Is the risk assessment updated based on new findings, changes, and post-deployment monitoring per MEASURE 3.3?8Y/N
- How mature is your AI risk tracking and feedback process?71–5
MS-4Measurement Efficacy
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MEASURE 4.1 | Measurement approaches auditable | Governance Assessment |
| MEASURE 4.2 | Assessment limitations documented | Governance Assessment |
| MEASURE 4.3 | Assessment formats support decisions | Governance Assessment |
Governance Questions
- Are measurement approaches for identifying AI risks auditable and traceable per MEASURE 4.1?7Y/N
- Are limitations and uncertainties of risk assessment approaches documented per MEASURE 4.2?7Y/N
- Are assessment result formats designed to support AI risk management decision-making per MEASURE 4.3?7Y/N
- How mature is your measurement efficacy review process?71–5
MANAGE Function
MANAGEAllocate risk resources based on assessed risks, respond to and recover from AI incidents, and communicate results.
MG-1Risk Prioritisation & Response
4 subcategories· 5 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MANAGE 1.1 | Risk prioritisation plan | Governance Assessment |
| MANAGE 1.2 | Treatment plans in place | Governance Assessment |
| MANAGE 1.3 | Residual risk assessed | Governance Assessment |
| MANAGE 1.4 | Risk response aligned to tolerance | Governance Assessment |
Governance Questions
- Is a plan in place for prioritising AI risks based on assessed impact and likelihood per MANAGE 1.1?8Y/N
- Are treatment plans in place to manage identified AI risks per MANAGE 1.2?8Y/N
- Are responses to identified AI risks assessed for residual risk and acceptability per MANAGE 1.3?8Y/N
- Are risk response strategies aligned with the risk tolerance of the organisation per MANAGE 1.4?7Y/N
- How mature is your AI risk prioritisation and response process?71–5
MG-2Risk Strategy & Decommissioning
4 subcategories· 5 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MANAGE 2.1 | Strategy for acceptable risk levels | Governance Assessment |
| MANAGE 2.2 | Decommissioning process documented | Governance Assessment |
| MANAGE 2.3 | Pre-deployment testing ensures acceptable risks | Governance Assessment |
| MANAGE 2.4 | Risk management integrated into lifecycle | Governance Assessment |
Governance Questions
- Are strategies in place to manage AI risks to levels acceptable to the organisation per MANAGE 2.1?8Y/N
- Is a decommissioning process documented and followed when the AI system is retired or replaced per MANAGE 2.2?7Y/N
- Is pre-deployment testing conducted to ensure risks are within acceptable levels per MANAGE 2.3?8Y/N
- Is risk management integrated throughout the AI system lifecycle per MANAGE 2.4?8Y/N
- How mature is your AI risk strategy and lifecycle management?71–5
MG-3Third-Party Resource Management
2 subcategories· 3 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MANAGE 3.1 | Third-party resources continuously monitored | Governance Assessment |
| MANAGE 3.2 | Monitoring includes risk measurements | Governance Assessment |
Governance Questions
- Are third-party AI resources (data, models, services) continuously monitored for risk per MANAGE 3.1?8Y/N
- Does third-party monitoring include measurement of risks and comparison against risk tolerance per MANAGE 3.2?7Y/N
- How mature is your third-party AI resource management?71–5
MG-4Monitoring, Recovery & Communication
3 subcategories· 4 governance questions
| Reference | Subcategory | Coverage |
|---|---|---|
| MANAGE 4.1 | Post-deployment monitoring plan | Governance Assessment |
| MANAGE 4.2 | Incident capture mechanisms | Governance Assessment |
| MANAGE 4.3 | Risk evaluation communicated | Governance Assessment |
Governance Questions
- Is a post-deployment monitoring plan in place, including mechanisms for system performance and anomaly detection per MANAGE 4.1?8Y/N
- Are mechanisms in place for capturing and responding to AI system incidents, errors, and user complaints per MANAGE 4.2?8Y/N
- Are AI risk evaluation results and incident information communicated to relevant stakeholders per MANAGE 4.3?7Y/N
- How mature is your post-deployment monitoring, incident response, and stakeholder communication?81–5
NIST AI 600-1 Generative AI Profile
The NIST AI 600-1 Generative AI Profile (July 2024) identifies 12 GAI-specific risk categories. The table below shows how Probe Six's assessment maps to each risk category.
| GAI Risk | Probe Six Coverage | Subcategory |
|---|---|---|
| CBRN Information | harmful:chemical-biological-weapons + MS 2.4 governance | MS 2.4 |
| Confabulation | hallucination, factual-accuracy, confidence-calibration | MS 2.3, 2.5 |
| Dangerous Content | 13 harmful:* plugins covering all dangerous content variants | MS 2.4 |
| Data Privacy | pii:*, cross-session-leak, training-data-extraction | MS 2.10 |
| Environmental Impacts | Governance question only (no automated probes) | MS 2.12 |
| Harmful Bias | 9 bias:* plugins across protected characteristics | MS 2.11 |
| Human-AI Configuration | overreliance, sycophancy + governance questions | MS 2.3 |
| Information Integrity | factual-accuracy, citation-verification, confidence-calibration | MS 2.5 |
| Information Security | 15 security plugins (injection, access control, exfiltration) | MS 2.7 |
| Intellectual Property | contracts plugin + governance questions | MS 2.3 |
| Obscene/Abusive Content | graphic-content, profanity plugins | MS 2.4 |
| Value Chain & Component Integration | Governance questions across GV-6, MP-4, MG-3 | GV-6, MP-4, MG-3 |
Out-of-Scope Items
The following items from the NIST AI RMF ecosystem are not included in the Probe Six assessment because they represent customisation guidance rather than assessable requirements.
| Item | Rationale |
|---|---|
| AI RMF Profiles | Profiles are organisation-specific customisations of the framework — they describe how an organisation applies the AI RMF to its context, not assessable requirements. |
| Playbook activities | The AI RMF Playbook provides suggested actions for each subcategory — these are implementation guidance, not assessable controls. |
Running a NIST AI RMF Assessment
To run a NIST AI Risk Management Framework assessment:
- Register your endpoint— Add the AI system you want to assess via the Endpoints page
- Select the NIST AI RMF template— Choose individual categories for targeted testing or select all 19 for comprehensive coverage
- Complete governance questions— When you select a category, its governance questions appear inline below the category row. Answer them in context — your responses auto-save and persist across scans
- Review category-level results— Each finding in your report includes exact NIST subcategory references alongside results from automated testing and governance assessment
The assessment produces a per-category compliance view grouped by NIST function, showing which subcategories were tested, pass rates, and severity levels. Governance-only categories (GOVERN, MAP, and most MANAGE categories) appear with governance assessment results only.
Note:This assessment is a technical evaluation tool, not a compliance certification. Results should be reviewed alongside appropriate risk management expertise. The NIST AI RMF is a voluntary framework — the assessment helps identify gaps and provides evidence for your AI risk management documentation.
References
- NIST AI RMF 1.0 (AI 100-1) — AI Risk Management Framework, January 2023
- NIST AI RMF Playbook — Suggested actions for each subcategory
- NIST AI 600-1: Generative AI Profile — GAI-specific risk categories mapped to the AI RMF, July 2024
- NIST Artificial Intelligence — NIST AI programme homepage
- NIST AI Resource Centre (AIRC) — Hub for AI RMF resources, profiles, and crosswalks