Getting Started

Probe Six is an automated security assessment platform for Large Language Models (LLMs). It tests how your AI models behave under adversarial conditions, identifying vulnerabilities before they reach production.

The Problem

LLMs are being deployed in production without adequate security testing. Traditional security tools weren't designed for AI-specific risks like prompt injection, data leakage or bias. Probe Six fills that gap with automated, evidence-based assessments that map directly to compliance frameworks.

Quick Start

Endpoints

An endpoint represents the LLM you want to assess. Probe Six supports two connection methods: HTTP API endpoints and AWS Bedrock (via cross-account role assumption).

API Endpoint

Connect to any LLM that exposes an HTTP API. You provide the URL and any authentication headers needed.

AWS Bedrock

Connect to AWS Bedrock models in your AWS account using IAM cross-account role assumption. Probe Six assumes a role in your account to access the Bedrock model directly.

Setting up cross-account access

You need to create an IAM role in your AWS account that allows Probe Six to invoke your Bedrock models on your behalf. The steps below walk through the full setup.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::450328359362:role/probesix-cross-account-scanner"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "YOUR_EXTERNAL_ID"
        }
      }
    }
  ]
}

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:ListFoundationModels"
      ],
      "Resource": "*"
    }
  ]
}

Managing Endpoints

All endpoints are organised under projects. When you create your first endpoint, a default project is created automatically. You can view, edit and delete endpoints from the dashboard. Each endpoint shows its connection status (active, pending validation or invalid) and the date of its last scan.

Running Scans

A scan executes a suite of security tests against your LLM endpoint. Each scan uses a compliance template that determines which test categories are included.

Scan Templates

Choose a template based on the compliance framework or security standard you need to assess against:

How a Scan Works

Scan Phases

Every scan progresses through four phases:

Re-running Scans

Any completed scan can be re-run at any time using the same endpoint and template configuration. Re-runs let you track how your model's security posture changes over time — for example, after applying remediations or updating your model. Each re-run produces a new report, so you can compare results side by side.

What Affects Scan Duration

During the scanning phase, you may notice that some tests complete faster than others. This is normal and reflects the different types of security tests being run.

Direct Tests

The first portion of a scan consists of direct tests— individual adversarial prompts sent to your endpoint. Each test follows a simple cycle: send the prompt, receive the response, grade the result. These tests typically complete at a rate of 10–20 per minute.

Adaptive Tests

The later portion of a scan uses adaptive testing strategiessuch as jailbreak techniques, multi-step prompt injection, and iterative refinement attacks. These tests are more sophisticated — if the initial attempt doesn't bypass your model's defences, the test adapts its approach and tries again with a modified prompt. This multi-round process means each adaptive test requires several interactions with your endpoint, typically completing at 2–5 per minute.

Understanding Reports

After a scan completes, Probe Six generates a detailed assessment report. The report provides a complete picture of your model's security posture, combining automated test results with governance assessment responses.

Report Structure

Risk Scoring

Each finding receives a risk score from 0 to 10, derived from the attack success rate (ASR) — the percentage of tests where the model failed to resist the attack. The score incorporates:

• Attack impact — The potential damage if exploited (low to critical)
• Exploit difficulty — How difficult the attack is to execute
• Human oversight factor — Whether a human is in the loop
• Penalty modifiers — Adjustments for high ASR or specific risk patterns

Severity thresholds: ≥7.5 critical, ≥5.0 high, ≥2.5 medium, >0 low, =0none. The overall security score (0–100%) is calculated as 100 − 10 × WRI, where WRI is a weighted risk index across all categories (critical 4×, high 3×, medium 2×, low 1×).

PDF Export

Full reports can be exported as PDF documents for offline review, sharing with stakeholders, or inclusion in compliance documentation. The PDF includes all report sections — score cards, vulnerability breakdown, findings, framework coverage, remediation guidance and executive summary.

Free vs Full Reports

Finding Categories

Probe Six tests across a wide range of LLM security categories. Each category targets a specific class of vulnerability or risk. Below is a reference of the major categories currently assessed.

Prompt Injection & Evasion

Security Vulnerabilities

RAG & Agentic Security

Content Safety

Bias & Fairness

Data Protection & Privacy

EU AI Act Compliance

Accuracy & Robustness

Compliance Mapping

Every finding in a Probe Six assessment maps to one or more compliance frameworks. This mapping provides auditable evidence that your AI systems have been tested against recognised security and governance standards.

Supported Frameworks

How Mapping Works

When Probe Six generates remediation guidance for a finding, it includes a references section that maps the issue to specific codes and clauses within relevant frameworks. For example:

These references allow security teams and auditors to trace each finding back to the specific compliance requirement it addresses, providing the evidence trail needed for regulatory reporting and governance reviews.