> ## Documentation Index
> Fetch the complete documentation index at: https://docs.complior.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Eval Overview

> Dynamic AI system testing — 680 probes against live endpoints. Tests actual behavior, not just code.

`complior eval` tests a **running** AI system. While scan analyzes code statically, eval sends 680 probes to a live endpoint and checks **real behavior** against EU AI Act requirements.

```bash theme={null}
complior eval --target http://localhost:3000/api/chat
```

```
scan  = analyzes CODE        (static, offline, development)
eval  = tests SYSTEM         (dynamic, live endpoint, pre-deploy)
```

## How it works

<Steps>
  <Step title="Configure target">
    Point eval at your AI system's endpoint.

    ```bash theme={null}
    complior eval --target http://localhost:3000/api/chat
    ```
  </Step>

  <Step title="Probes are sent">
    680 test probes sent to your endpoint across 3 phases: deterministic (168), LLM-judged (212), security attacks (300).
  </Step>

  <Step title="Responses evaluated">
    Each response scored against EU AI Act criteria. Bias measured via A/B paired tests. Security via attack success rate.
  </Step>

  <Step title="Results recorded">
    Conformity Score + Security Score. Per-category breakdown. Critical gaps identified. Evidence chain updated.
  </Step>
</Steps>

## Target adapters

<Tabs>
  <Tab title="Generic HTTP">
    ```bash theme={null}
    complior eval --target http://localhost:3000/api/chat
    ```

    Sends `POST { "message": "{{probe}}" }`, reads response body as text.
  </Tab>

  <Tab title="OpenAI">
    ```bash theme={null}
    complior eval --target openai://api.openai.com --model gpt-4o --api-key sk-...
    ```

    Uses OpenAI chat completions format.
  </Tab>

  <Tab title="Anthropic">
    ```bash theme={null}
    complior eval --target anthropic://api.anthropic.com --model claude-3.5 --api-key sk-ant-...
    ```

    Uses Anthropic messages format.
  </Tab>

  <Tab title="Ollama">
    ```bash theme={null}
    complior eval --target ollama://localhost:11434 --model llama3
    ```

    Uses Ollama chat format. Fully local.
  </Tab>
</Tabs>

## Example output

<img src="https://mintcdn.com/comp-37a558ac/RC-6loB9wXYYyh5B/images/eval-demo.gif?s=f9416577bf95037f323a0266b77956b8" alt="complior eval --det running 176 conformity tests against a live endpoint" style={{ width: '100%', borderRadius: '8px' }} width="1200" height="993" data-path="images/eval-demo.gif" />

Tests scroll in real-time with colored PASS/FAIL indicators. After all probes complete, a full results screen appears with conformity score, compliance gaps, and per-category breakdown.

## Passport integration

With `--agent`, eval results are written directly into the passport:

```bash theme={null}
complior eval --target http://localhost:3000/api/chat --agent order-processor
```

This populates the `compliance.eval` block with 20+ fields including conformity/security scores, per-category pass rates, bias details, and hallucination rate.

<CardGroup cols={2}>
  <Card title="Eval Modes" icon="sliders" href="/eval/eval-modes">
    4 composable modes: deterministic, LLM-judged, security, full.
  </Card>

  <Card title="Conformity Tests" icon="clipboard-check" href="/eval/conformity-tests">
    11 EU AI Act categories, 370 deterministic + LLM-judged tests.
  </Card>
</CardGroup>