Comparison
| Aspect | complior scan | complior eval |
|---|
| Analyzes | Source code (static) | Running system (dynamic) |
| Phase | Development, every commit | Pre-deployment |
| Speed | 2–30 seconds | 1–10 minutes |
| Requires | Just code | Live endpoint |
| Finds | Missing patterns, dependencies, secrets | Actual behavior, real bias, explanation quality |
| Cannot find | How system actually behaves | Code structure, internal patterns |
| Tests | 40+ AST patterns | 680 probes |
| Score | Compliance + Security (from code) | Conformity + Security (from behavior) |
Why you need both
| Scan sees… | Eval checks… |
|---|
| Disclosure code exists in source | Bot actually says “I am AI” in 5 languages |
| Kill-switch pattern in codebase | Kill-switch actually fires when triggered |
| Bias-check hook is imported | Real discrimination via 30 A/B paired tests |
| Logging calls written in code | Logs are actually written and PII-masked |
| Try/catch blocks exist | System actually recovers after errors |
| (Cannot check) | Explanation quality scored by LLM-judge |
| (Cannot check) | Hallucination rate via factual tests |
~60% of eval tests check data completely unavailable through static code analysis (CT-3 Explanation, CT-4 Bias A/B, CT-5 Accuracy, CT-9 Risk Awareness, CT-11 Industry-specific).
Recommended workflow
# During development (every commit):
complior scan
# Before deployment (before release):
complior eval --target <staging-url> --full
# Full audit package:
complior audit --scan . --target <url>
