Evidence Pack
This folder is for dissertation evidence that can be regenerated on demand.
Quick Start
Run from repo root:
bash scripts/collect_evidence.sh
Optional custom run id:
bash scripts/collect_evidence.sh 2026-04-16-baseline
Outputs are written to evidence/runs/<run-id>/.
To autofill templates for an existing run:
bash scripts/autofill_evidence_templates.sh <run-id>
To capture UI screenshots for an existing run:
bash scripts/capture_ui_screenshots.sh <run-id>
To include screenshot capture during evidence collection:
KEYWARDEN_EVIDENCE_CAPTURE_SCREENSHOTS=1 bash scripts/collect_evidence.sh <run-id>
What Is Collected
- Git and environment snapshot (
meta/) - Docker Compose state and container runtime info (
runtime/) - Django test execution output (
tests/) - Coverage summary + XML when available (
tests/coverage-report.txt,tests/coverage-xml.xml) - Log extracts and endpoint/status summaries (
logs/) - Derived operational metrics and summary (
evaluation/) - RBAC/role-claim grep evidence (
docs/) - Copy of diagram sources and report templates (
diagrams/,templates/) - Autofilled template versions in each run (
templates/*.md) - Optional automated UI screenshots (
screenshots/)
Operational Metrics
The collector now derives evaluation-friendly metrics from agent-access logs:
- heartbeat intervals per server
- endpoint/status distributions per server
- heartbeat error-to-recovery timings
- markdown summary for direct report inclusion
Artifacts are written to evidence/runs/<run-id>/evaluation/.
Diagram Sources
Mermaid sources live in evidence/diagrams/:
architecture.mmddeployment.mmderd.mmdaccess-sync-flow.mmd
These are plain text and version-controlled, so they are easy to revise and re-render.
Manual Evidence Templates
Templates for remaining manual work are in evidence/templates/:
- browser test matrix
- security test report
- standards checklist (OWASP/NIST-oriented)