ntbops-ansible/docs/compose-update-policy.md

Compose Update Policy

Compose-backed services are deployed from role-owned Jinja templates. Vendored upstream files are reference snapshots only and must not be deployed directly.

Layout

Each Docker-backed role has:

• templates/compose.yaml.j2: base deployed Compose file
• templates/compose.override.yaml.j2: NBOPS-specific local override
• templates/env.j2: generated environment file
• upstream/: copied upstream reference files

docs/upstream-sources.yml is the authoritative inventory of upstream Git repositories, README paths, Compose paths, fallback Compose URLs, local Compose references, refs, and verification status.

Role entries support one local fallback:

• local_compose_file: repo-local Compose file to copy as role reference material when upstream does not provide a useful Compose example.

Repository entries support three upstream Compose discovery sources:

• compose_paths: paths to Compose files inside the checked-out repository.
• compose_urls: direct http or https URLs that return Compose YAML.
• README fenced YAML blocks, parsed when a README is listed in readme_paths.

Repository entries that only use compose_urls may omit url and ref. Entries that use readme_paths or compose_paths must include both url and ref so the script can clone the source repository.

Update Workflow

1. Read docs/upstream-sources.yml for the role being updated.

2. Test the upstream inventory without writing snapshots:

   scripts/vendor-upstream-snapshots.py <role>
   

3. Fetch or copy the listed upstream README and Compose files into roles/<role>/upstream/:

   scripts/vendor-upstream-snapshots.py --apply <role>
   

   The script also looks for Compose-shaped fenced YAML blocks in README files. When exactly one candidate is found, --apply writes it as compose.from-readme.yaml. Compose YAML fetched from compose_urls is written as compose.from-url.yaml, or compose.from-url-N.yaml when a repository has multiple fallback URLs. A role-level local_compose_file is copied under roles/<role>/upstream/local/.

4. Update roles/<role>/upstream/SOURCE.md with source URL, ref, snapshot date, fetched paths, and any manual verification notes.

5. Commit upstream snapshot changes separately from NBOPS deployment template changes.

6. Diff upstream snapshots against current role templates.

7. Apply upstream structural changes to templates/compose.yaml.j2.

8. Keep NBOPS-local changes in templates/compose.override.yaml.j2, including: Traefik labels, internal networks, host paths, ports, restart policy, logging, resource limits, and deployment-specific environment wiring.

9. Replace mutable image tags with pinned versions where practical.

10. Document breaking changes or operator actions in the role README.

Rules

• Do not edit generated Compose files on VMs.
• Do not deploy files from upstream/ directly.
• Do not treat status: verify sources as trusted until manually checked.
• Prefer pinned tags or digests over latest and other mutable tags.
• Keep HTTP application ports unpublished in Traefik mode; document internal service ports as comments in generated Compose.

Validation

Run after every Compose template update:

ansible-playbook --syntax-check playbooks/site.yml
git diff --check

Later, add a render check that runs docker compose config against generated compose.yaml plus compose.override.yaml for selected roles.