# Compose Update Policy

Compose-backed services are deployed from role-owned Jinja templates. Vendored
upstream files are reference snapshots only and must not be deployed directly.

## Layout

Each Docker-backed role has:

- `templates/compose.yaml.j2`: deployed Compose template
- `templates/env.j2`: generated environment file
- `upstream/`: copied upstream reference files

`docs/upstream-sources.yml` is the authoritative inventory of upstream Git
repositories, README paths, Compose paths, fallback Compose URLs, local Compose
references, refs, and verification status.

Role entries support one local fallback:

- `local_compose_file`: repo-local Compose file to copy as role reference
  material when upstream does not provide a useful Compose example.

Repository entries support three upstream Compose discovery sources:

- `compose_paths`: paths to Compose files inside the checked-out repository.
- `compose_urls`: direct `http` or `https` URLs that return Compose YAML.
- README fenced YAML blocks, parsed when a README is listed in `readme_paths`.

Repository entries that only use `compose_urls` may omit `url` and `ref`.
Entries that use `readme_paths` or `compose_paths` must include both `url` and
`ref` so the script can clone the source repository.

## Update Workflow

1. Read `docs/upstream-sources.yml` for the role being updated.
2. Test the upstream inventory without writing snapshots:

   ```sh
   scripts/vendor-upstream-snapshots.py <role>
   ```

3. Fetch or copy the listed upstream README and Compose files into
   `roles/<role>/upstream/`:

   ```sh
   scripts/vendor-upstream-snapshots.py --apply <role>
   ```

   The script also looks for Compose-shaped fenced YAML blocks in README files.
   When exactly one candidate is found, `--apply` writes it as
   `docker-compose.yml` inside that upstream purpose directory. Compose YAML
   copied from `compose_paths` or fetched from `compose_urls` is also normalized
   to `docker-compose.yml`. URL sources are fallback material and do not
   overwrite a Compose file already staged from the repository or README parse.
   A role-level `local_compose_file` is copied under `roles/<role>/upstream/local/`.

   Repository snapshots are written under `roles/<role>/upstream/<purpose>/`.
   On `--apply`, each `upstream/<purpose>/` directory is replaced with a fresh
   snapshot for that source, so stale README and Compose files are removed.
   The script displays a single-line progress bar on interactive terminals while
   processing sources.
4. Update `roles/<role>/upstream/SOURCE.md` with source URL, ref, snapshot date,
   fetched paths, and any manual verification notes.
5. Commit upstream snapshot changes separately from NBOPS deployment template
   changes.
6. Diff upstream snapshots against current role templates.
7. Apply upstream structural changes to `templates/compose.yaml.j2`. If a
   role-level `upstream/docker-compose.override.yml` exists, treat it as the
   primary source for the template; otherwise use `upstream/docker-compose.yml`.
8. Keep NBOPS-local changes in the deployed template, including Traefik labels,
   internal networks, host paths, ports, restart policy, logging, resource
   limits, and deployment-specific environment wiring.
9. Replace mutable image tags with pinned versions where practical.
10. Document breaking changes or operator actions in the role README.

## Rules

- Do not edit generated Compose files on VMs.
- Do not deploy files from `upstream/` directly.
- Do not treat `status: verify` sources as trusted until manually checked.
- Treat `upstream/<purpose>/` as generated snapshot output; do not keep manual
  edits there.
- To remove generated purpose snapshots and any template files explicitly marked
  with `nbops: derived-from-upstream`, run:

  ```sh
  scripts/vendor-upstream-snapshots.py --clean-upstreams <role>
  ```

  This deletes directories like `upstream/application/` and
  `upstream/container/`, but leaves files such as `upstream/SOURCE.md` and
  root-level override YAMLs alone. The old `--clean-purposes` flag is retained
  as a compatibility alias.
- To target a specific upstream purpose, add `--upstream`:

  ```sh
  scripts/vendor-upstream-snapshots.py --clean-upstreams --upstream container <role>
  ```

  `--upstream` may be repeated, or `--upstreams container,application` may be
  used for comma-separated values.
- Prefer pinned tags or digests over `latest` and other mutable tags.
- Keep HTTP application ports unpublished in Traefik mode; document internal
  service ports as comments in generated Compose.

## Validation

Run after every Compose template update:

```sh
ansible-playbook --syntax-check playbooks/site.yml
git diff --check
```

Later, add a render check that runs `docker compose config` against generated
`compose.yaml` plus `compose.override.yaml` for selected roles.