implemented compose by url, parse, local yaml.

docs/compose-update-policy.md

@@ -0,0 +1,87 @@
+# Compose Update Policy
+
+Compose-backed services are deployed from role-owned Jinja templates. Vendored
+upstream files are reference snapshots only and must not be deployed directly.
+
+## Layout
+
+Each Docker-backed role has:
+
+- `templates/compose.yaml.j2`: base deployed Compose file
+- `templates/compose.override.yaml.j2`: NBOPS-specific local override
+- `templates/env.j2`: generated environment file
+- `upstream/`: copied upstream reference files
+
+`docs/upstream-sources.yml` is the authoritative inventory of upstream Git
+repositories, README paths, Compose paths, fallback Compose URLs, local Compose
+references, refs, and verification status.
+
+Role entries support one local fallback:
+
+- `local_compose_file`: repo-local Compose file to copy as role reference
+  material when upstream does not provide a useful Compose example.
+
+Repository entries support three upstream Compose discovery sources:
+
+- `compose_paths`: paths to Compose files inside the checked-out repository.
+- `compose_urls`: direct `http` or `https` URLs that return Compose YAML.
+- README fenced YAML blocks, parsed when a README is listed in `readme_paths`.
+
+Repository entries that only use `compose_urls` may omit `url` and `ref`.
+Entries that use `readme_paths` or `compose_paths` must include both `url` and
+`ref` so the script can clone the source repository.
+
+## Update Workflow
+
+1. Read `docs/upstream-sources.yml` for the role being updated.
+2. Test the upstream inventory without writing snapshots:
+
+   ```sh
+   scripts/vendor-upstream-snapshots.py <role>
+   ```
+
+3. Fetch or copy the listed upstream README and Compose files into
+   `roles/<role>/upstream/`:
+
+   ```sh
+   scripts/vendor-upstream-snapshots.py --apply <role>
+   ```
+
+   The script also looks for Compose-shaped fenced YAML blocks in README files.
+   When exactly one candidate is found, `--apply` writes it as
+   `compose.from-readme.yaml`. Compose YAML fetched from `compose_urls` is
+   written as `compose.from-url.yaml`, or `compose.from-url-N.yaml` when a
+   repository has multiple fallback URLs. A role-level `local_compose_file` is
+   copied under `roles/<role>/upstream/local/`.
+4. Update `roles/<role>/upstream/SOURCE.md` with source URL, ref, snapshot date,
+   fetched paths, and any manual verification notes.
+5. Commit upstream snapshot changes separately from NBOPS deployment template
+   changes.
+6. Diff upstream snapshots against current role templates.
+7. Apply upstream structural changes to `templates/compose.yaml.j2`.
+8. Keep NBOPS-local changes in `templates/compose.override.yaml.j2`, including:
+   Traefik labels, internal networks, host paths, ports, restart policy, logging,
+   resource limits, and deployment-specific environment wiring.
+9. Replace mutable image tags with pinned versions where practical.
+10. Document breaking changes or operator actions in the role README.
+
+## Rules
+
+- Do not edit generated Compose files on VMs.
+- Do not deploy files from `upstream/` directly.
+- Do not treat `status: verify` sources as trusted until manually checked.
+- Prefer pinned tags or digests over `latest` and other mutable tags.
+- Keep HTTP application ports unpublished in Traefik mode; document internal
+  service ports as comments in generated Compose.
+
+## Validation
+
+Run after every Compose template update:
+
+```sh
+ansible-playbook --syntax-check playbooks/site.yml
+git diff --check
+```
+
+Later, add a render check that runs `docker compose config` against generated
+`compose.yaml` plus `compose.override.yaml` for selected roles.