117 lines
4.7 KiB
Markdown
117 lines
4.7 KiB
Markdown
# Compose Update Policy
|
|
|
|
Compose-backed services are deployed from role-owned Jinja templates. Vendored
|
|
upstream files are reference snapshots only and must not be deployed directly.
|
|
|
|
## Layout
|
|
|
|
Each Docker-backed role has:
|
|
|
|
- `templates/compose.yaml.j2`: deployed Compose template
|
|
- `templates/env.j2`: generated environment file
|
|
- `upstream/`: copied upstream reference files
|
|
|
|
`docs/upstream-sources.yml` is the authoritative inventory of upstream Git
|
|
repositories, README paths, Compose paths, fallback Compose URLs, local Compose
|
|
references, refs, and verification status.
|
|
|
|
Role entries support one local fallback:
|
|
|
|
- `local_compose_file`: repo-local Compose file to copy as role reference
|
|
material when upstream does not provide a useful Compose example.
|
|
|
|
Repository entries support three upstream Compose discovery sources:
|
|
|
|
- `compose_paths`: paths to Compose files inside the checked-out repository.
|
|
- `compose_urls`: direct `http` or `https` URLs that return Compose YAML.
|
|
- README fenced YAML blocks, parsed when a README is listed in `readme_paths`.
|
|
|
|
Repository entries that only use `compose_urls` may omit `url` and `ref`.
|
|
Entries that use `readme_paths` or `compose_paths` must include both `url` and
|
|
`ref` so the script can clone the source repository.
|
|
|
|
## Update Workflow
|
|
|
|
1. Read `docs/upstream-sources.yml` for the role being updated.
|
|
2. Test the upstream inventory without writing snapshots:
|
|
|
|
```sh
|
|
scripts/vendor-upstream-snapshots.py <role>
|
|
```
|
|
|
|
3. Fetch or copy the listed upstream README and Compose files into
|
|
`roles/<role>/upstream/`:
|
|
|
|
```sh
|
|
scripts/vendor-upstream-snapshots.py --apply <role>
|
|
```
|
|
|
|
The script also looks for Compose-shaped fenced YAML blocks in README files.
|
|
When exactly one candidate is found, `--apply` writes it as
|
|
`docker-compose.yml` inside that upstream purpose directory. Compose YAML
|
|
copied from `compose_paths` or fetched from `compose_urls` is also normalized
|
|
to `docker-compose.yml`. URL sources are fallback material and do not
|
|
overwrite a Compose file already staged from the repository or README parse.
|
|
A role-level `local_compose_file` is copied under `roles/<role>/upstream/local/`.
|
|
|
|
Repository snapshots are written under `roles/<role>/upstream/<purpose>/`.
|
|
On `--apply`, each `upstream/<purpose>/` directory is replaced with a fresh
|
|
snapshot for that source, so stale README and Compose files are removed.
|
|
The script displays a single-line progress bar on interactive terminals while
|
|
processing sources.
|
|
4. Update `roles/<role>/upstream/SOURCE.md` with source URL, ref, snapshot date,
|
|
fetched paths, and any manual verification notes.
|
|
5. Commit upstream snapshot changes separately from NBOPS deployment template
|
|
changes.
|
|
6. Diff upstream snapshots against current role templates.
|
|
7. Apply upstream structural changes to `templates/compose.yaml.j2`. If a
|
|
role-level `upstream/docker-compose.override.yml` exists, treat it as the
|
|
primary source for the template; otherwise use `upstream/docker-compose.yml`.
|
|
8. Keep NBOPS-local changes in the deployed template, including Traefik labels,
|
|
internal networks, host paths, ports, restart policy, logging, resource
|
|
limits, and deployment-specific environment wiring.
|
|
9. Replace mutable image tags with pinned versions where practical.
|
|
10. Document breaking changes or operator actions in the role README.
|
|
|
|
## Rules
|
|
|
|
- Do not edit generated Compose files on VMs.
|
|
- Do not deploy files from `upstream/` directly.
|
|
- Do not treat `status: verify` sources as trusted until manually checked.
|
|
- Treat `upstream/<purpose>/` as generated snapshot output; do not keep manual
|
|
edits there.
|
|
- To remove generated purpose snapshots and any template files explicitly marked
|
|
with `nbops: derived-from-upstream`, run:
|
|
|
|
```sh
|
|
scripts/vendor-upstream-snapshots.py --clean-upstreams <role>
|
|
```
|
|
|
|
This deletes directories like `upstream/application/` and
|
|
`upstream/container/`, but leaves files such as `upstream/SOURCE.md` and
|
|
root-level override YAMLs alone. The old `--clean-purposes` flag is retained
|
|
as a compatibility alias.
|
|
- To target a specific upstream purpose, add `--upstream`:
|
|
|
|
```sh
|
|
scripts/vendor-upstream-snapshots.py --clean-upstreams --upstream container <role>
|
|
```
|
|
|
|
`--upstream` may be repeated, or `--upstreams container,application` may be
|
|
used for comma-separated values.
|
|
- Prefer pinned tags or digests over `latest` and other mutable tags.
|
|
- Keep HTTP application ports unpublished in Traefik mode; document internal
|
|
service ports as comments in generated Compose.
|
|
|
|
## Validation
|
|
|
|
Run after every Compose template update:
|
|
|
|
```sh
|
|
ansible-playbook --syntax-check playbooks/site.yml
|
|
git diff --check
|
|
```
|
|
|
|
Later, add a render check that runs `docker compose config` against generated
|
|
`compose.yaml` plus `compose.override.yaml` for selected roles.
|