honestly its been months, i found unstaged changes and forgot

docs/compose-update-policy.md

@@ -7,8 +7,7 @@ upstream files are reference snapshots only and must not be deployed directly.
 
 Each Docker-backed role has:
 
-- `templates/compose.yaml.j2`: base deployed Compose file
-- `templates/compose.override.yaml.j2`: NBOPS-specific local override
+- `templates/compose.yaml.j2`: deployed Compose template
 - `templates/env.j2`: generated environment file
 - `upstream/`: copied upstream reference files
 
@@ -49,19 +48,28 @@ Entries that use `readme_paths` or `compose_paths` must include both `url` and
 
    The script also looks for Compose-shaped fenced YAML blocks in README files.
    When exactly one candidate is found, `--apply` writes it as
-   `compose.from-readme.yaml`. Compose YAML fetched from `compose_urls` is
-   written as `compose.from-url.yaml`, or `compose.from-url-N.yaml` when a
-   repository has multiple fallback URLs. A role-level `local_compose_file` is
-   copied under `roles/<role>/upstream/local/`.
+   `docker-compose.yml` inside that upstream purpose directory. Compose YAML
+   copied from `compose_paths` or fetched from `compose_urls` is also normalized
+   to `docker-compose.yml`. URL sources are fallback material and do not
+   overwrite a Compose file already staged from the repository or README parse.
+   A role-level `local_compose_file` is copied under `roles/<role>/upstream/local/`.
+
+   Repository snapshots are written under `roles/<role>/upstream/<purpose>/`.
+   On `--apply`, each `upstream/<purpose>/` directory is replaced with a fresh
+   snapshot for that source, so stale README and Compose files are removed.
+   The script displays a single-line progress bar on interactive terminals while
+   processing sources.
 4. Update `roles/<role>/upstream/SOURCE.md` with source URL, ref, snapshot date,
    fetched paths, and any manual verification notes.
 5. Commit upstream snapshot changes separately from NBOPS deployment template
    changes.
 6. Diff upstream snapshots against current role templates.
-7. Apply upstream structural changes to `templates/compose.yaml.j2`.
-8. Keep NBOPS-local changes in `templates/compose.override.yaml.j2`, including:
-   Traefik labels, internal networks, host paths, ports, restart policy, logging,
-   resource limits, and deployment-specific environment wiring.
+7. Apply upstream structural changes to `templates/compose.yaml.j2`. If a
+   role-level `upstream/docker-compose.override.yml` exists, treat it as the
+   primary source for the template; otherwise use `upstream/docker-compose.yml`.
+8. Keep NBOPS-local changes in the deployed template, including Traefik labels,
+   internal networks, host paths, ports, restart policy, logging, resource
+   limits, and deployment-specific environment wiring.
 9. Replace mutable image tags with pinned versions where practical.
 10. Document breaking changes or operator actions in the role README.
 
@@ -70,6 +78,27 @@ Entries that use `readme_paths` or `compose_paths` must include both `url` and
 - Do not edit generated Compose files on VMs.
 - Do not deploy files from `upstream/` directly.
 - Do not treat `status: verify` sources as trusted until manually checked.
+- Treat `upstream/<purpose>/` as generated snapshot output; do not keep manual
+  edits there.
+- To remove generated purpose snapshots and any template files explicitly marked
+  with `nbops: derived-from-upstream`, run:
+
+  ```sh
+  scripts/vendor-upstream-snapshots.py --clean-upstreams <role>
+  ```
+
+  This deletes directories like `upstream/application/` and
+  `upstream/container/`, but leaves files such as `upstream/SOURCE.md` and
+  root-level override YAMLs alone. The old `--clean-purposes` flag is retained
+  as a compatibility alias.
+- To target a specific upstream purpose, add `--upstream`:
+
+  ```sh
+  scripts/vendor-upstream-snapshots.py --clean-upstreams --upstream container <role>
+  ```
+
+  `--upstream` may be repeated, or `--upstreams container,application` may be
+  used for comma-separated values.
 - Prefer pinned tags or digests over `latest` and other mutable tags.
 - Keep HTTP application ports unpublished in Traefik mode; document internal
   service ports as comments in generated Compose.