Version pinning and drift¶

A documentation build has more inputs than its own source, which creates dependency drift when declared or installed versions diverge. zensical renders the site, weasyprint lays out the PDF, and both are ordinary Python packages that resolve to whatever is newest unless you say otherwise.

That matters more than it first appears, because the way an upgrade shows up here is not a failed build. It is a published document that quietly differs from the one you reviewed.

The prodockit pins command supports two halves that only work together:

1. Pin the build inputs, so output changes when someone decides it should.
2. Watch for newer releases, so pinning does not mean going quietly stale.

Maintain a dependency safely¶

1. Check that the repository agrees with itself

   prodockit pins --check --offline
   

   This does not contact PyPI. It answers the pull-request question: do the version declarations already present in pyproject.toml and the workflows agree? It is the form used by ci.yml, because a new upstream release should not make an unrelated pull request fail.

2. Review drift information

   The scheduled drift.yml workflow asks the separate maintenance question: are newer releases available, and would they change the website or PDF? Read the issue it opens or run the workflow manually before selecting versions.

   For a quick inventory from a terminal:

   prodockit pins
   

   This contacts PyPI and offers the newest version, but it does not compare rendered artifacts for you.

3. Move every declaration together

   Set an reviewed version explicitly:

   prodockit pins --set zensical=0.0.55
   

   The tool preserves the role of each declaration: a library floor remains a floor and a publishing workflow's exact pin remains exact.

4. Rebuild in publishing order

   prodockit pdf
   zensical build --clean --strict
   pytest
   

   Compare the website and PDF with the pinned baseline recorded by the drift issue. Review pagination, generated indexes, diagrams, code blocks, and icons—not only whether the commands returned zero.

5. Confirm consistency before committing

   prodockit pins --check --offline
   git diff --check
   git status --short
   

   The final offline check prevents a partial version bump from reaching the pull request. The pull request then runs the same consistency gate in ci.yml.

Where a version gets declared¶

Pinning creates its own problem: the same version ends up written in several files at once, and nothing keeps them in step.

Both forms are correct in their own place. What is not correct is them disagreeing, which is easy to do by hand and invisible when it happens.

prodockit pins¶

Reads every declaration across all of those files, shows what is currently set against what is newest on PyPI, and moves them together.

prodockit pins

zensical
  pyproject.toml:34  zensical>=0.0.53
  .github/workflows/docs.yml:164  zensical==0.0.53
  .github/workflows/drift.yml:69  zensical==0.0.53
  newest on PyPI: 0.0.53  <- newer available

weasyprint
  .github/workflows/ci.yml:59  weasyprint==69.0
  .github/workflows/docs.yml:164  weasyprint==69.0
  newest on PyPI: 69.0

markdown
  pyproject.toml:25  markdown>=3.10.3
  .github/workflows/docs.yml:164  markdown==3.10.3
  .github/workflows/drift.yml:69  markdown==3.10.3
  newest on PyPI: 3.10.3

pymdown-extensions
  .github/workflows/docs.yml:164  pymdown-extensions==11.0.1
  .github/workflows/drift.yml:69  pymdown-extensions==11.0.1
  newest on PyPI: 11.0.1

zensical: version to set [0.0.53]:

Markdown and pymdown-extensions are in that list even though nothing installs them directly - they arrive under Zensical, which declares only floors for them. See the limitations below for why pinning Zensical alone left them free to move.

Press ++enter++ to take the newest release, or type a version. Each site keeps its own operator - the floor stays a floor, the pins stay pinned - so one answer updates every file correctly.

Options¶

--set is the unattended form, so it suppresses the prompt for every package, not only the one it names:

prodockit pins --set zensical=0.0.53

  pyproject.toml:34  zensical>=0.0.52 -> zensical>=0.0.53
  .github/workflows/docs.yml:136  zensical==0.0.52 -> zensical==0.0.53

Left untouched (no version given): weasyprint

Updated 2 declaration(s). Rebuild and diff before committing.

A package it was not given is reported and its files are not opened - name it with its own --set to move it, or leave it where it is. That is what makes the command safe in a script: it can neither hang on a prompt nor half-finish because one appeared.

--check is the one to put in CI:

prodockit pins --check --offline

It fails when a package is behind PyPI or when the files disagree with each other - the second being the failure that pinning across several files invites.

What it scans¶

Both CI hosts, so the same command works either way:

• pyproject.toml, setup.cfg
• .github/workflows/*.yml — GitHub Actions
• .gitlab-ci.yml and .gitlab/**/*.yml — GitLab CI
• requirements*.txt, constraints*.txt at the project root

Build output and virtualenvs (site/, public/, .venv/, node_modules/, …) are skipped, so a stale copy of a workflow inside one is not mistaken for a declaration.

Four shapes of declaration are recognised, because a build input is not always a pip package:

Pandoc version drift¶

Distribution Pandoc packages can lag several major versions behind upstream. Ubuntu 24.04, for example, supplies an older release than the one this repository currently tests and publishes with. Installing the distribution package locally while CI downloads a current release means the two builds can parse identical HTML differently.

That happened here when one Pandoc release accepted highlighted <pre><code> content that a newer release interpreted differently. Every fenced code block reflowed as ordinary justified prose in one environment, while the other environment continued publishing a correct PDF. Both builds reported success.

Pin Pandoc as a PANDOC_VERSION workflow variable and move it through prodockit pins, then compare complete PDF and website artifacts before and after the change. Pinning does not prevent incompatibility; it makes the change arrive in a reviewed commit instead of with an unannounced runner update.

Watching for drift¶

Pinning trades one risk for another: nothing tells you a newer release exists, or what it would do to your output.

The check worth running is not "is there a new version" - PyPI can answer that - but "would taking it change what we publish?" That needs a real build, twice.

The shape is the same on either host:

1. Build the docs with the pinned versions. Keep the result.
2. Upgrade to the newest and build again.
3. Diff the two, byte for byte.
4. Run your built-output checks against the newer build.
5. Report - do not fail.

Both builds run in the same job, so pandoc, Chrome, fonts and the runner image are identical between them and any difference is attributable to the upgraded packages alone.

Reporting, not failing¶

A newer release is information, not a broken build. A scheduled job that goes red every week trains everyone to ignore it, so open an issue instead - and keep one open at a time, updating it in place rather than filing a fresh one every Monday until somebody acts.

GitHub Actions¶

This project ships drift.yml doing exactly this. The essentials:

name: Dependency drift
on:
  schedule:
    - cron: "0 6 * * 1"   # Mondays, 06:00 UTC
  workflow_dispatch:
permissions:
  contents: read
  issues: write           # needed to open the issue
jobs:
  drift:
    runs-on: ubuntu-latest
    steps:
      # ... same build tooling as your docs job ...
      - name: Build with the pinned versions
        run: |
          pip install -e ".[testing]" "zensical==0.0.55" "weasyprint==69.0" "Markdown==3.10.3" "pymdown-extensions==11.0.1"
          prodockit pdf                      # PDF first ...
          zensical build --clean --strict    # ... then the site
          cp -R site /tmp/pinned-site
          cp docs/site_documentation.pdf /tmp/pinned.pdf

      - name: Build with the newest versions
        run: |
          pip install -qU zensical weasyprint
          prodockit pdf
          zensical build --clean --strict

      - name: Compare
        env:
          GH_TOKEN: ${{ github.token }}
        run: |
          cmp -s /tmp/pinned.pdf docs/site_documentation.pdf \
            && echo "PDF identical" || echo "PDF differs"
          diff -rq /tmp/pinned-site site | wc -l
          # ... then gh issue create / gh issue comment

GitLab CI¶

The same job, with GitLab's own scheduling and API. Add a pipeline schedule running weekly, and a project access token with the api scope exposed as DRIFT_TOKEN so the job can open an issue.

drift:
  image: python:3.13
  rules:
    # Only on the schedule - never on a normal push pipeline.
    - if: $CI_PIPELINE_SOURCE == "schedule"
  before_script:
    - apt-get update && apt-get install -y pandoc libpango-1.0-0 libpangoft2-1.0-0 libharfbuzz-subset0 jq curl
  script:
    - pip install -e ".[testing]" "zensical==0.0.55" "weasyprint==69.0" "Markdown==3.10.3" "pymdown-extensions==11.0.1"
    - prodockit pdf                     # PDF first ...
    - zensical build --clean --strict   # ... then the site
    - cp -R site /tmp/pinned-site && cp docs/site_documentation.pdf /tmp/pinned.pdf
    - pip install -qU zensical weasyprint
    - PINNED=$(pip show zensical | awk '/^Version:/{print $2}')
    - prodockit pdf
    - zensical build --clean --strict
    - LATEST=$(pip show zensical | awk '/^Version:/{print $2}')
    - |
      if [ "$PINNED" = "$LATEST" ]; then
        echo "Pins are current."; exit 0
      fi
      CHANGED=$(diff -rq /tmp/pinned-site site | wc -l)
      cmp -s /tmp/pinned.pdf docs/site_documentation.pdf && PDF=identical || PDF=differs
      # One open issue at a time.
      EXISTING=$(curl -sf --header "PRIVATE-TOKEN: $DRIFT_TOKEN" \
        "$CI_API_V4_URL/projects/$CI_PROJECT_ID/issues?state=opened&search=Dependency+drift" \
        | jq -r '.[0].iid // empty')
      BODY="zensical $PINNED -> $LATEST. PDF: $PDF. Website: $CHANGED files differ."
      if [ -n "$EXISTING" ]; then
        curl -sf --request POST --header "PRIVATE-TOKEN: $DRIFT_TOKEN" \
          --data-urlencode "body=$BODY" \
          "$CI_API_V4_URL/projects/$CI_PROJECT_ID/issues/$EXISTING/notes" > /dev/null
      else
        curl -sf --request POST --header "PRIVATE-TOKEN: $DRIFT_TOKEN" \
          --data-urlencode "title=Dependency drift: newer build inputs than the docs pins" \
          --data-urlencode "description=$BODY" \
          "$CI_API_V4_URL/projects/$CI_PROJECT_ID/issues" > /dev/null
      fi
  allow_failure: true

allow_failure: true keeps the pipeline green: the job's purpose is the issue it opens, not its own exit status.

The input pip cannot reach¶

prodockit pins manages Python packages. The CI runner image is the other half, and nothing in pyproject.toml or a workflow's pip install line touches it:

• pandoc comes from the image's own package archive. Distribution packages lag upstream far enough that some Markdown edge cases parse differently—see Pandoc version drift.
• Fonts the PDF embeds (fonts-inter, fonts-jetbrains-mono).
• Chrome, which rasterises Mermaid diagrams.

On runs-on: ubuntu-latest all three move the day the label migrates to a new LTS, with nothing committed - the same silent change the package pins exist to prevent, one layer down. Naming the image freezes them together:

jobs:
  deploy:
    runs-on: ubuntu-24.04   # not ubuntu-latest

GitLab's equivalent is the job image:, which most projects already pin by habit - python:3.13 rather than python:latest.

prodockit pins manages both, so the image is inventoried and moved the same way as everything else - name it with -p:

prodockit pins -p ubuntu       # runs-on: ubuntu-24.04, across every workflow
prodockit pins -p python       # image: python:3.13, in GitLab CI

ubuntu
  .github/workflows/ci.yml:11  ubuntu-24.04
  .github/workflows/docs.yml:69  ubuntu-24.04
  .github/workflows/drift.yml:36  ubuntu-24.04
  .github/workflows/publish.yml:10  ubuntu-24.04
  not on PyPI - set the version yourself

ubuntu: version to set [24.04]:

There is no suggested version for these: PyPI has nothing to say about a runner image, and asking it for "ubuntu" would at best miss and at worst find an unrelated package of that name and propose a nonsense upgrade. The default is what is currently set, so ++enter++ is a no-op and you type the new one deliberately - which suits a change you make once every couple of years.

--check still applies, and catches the failure that matters here: some jobs left on the old image after a partial migration.

Taking an upgrade¶

When drift reports something worth having, use the complete maintenance flow above. The short command sequence is:

prodockit pins --set zensical=0.0.55
prodockit pdf
zensical build --clean --strict
pytest
prodockit pins --check --offline

Substitute the package and reviewed version reported by drift. Then diff the built output against the previous version before committing. That is the step the whole arrangement exists to make possible: seeing what an upgrade does to your document before your readers do.

Limitations and workarounds¶

See Implementation limitations for the general list. Specific to pinning:

• A floor still floats. zensical>=0.0.55 in pyproject.toml records a version; it does not control one. Only the exact pin in the build job does. Both exist deliberately - see the table above.
• A pinned package's own dependencies float too, which is the sharper version of the same trap: pinning a direct dependency exactly does nothing for the transitive ones underneath it, because their versions come from floors in its metadata. Zensical is pinned exactly here, and still declares only floors for Markdown and pymdown-extensions - the two packages that actually turn every page into HTML. A build pinning Zensical alone therefore rendered with whatever those two resolved to on the morning it ran. Both are now pinned alongside it, and both are managed by prodockit pins (#178). If your project pins something whose rendering you depend on, check what it pulls in: pip show <package> lists its requirements, and a floor there is a version you are not controlling.
• Only pip packages are watched. pandoc and Chrome arrive from the runner image, so a drift job that installs both builds in one job cannot see them change between weeks. Pinning the image is the lever for those - see below.
• Comments are not scanned. A version specifier written in prose - explaining why something is pinned, say - is deliberately not treated as a declaration, so it is neither reported nor rewritten by --set. Only the part of a line before a # is read (#184); a trailing comment after a real declaration still leaves that declaration findable.
• prodockit pins needs network for --latest and the suggested default. Use --offline to report what the files declare without asking PyPI.