Skip to main content
Checksums Don't Prove Who Built It
  1. Posts/

Checksums Don't Prove Who Built It

William Collins
Author
William Collins
Building at the intersection of cloud, automation, and AI. Host of The Cloud Gambit podcast.
Table of Contents

For a long time, my definition of a successful software release was pretty simple: the tests passed, the binaries built, and the checksums matched. Ship it! That still sounds reasonable, but there is a gap hiding between we built some bytes and users downloaded the bytes we intended to publish.

I ran into that gap while changing the release process for gridctl. What I found interesting here was the idea of treating publication like a real security boundary rather than a last build step.

What a Checksum Actually Tells You
#

A checksum answers one useful question: does this file match the digest in the checksum file?

It does not tell you who created either file. If somebody replaces the archive and the checksum file, the math still works. I have shipped releases that stopped at that check because it felt finished.

Authenticated provenance adds those missing claims. For this release process, verification can constrain:

  • The GitHub repository
  • The workflow that built and signed the assets
  • The release tag and full source commit
  • The GitHub Actions identity provider
  • The exact bytes downloaded by the user

That is a much stronger answer than “the checksum matched.”

Note

Provenance doesn’t prove software is harmless, reproducible, or free of vulnerabilities. It proves that specific bytes came through a specific build identity and source context. Narrow claims are more useful than ambitious claims the pipeline cannot support.

Before: Build and Publish in One Motion
#

The original release workflow was easy to understand. A version tag triggered one job, GoReleaser built four archives, generated checksums, published the GitHub release, and updated Homebrew.

1
2
3
4
5
tag push
  -> build archives
  -> generate checksums
  -> publish release
  -> update Homebrew

Simple is normally something I fight to preserve. In this case, though, simplicity had collapsed several different trust decisions into one privileged job.

The workflow did not rerun every required test and security gate against the exact tagged commit. GoReleaser was selected with latest, several GitHub Actions used mutable version tags, and the release became public while it was still being assembled. Homebrew could advance without independently authenticating the public release bytes.

There was also no safe checkpoint between building and publishing. If the assets were incomplete or the source identity was wrong, they were already public. Replacing a published archive later would create an even worse trust problem.

The old process was not unusual. It looked like many release pipelines that grow organically: add a build command, give it permission to publish, and skip into fields of green. I have built pipelines this way myself because it works right up until your definition of “works” becomes a tad more demanding.

Now: Assemble, Verify, Then Publish
#

The new release workflow makes publication conditional on a sequence of smaller decisions:

The release workflow validates the source, assembles a draft, verifies it on Linux and macOS, publishes it, and then updates Homebrew

Let’s not stand on ceremony; this is about the machinery. The benefit is that each job has one clear responsibility, narrower permissions, and an artifact it can inspect independently.

Gate the Source You Are Releasing
#

The release starts by calling the same Gatekeeper workflow used elsewhere in the repository. Its final job runs with if: always() and checks that every required result is exactly success. Failed, canceled, missing, or skipped gates stop the release.

That detail matters. A green pull request from yesterday might cover a different merge commit. A release should validate the exact source it claims to distribute, not borrow confidence from a nearby commit that happened to pass.

Tags introduce another small trap. An annotated Git tag points to a tag object, which then points to a commit. Comparing raw object IDs can reject a valid tag or bind a release to the wrong object. The new workflow explicitly peels HEAD, the event SHA, and the tag reference to commits, then requires all three to equal the source SHA returned by validation.

Sometimes the most expensive security bugs begin with two identifiers that look interchangeable until they aren’t.

Sign the Final Assets
#

GoReleaser now builds a draft release instead of publishing immediately. The pipeline generates four platform archives, SPDX inventory documents, a release inventory, the Homebrew cask, and checksums. It then creates a downloadable Sigstore provenance bundle for the final subject set.

If you sign a binary and compress or repackage it afterward, users download different bytes than the ones you authenticated. The archive is the distribution unit, so the archive must be the subject.

There is also a circular dependency to avoid. A signature bundle cannot authenticate itself, and a checksum file cannot include its own checksum. The process handles this in layers:

  1. checksums.txt covers the data assets.
  2. The attestation covers those assets plus checksums.txt.
  3. provenance.sigstore.json is generated last and excluded from its own subject set.

The order is part of the design.

Read the Bytes Back
#

After upload, the assembly job downloads the completed draft again. Separate Linux and macOS jobs verify every expected subject against the repository, workflow identity, tag, source SHA, issuer, and provenance type.

Here is the actual gh attestation verify policy used by those jobs:

1
2
3
4
5
6
7
8
9
gh attestation verify "$archive" \
  --bundle provenance.sigstore.json \
  --repo gridctl/gridctl \
  --cert-identity "https://github.com/gridctl/gridctl/.github/workflows/release.yaml@refs/tags/$tag" \
  --source-ref "refs/tags/$tag" \
  --source-digest "$sha" \
  --cert-oidc-issuer https://token.actions.githubusercontent.com \
  --predicate-type https://slsa.dev/provenance/v1 \
  --deny-self-hosted-runners
A successful release provenance check matching the expected repository, workflow, tag, and source commit

That is the check I want a user to be able to run without a GitHub login. v1.0.0-rc.1 is the first release covered by this policy. Its notes identify the full source commit, and its assets include provenance.sigstore.json.

The installer still checks checksums.txt, not the full provenance policy. I would rather say that plainly than pretend curl | sh grew a supply-chain conscience overnight. The release verification guide shows how to download, authenticate, and install the same local bytes.

The verification tests also change one policy input at a time and require rejection. They try the wrong repository, workflow, tag, commit, issuer, and predicate type, then tamper with an archive. This is one of my favorite parts of the design because success tests only prove that the happy path works. Security policy also needs evidence that the wrong thing fails for the right reason.

Tip

Treat a timeout, login prompt, or empty diagnostic as an infrastructure failure. It is not proof that invalid provenance was rejected. Cryptographic rejection and “the verifier fell over” are very different outcomes.

The publisher repeats the checks against the current draft, publishes it, downloads each asset from its public URL, and verifies those bytes again. Only then does the Homebrew job authenticate the public release and advance the cask.

This repetition is intentional. A successful local build doesn’t prove the upload worked. A successful upload doesn’t prove the public download serves the same file. Each boundary gets checked using the bytes available on the other side.

Make Publishing Boring
#

To me, the most useful change is moving publication to the end.

A draft is just a pile of files nobody should install yet. We can pull those files back down, check them on Linux and macOS, and delete the whole thing if the identity is wrong. Users never see a half-built release, and Homebrew doesn’t move until the public URL serves the same bytes.

There are tradeoffs. The release is slower, network-heavy, and more dependent on pinned verification tooling. Hosted runner labels still do not identify an immutable machine image. Local bundle verification may need network access for trust roots, and existing installers still enforce checksums rather than the full provenance policy.

Those limitations are documented because trust tends to disappear when claims outrun implementation. The goal was not to slap a supply-chain badge on a release page. The goal was to make a precise statement about where the final bytes came from and have the pipeline prove that statement before publishing them.

Where I Landed
#

I still prefer simple release pipelines. I just have a stricter definition of simple now.

One job that builds, signs, publishes, and updates a package manager may be short, but it hides every trust boundary inside a single green box. A slightly longer graph can be easier to reason about because validation, assembly, verification, publication, and distribution are separate decisions.

Checksums still matter. Tests still matter. I just don’t want the last step in the pipeline to be the first time we ask whether the file on the other side of the download is the one we built.

Related

Stop Giving Your AI Agents Root Shell

Here’s a question I got asked recently: If a skill can already call a REST API using Bash, why bother with MCP? The surface-level answer is “MCP is cleaner.” That’s not wrong, but it undersells what’s actually different - and I think it’s a genuinely useful distinction to understand if you’re serious about building reliable agent workflows. Also, common-sense needs a resurgence given the massive amount of all old things are DEAD when new thing comes out clickbait that is proliferating on LinkedIn.

Vibe Coding Got Us Here. Can Spec-Driven Development Save Us?

Let me paint you a picture. It’s 2025. You’ve discovered that you can describe a feature in plain English and an LLM will just… build it. The dopamine hit rivals or even eclipses social media. You feel as if you’re shipping things in an afternoon that used to take a week. You’re not reading diffs. You’re not understanding the internals. You’re just vibing - and it feels amazing.