Agents Gone Awry on Postgres SBOMs: Start Over

If you were wondering, it’s the SBOM thing that I mentioned the other day.

Postgres Extensions in containers with full inventory, provenance and attestation. I’ve been using plenty of AI Agents to put this together. This blog is a little scattered (apologies) but as they say, I didn’t have time to write a short letter.

Here’s what I believe to be the exact structure of current CloudNativePG images:

I have a bunch of irons in the fire related to this project:

• A downstream fork of CloudNativePG/postgres-extensions-containers which can host a bunch of Open Source code which isn’t allowed by CNCF.
  • I track upstream build infra, design my changes to minimize merge conflicts
• A set of patches fixing issues I’ve found, which I’ve submitted upstream
  • Patches are applied on my fork so that I can get stuff working
  • Upstream often tweaks stuff, so then I need to deal with the merge
• This huge new feature – adding proper SBOMs – which is in a feature branch off my repo
  • Planning to submit this upstream
  • Stacked on top of my other fix PRs
• Another huge new feature – PGRX build support – which is stacked on this SBOM feature
  • Not submitted upstream, will only live in my fork
  • Still want to structure code to minimize merge conflicts from upstream

After working on this for like a week, I realized that the set of commands to validate the security and provenance info was going to be totally different for PGRX than for upstream.

I think this is too confusing to users. There needs to be one simple, consistent command to verify provenance and SBOM material. I didn’t have that at the beginning and the AI Agent enabled racing ahead with the code. I didn’t realize the issue until now.

One thing I had been focused on was not having a bunch of things to copy, if someone needs to mirror images to a private container registry. With that focus, this is what I had inadvertantly ended up with.

For Debian-based extensions:

On the PGRX side, builds were timing out because they ran under qemu emulation. (Which is fine if you’re just installing debian packages.) I refactored the PGRX build to use native GitHub Runners, so that it would use this new design:

But after that refactor, the Debian images would look the same as previously, while the PGRX images would now look like this:

Here’s how Codex summarized it for me:

---

The main issue is that now it’s a totally different set of commands for users to validate PGRX containers. Most users don’t know or care how I built the container. They just want to use a Postgres extension.

Well this was no good. Time to go back to the drawing board and start over on the design.

Docker has a neat feature where you can provide a custom SBOM generator. This approach didn’t get used in my original design.

So using the agents, I rebuilt everything to use this approach. I also included a plugin API which was designed to be used by my PGRX build process later. Codex was able to leverage the previous code for actually composing the SBOM when it did the refactor. So far, this re-design is looking far more elegant and clean.

Most of the logic ends up self-contained in the custom sbom-generator module. The required changes to the build pipelines were surprisingly minimal. After a lot of work, here’s what I ended up with:

This link will probably break once I’ve merged things and cleaned up working branches, but if anyone is curious: https://github.com/ardentperf/postgres-extensions-containers/tree/x-ai/ardentperf/cnpg-sbom-generator/sbom-generator

One thing that AI could not do: correctly tell me what the right design was. Because it relies on me to know the right questions to ask, telling it the goals & priorities of the design.

Next up: refactoring the PGRX branch on top of this one (without losing native build hosts), cleaning up docs & code here and submitting this PR upstream, continuing to chase my handful of other open PRs upstream.

In the meantime – if anyone is running CloudNativePG and you need extensions… the CNPG-Extensions Project is available for testing now! It’s got quite a bit more advanced automation than the upstream at this point. Renovate is fully automated so that when a new extension version drops, you should automatically see it here (and in the Image Catalogs) quickly.

PG-Cron, PG-Partman, PG-Hint-Plan, PG-Stat-KCache, PGSentinel, PLDebugger, PLProfiler, MySQL/MSSQL FDWs… and lots more!

https://github.com/cnpg-extensions/postgres-extensions-containers

And soon (once I merge the first big PR): with full detailed and attested SBOMs and Provenance.