Ardent Performance Computing
troubleshooting
Why Postgres Breaks Kubernetes container_memory_working_set_bytes Metric
Kubernetes metric container_memory_working_set_bytes is used for evicting/killing pods with too much memory use, especially if memory request < limit (don’t do this with Postgres). The metric is calculated from cgroups v2 memory.stat as current-inactive_file [source]. You’d assume it’s a good metric for memory usage in kubernetes. But with Postgres, this metric is very inaccurate for … Continue reading
Happiness Hint: Alarm on Checkpoint Time
Before starting, I want to put a few things at the top: You probably should not change the checkpoint_timeout setting from its default of 5 minutes. Users who read tuning advice on the internet (or get bad AI advice) about increasing this setting usually don’t understand the risk and significance of trading for RTO/availability. Let’s … Continue reading
PGConf.dev 2026 Trip Summary
I’m back home from Vancouver. What a great week – in every way. I’ll try to share a few highlights here. Updated Happiness Hints First and foremost: after many years, the Happiness Hints have received a major update! Before the conference, I updated the hints based on all the feedback I’ve collected over the past … Continue reading
Zero autovacuum_vacuum_cost_delay, Write Storms, and You
A few days ago, Shaun Thomas published an article over on the pgEdge blog called [Checkpoints, Write Storms, and You]. I left a few comments [on LinkedIn]. This article is a great technical read about an important and overlooked topic – and I always love seeing real test results to illustrate the details. I don’t … Continue reading
How Blocking-Lock Brownouts Can Escalate from Row-Level to Complete System Outages
This article is a shortened version. For the full writeup, go to https://github.com/ardentperf/pg-idle-test/tree/main/conn_exhaustion This test suite demonstrates a failure mode when application bugs which poison connection pools collide with PgBouncers that are missing peer config and positioned behind a load balancer. PgBouncer’s peering feature (added with v1.19 in 2023) should be configured if multiple PgBouncers are being used … Continue reading
Explaining IPC:SyncRep – Postgres Sync Replication is Not Actually Sync Replication
Postgres database-level “synchronous replication” does not actually mean the replication is synchronous. It’s a bit of a lie really. The replication is actually – always – asynchronous. What it actually means is “when the client issues a COMMIT then pause until we know the transaction is replicated.” In fact the primary writer database doesn’t need … Continue reading
Postgres Per-Connection Statistics
I’ve had a wish list for a few years now of observability-related things I’d love to see someday in community/open-source Postgres. A few items from my wish list: Wait event counters and cumulative time Wait event arguments (like object, block, etc – specific argument depends on the wait event) Comprehensive tracking of CPU time (ie. … Continue reading
Postgres Indexes, Partitioning and LWLock:LockManager Scalability
I have decided that – in Postgres circles – I shall henceforth refer to 2023 as THE YEAR OF THE LOCK MANAGER’S REVENGE. Let me explain. Lets start with Bruce Momjian. He has an in-depth presentation about locking in general with PostgreSQL called “Unlocking the Postgres Lock Manager“. I see online that he’s been giving … Continue reading
A Hairy PostgreSQL Incident
It was 5:17pm today, just as I was wrapping up work for the day, and my manager pinged me with the following chat: <manager>: Hi Jeremy – we have a <other team> ticket – escalated to <leader>, <leader>, etc. <principal> is on trying to advise as well. Are you available this evening if needed for … Continue reading
PostgreSQL Invalid Page and Checksum Verification Failed
At the Seattle PostgreSQL User Group meetup this past Tuesday, we got onto the topic of invalid pages in PostgreSQL. It was a fun discussion and it made me realize that it’d be worth writing down a bunch of the stuff we talked about – it might be interesting to a few more people too! … Continue reading
Disclaimer
This is my personal website. The views expressed here are mine alone and may not reflect the views of my employer.
contact: schneider @ ardentperf.com
Recent Comments