This was also published here: https://medium.com/p/20d3749ed680, and I’ve updated it there at various times (though I’ll admit it doesn’t have the new features in PG13-14 yet) but had forgotten about the Gist version with those updates.
I don't see anything relevant in the PG13 release notes, but in the release notes for 14 there are these items:
- "Allow partitions to be detached in a non-blocking manner. The syntax is ALTER TABLE ... DETACH PARTITION ... CONCURRENTLY, and FINALIZE." I didn't cover partitioning in the guide, but this is a real improvement to DDL with partitions.
- "Allow index commands using CONCURRENTLY to avoid waiting for the completion of other operations using CONCURRENTLY." This doesn't add new lower-lock level DDL, but it does improve the speed of existing such DDL.
- "Allow vacuum to be more aggressive in removing dead rows during minimal-locking index operations. Specifically, CREATE INDEX CONCURRENTLY and REINDEX CONCURRENTLY no longer limit the dead row removal of other relations." Similarly this doesn't add new lower-lock level DDL, but it does improve the safety of existing such DDL by eliminating a case where concurrent index creation could result in performance degradation of the system as a whole.
Something that doesn't appear to be mentioned in here that bit me recently is that views follow a table through renames. Ended up having views that were subtly broken because they were pointing to the tables that I had renamed to have an _old suffix instead of the newly created ones I had expected them to point to.
This is a very good example of why you should be using a schema diff/comparison tool.
Run your rename on a copy of your production schema, then run a diff tool to see what's actually changed - in this case it would show a view definition had changed and even autogenerate the replace statement you would need to add to fix the script.
I commented elsewhere that the view renaming thing was surprising to me, but when you put it like this it makes sense. The view references the oid or whatever, some internal representation I try to pretend doesn't exist.
It's kind of surprising in some cases when I haven't seen it before (like views) but is consistent and the alternative would be a lot harder to work with.
It definitely does make sense and it probably what most people expect if they've never worked with a database before. But coming from my experience with other databases (primarily MS SQL Server) it's a surprising difference.
On me for not testing better, but it was some quick and dirty work in a quick and dirty environment that doesn't even have a dev instance.
Off the top of my head, the exception to this is functions (i.e. that you declare inside Postgres), because the function body is really just a piece of text, in any of a bunch of different languages. So if you have a function which refers to specific database objects that are being renamed, you'll also need to update the function definition with the new names.
Note that SQL-standard bodies are supported as of 14 [1]:
Allow SQL-language functions and procedures to use SQL-standard function bodies (Peter Eisentraut)
Previously only string-literal function bodies were supported. When writing a function or procedure in SQL-standard syntax, the body is parsed immediately and stored as a parse tree. This allows better tracking of function dependencies, and can have security benefits.
Postgres is one of the very few technologies that I think is like, actually very good. But it is full of stuff like this. If I had thought about it for a minute it's possible I could have predicted this behavior, but maybe not!
This post is absolutely terrific and has been been my main reference for Reshape, an automated, zero-downtime schema migration tool: https://github.com/fabianlindfors/reshape
That's right! "Simple" views (ones mapping directly to a table) are quite powerful in Postgres and an application interacting with them won't know the difference.
As they note table lock acquisition can be super painful. It's possible to write a script to use pg_stat_activity to detect that the table alteration is waiting on a lock and kill conflicting work. For some workloads the kill is unacceptable, but on highly active systems killing off a few operations is usually much less damaging than letting sessions pile up on a lock wait.
Personally I would love it if it were possible to set a mode on postgres whereby access exclusive locks break other locks and cause the transactions holding them to roll back (similar to what happens with vacuums).
> Multiple concurrent index creations on a single table will not return from either CREATE INDEX CONCURRENTLY ... statement until the slowest one completes.
I am not sure this is true anymore, or at least, it was improved in v14 to only apply when "the indexes involved are partial or have columns that are not simple column references." [1]
Another thing worth noting here is that the exclusive locks taken on system catalogues during a DDL change also replicate to your replicas (necessarily, ha!) and they will have the same behavior described, i.e., they will line up behind queries. So for instance if you run long ETL queries against a replica, the DDL lock will line up behind it and no other queries can run until the ETL query is done and the DDL lock goes through.
This was also published here: https://medium.com/p/20d3749ed680, and I’ve updated it there at various times (though I’ll admit it doesn’t have the new features in PG13-14 yet) but had forgotten about the Gist version with those updates.