aboutsummaryrefslogtreecommitdiff
path: root/content/posts/2021-11-05-postgres-query-rewrite.smd
diff options
context:
space:
mode:
authorMartin Ashby <martin@ashbysoft.com>2024-12-21 22:18:18 +0000
committerMartin Ashby <martin@ashbysoft.com>2024-12-21 22:18:18 +0000
commit82e7ac75954c973a5c59a02684788c0fb26e14dd (patch)
treea1b09939f30bc4c232e69f980c2a47e48625ee1d /content/posts/2021-11-05-postgres-query-rewrite.smd
parentff323f79e03174e4cdf2a709c095ff83e7ea3669 (diff)
downloadmfashby.net-82e7ac75954c973a5c59a02684788c0fb26e14dd.tar.gz
mfashby.net-82e7ac75954c973a5c59a02684788c0fb26e14dd.tar.bz2
mfashby.net-82e7ac75954c973a5c59a02684788c0fb26e14dd.tar.xz
mfashby.net-82e7ac75954c973a5c59a02684788c0fb26e14dd.zip
Bump zine 0.3.0 -> 0.8.0
Fix a lot of associated breakage Fix broken rss.xml file as highlighted to me by Matthijs van der Wild (thanks!)
Diffstat (limited to 'content/posts/2021-11-05-postgres-query-rewrite.smd')
-rw-r--r--content/posts/2021-11-05-postgres-query-rewrite.smd19
1 files changed, 19 insertions, 0 deletions
diff --git a/content/posts/2021-11-05-postgres-query-rewrite.smd b/content/posts/2021-11-05-postgres-query-rewrite.smd
new file mode 100644
index 0000000..d3ce7ea
--- /dev/null
+++ b/content/posts/2021-11-05-postgres-query-rewrite.smd
@@ -0,0 +1,19 @@
+---
+.title = "Postgres Query Rewrite",
+.author = "Martin Ashby",
+.date = @date("2021-11-05T11:57:12Z"),
+.layout = "single.shtml",
+.custom = {"comments": true},
+---
+
+Recently at work, I needed to be able to rewrite some [postgresql](https://www.postgresql.org/) queries which were coming from a closed-source application in order to integrate the app into our own system testing setup. Specifically I needed to replace instances of `current_timestamp` with invocation of `now()` function.
+
+I found a couple of options to do this:
+1. [pgbouncer-rr](https://github.com/awslabs/pgbouncer-rr-patch) is a _patch_ for [pgbouncer](https://www.pgbouncer.org/), initially written by AWS labs, which extends pgbouncer to include rewrite and rerouting capability.
+2. [pg_query_rewrite](https://github.com/pierreforstmann/pg_query_rewrite) is an _extension_ for postgres, which supports substitution of an entire matching query with another one.
+
+Unfortunately I hit some roadblocks with each. `pg_query_rewrite` simply wasn't flexible enough. I needed to do a string substitution on the query, not entire query replacement as I didn't have the exact set of every query the application would issue. Also providers of managed postgresql instances like Google Cloud or AWS won't let you install your own extensions. `pgbouncer-rr` was moderately difficult to configure given that I only wanted a single feature (rewriting) and none of the usual things pgbouncer is used for (connection pooling, load balancing).
+
+In the end, I found a neat library [pgproto3](https://github.com/jackc/pgproto3) which implements the postgresql wire format and can be used to write both a client and server! I used this to write [an application](https://github.com/patientsknowbest/pg-rewrite-proxy) that can act as a proxy to a postgresql server and will rewrite the queries before forwarding them. The proxy approach seems more flexible than an extension as it can be bundled either on the application server, or on the database server, or somewhere in-between depending on the requirement.
+
+At the moment this application only supports simple string replacement, but I hope to extend it with [LUA](https://www.lua.org/) support soon for arbitrary query rewriting.