You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/features/load-balancer/healthchecks.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,7 @@
1
1
---
2
2
icon: material/lan-check
3
3
---
4
+
4
5
# Health checks
5
6
6
7
All databases load balanced by PgDog are regularly checked with health checks. A health check is a small query that ensures the database is reachable and able to handle requests.
@@ -90,7 +91,7 @@ The **default** value is **5 minutes** (`300_000` milliseconds).
90
91
!!! note
91
92
A database will not be placed back into the load balancer until it passes a health check again.
92
93
93
-
Make sure that `idle_healthcheck_timeout` is set to a lower setting than `ban_timeout`, so health checks have time to run before you expect the database to resume serving traffic.
94
+
Make sure that `idle_healthcheck_interval` is set to a lower value than `ban_timeout`, so health checks have time to run before you expect the database to resume serving traffic.
Copy file name to clipboardExpand all lines: docs/features/load-balancer/index.md
+5-121Lines changed: 5 additions & 121 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,9 @@
1
1
---
2
2
next_steps:
3
-
- ["Health checks", "/features/healthchecks/", "Ensure replica databases are up and running. Block offline databases from serving queries."]
3
+
- ["Health checks", "/features/load-balancer/healthchecks", "Ensure replica databases are up and running. Block offline databases from serving queries."]
4
+
- ["Replication & failover", "/features/load-balancer/replication-failover", "Replica lag detection and automatic traffic failover on replica promotion."]
5
+
- ["Transactions", "/features/load-balancer/transactions", "Handling of manually-started transactions."]
6
+
- ["Manual routing", "/features/load-balancer/manual-routing", "Overriding the load balancer using connection parameters or query comments."]
4
7
icon: material/lan
5
8
---
6
9
@@ -89,7 +92,7 @@ The load balancer detects this and will send this query to the primary database
89
92
90
93
!!! note "Transaction required"
91
94
92
-
`SELECT FOR UPDATE` is used inside manual [transactions](#transactions) (i.e., started with `BEGIN`), which are routed to the primary database by default.
95
+
`SELECT FOR UPDATE` is used inside manual [transactions](transactions.md) (i.e., started with `BEGIN`), which are routed to the primary database by default.
93
96
94
97
### Write CTEs
95
98
@@ -104,114 +107,6 @@ SELECT * FROM users INNER JOIN t ON t.id = users.id
104
107
105
108
The load balancer recursively checks CTEs and, if any of them contains a query that could trigger a write, it will send the whole statement to the primary database.
106
109
107
-
### Transactions
108
-
109
-
All manual transactions are sent to the primary database by default. Transactions are started by sending the `BEGIN` command, for example:
110
-
111
-
```postgresql
112
-
BEGIN;
113
-
INSERT INTO users (email, created_at) VALUES ($1, NOW()) RETURNING *;
114
-
COMMIT;
115
-
```
116
-
117
-
PgDog processes queries immediately upon receiving them, and since transactions can contain multiple statements, it isn't possible to determine whether the whole transaction writes to the database. Therefore, it is more reliable to send it to the primary database.
118
-
119
-
!!! note "Replica lag"
120
-
While transactions are used to atomically change multiple tables, they can also be used to manually route `SELECT` queries to the primary database. For example:
121
-
122
-
```postgresql
123
-
BEGIN;
124
-
SELECT * FROM users WHERE id = $1;
125
-
COMMIT;
126
-
```
127
-
128
-
129
-
This is useful when the data in the table(s) has been recently updated and you want to avoid errors caused by replication lag. This often manifests as "record not-found"-style errors, for example:
130
-
131
-
```
132
-
ActiveRecord::RecordNotFound (Couldn't find User with 'id'=9999):
133
-
```
134
-
135
-
While sending read queries to the primary adds load, it is often necessary in real-time systems that are not equipped to handle replication delays.
136
-
137
-
138
-
#### Read-only transactions
139
-
140
-
The PostgreSQL query language allows you to declare a transaction as read-only. This prevents it from writing data to the database. PgDog takes advantage of this property and will send such transactions to a replica database.
141
-
142
-
Read-only transactions can be started with the `BEGIN READ ONLY` command, for example:
143
-
144
-
```postgresql
145
-
BEGIN READ ONLY;
146
-
SELECT * FROM users WHERE id = $1;
147
-
COMMIT;
148
-
```
149
-
150
-
Read-only transactions are useful when queries depend on each other's results and need a consistent view of the database. Some Postgres database drivers allow this option to be set in the code, for example:
151
-
152
-
=== "pgx (go)"
153
-
```go
154
-
tx, err := conn.BeginTx(ctx, pgx.TxOptions{
155
-
AccessMode: pgx.ReadOnly,
156
-
})
157
-
```
158
-
=== "Sequelize (node)"
159
-
```javascript
160
-
const tx = await sequelize.transaction({
161
-
readOnly: true,
162
-
});
163
-
```
164
-
=== "SQLAlchemy (python)"
165
-
Add `postgresql_readonly=True` to [execution options](https://docs.sqlalchemy.org/en/20/core/connections.html#sqlalchemy.engine.Engine.execution_options), like so:
If you need to override the load balancer routing decision and send a query (or all queries) to the primary, it's possible to do so by configuring the `pgdog.role` connection parameter.
174
-
175
-
Configuring this connection parameter can be done at connection creation:
|`primary`| Queries are sent to the primary database only. |
212
-
|`replica`| Queries are load balanced between primary and replicas, depending on the value of the [`read_write_split`](../../configuration/pgdog.toml/general.md#read_write_split) setting. |
213
-
214
-
215
110
## Using the load balancer
216
111
217
112
The load balancer is **enabled by default** when more than one database with the same `name` property is configured in [pgdog.toml](../../configuration/pgdog.toml/databases.md), for example:
@@ -248,17 +143,6 @@ In case one of your replicas fails, you can configure the primary to serve read
This feature was added in commit version [`c49339f`](https://github.com/pgdogdev/pgdog/commit/c49339f70db8be63b76ebb3aa0f31433c4266f21). If using this feature, make sure to run the latest version of PgDog.
255
-
256
-
If your query is replica-lag sensitive (e.g., you are reading data that you just wrote), you can route it to the primary manually. The query router supports doing this with a query comment:
257
-
258
-
```postgresql
259
-
/* pgdog_role: primary */ SELECT * FROM users WHERE id = $1
PgDog's load balancer uses the PostgreSQL parser to understand and route queries between the primary and replicas. In certain situations, the overhead of parsing queries may be too high, e.g., when not using prepared statements.
7
+
PgDog's load balancer uses the PostgreSQL parser to understand and route queries between the primary and replicas. If you want more control, you can provide the load balancer with hints, influencing its routing decisions.
8
8
9
-
To take advantage of the load balancer without the parser's overhead, you can provide a routing hint at connection creation.
9
+
This can be done on a per-query basis by using a comment, or on the entire client connection, with a session parameter.
10
10
11
-
## How it works
11
+
## Query comments
12
+
13
+
If your query is replica-lag sensitive (e.g., you are reading data that you just wrote), you can route it to the primary manually. The load balancer supports doing this with a query comment:
14
+
15
+
```postgresql
16
+
/* pgdog_role: primary */ SELECT * FROM users WHERE id = $1
17
+
```
18
+
19
+
Query comments are supported in all types of queries, including prepared statements. If you're using the latter, the comments are parsed only once per client connection, removing any performance overhead of extracting them from the query.
20
+
21
+
22
+
## Parameters
12
23
13
24
Startup parameters are connection-specific settings that are typically set on connection creation to configure database behavior. For example, this is how ORMs and web frameworks control settings like `application_name`, `work_mem`, `statement_timeout` and many others.
14
25
15
26
The Postgres protocol doesn't have any restrictions on parameter names or values, and PgDog can choose to forward those settings to Postgres (or not).
16
27
17
-
### Parameters
18
-
19
28
PgDog has two parameters that control which database is used for all queries on a client connection:
Depending on the environment, the parameters may need to be URL-encoded, e.g., `%20` is a space and `%3D` is the equals (`=`) sign.
48
57
49
-
#### asyncpg
50
-
51
-
[asyncpg](https://pypi.org/project/asyncpg/) is a popular PostgreSQL driver for asynchronous Python applications. It allows you to set connection parameters when creating a connection:
52
-
53
-
```python
54
-
conn =await asyncpg.connect(
55
-
user="pgdog",
56
-
password="pgdog",
57
-
database="pgdog",
58
-
host="10.0.0.0",
59
-
port=6432,
60
-
server_settings={
61
-
"pgdog.role": "primary",
62
-
}
63
-
)
58
+
=== "asyncpg"
59
+
60
+
[asyncpg](https://pypi.org/project/asyncpg/) is a popular PostgreSQL driver for asynchronous Python applications. It allows you to set connection parameters when creating a connection:
61
+
62
+
```python
63
+
conn = await asyncpg.connect(
64
+
user="pgdog",
65
+
password="pgdog",
66
+
database="pgdog",
67
+
host="10.0.0.0",
68
+
port=6432,
69
+
server_settings={
70
+
"pgdog.role": "primary",
71
+
}
72
+
)
73
+
```
74
+
75
+
=== "SQLAlchemy"
76
+
77
+
[SQLAlchemy](https://www.sqlalchemy.org/) is a popular Python ORM, which supports any number of PostgreSQL connection drivers. For example, if you're using `asyncpg`, you can set connection parameters as follows:
[Rails](https://rubyonrails.org/) and ActiveRecord support passing connection parameters in the `database.yml` configuration file:
94
+
95
+
```yaml
96
+
# config/database.yml
97
+
production:
98
+
adapter: postgresql
99
+
database: pgdog
100
+
username: user
101
+
password: password
102
+
host: 10.0.0.0
103
+
options: "-c pgdog.role=replica -c pgdog.shard=0"
104
+
```
105
+
106
+
These options are passed to the [`pg`](https://github.com/ged/ruby-pg) driver, so if you're using it directly, you can create connections manually like so:
107
+
108
+
```ruby
109
+
require "pg"
110
+
111
+
conn = PG.connect(
112
+
host: "10.0.0.0",
113
+
# user, password, etc.
114
+
options: "-c pgdog.role=primary -c pgdog.shard=1"
115
+
)
116
+
```
117
+
118
+
### Using `SET`
119
+
120
+
The PostgreSQL protocol supports setting connection parameters using the `SET` statement. This also works for configuring both `pgdog.role` and `pgdog.shard` parameters.
121
+
122
+
For example, if you want all subsequent queries to be sent to the primary, you can execute the following statement:
123
+
124
+
```postgresql
125
+
SET pgdog.role TO "primary";
64
126
```
65
127
66
-
#### SQLAlchemy
128
+
#### Inside transactions
67
129
68
-
[SQLAlchemy](https://www.sqlalchemy.org/) is a popular Python ORM, which supports any number of PostgreSQL connection drivers. For example, if you're using `asyncpg`, you can set connection parameters as follows:
130
+
If you want to provide a transaction routing hint without affecting the rest of the connection, youcan use `SET LOCAL`:
In this example, all transaction statements (including the `BEGIN` statement) will be sent to the primary database. Whether the transaction is committed or reverted, the value of `pgdog.role` will be reset to its previous value.
83
138
84
-
[Rails](https://rubyonrails.org/) and ActiveRecord support passing connection parameters in the `database.yml` configuration file:
139
+
!!! note "Statement ordering"
140
+
To make sure PgDog intercepts the routing hint early enough in the transaction flow, make sure to send all hints _before_ executing actual queries.
85
141
86
-
```yaml
87
-
# config/database.yml
88
-
production:
89
-
adapter: postgresql
90
-
database: pgdog
91
-
username: user
92
-
password: password
93
-
host: 10.0.0.0
94
-
options: "-c pgdog.role=replica -c pgdog.shard=0"
95
-
```
142
+
The following flow, for example, _will not_ work:
96
143
97
-
These options are passed to the [`pg`](https://github.com/ged/ruby-pg) driver, so if you're using it directly, you can create connections manually like so:
144
+
```postgresql
145
+
BEGIN;
146
+
SELECT * FROM users WHERE id = $1;
147
+
SET LOCAL pgdog.role TO "primary"; -- The client is already connected to a server.
148
+
INSERT INTO users (id) VALUES ($1); -- If connected to a replica, this will fail.
149
+
```
98
150
99
-
```ruby
100
-
require "pg"
101
151
102
-
conn = PG.connect(
103
-
host: "10.0.0.0",
104
-
# user, password, etc.
105
-
options: "-c pgdog.role=primary -c pgdog.shard=1"
106
-
)
107
-
```
108
152
109
-
## Disable the parser
153
+
## Disabling the parser
154
+
155
+
In certain situations, the overhead of parsing queries may be too high, e.g., when your application can't use prepared statements.
110
156
111
-
Once you've configured the desired database role (and/or shard) for each of your application connections, you can disable the query parser in [pgdog.toml](../../configuration/pgdog.toml/general.md#query_parser):
157
+
If you've configured the desired database role (and/or shard) for each of your application connections, you can safely disable the query parser in [pgdog.toml](../../configuration/pgdog.toml/general.md#query_parser):
112
158
113
159
```toml
114
160
[general]
115
161
query_parser = "off"
116
162
```
117
163
118
-
!!! note "Session state"
119
-
The query parser is used to intercept `SET` commands, which configure session variables at runtime. If the parser is disabled and your application uses `SET` commands to configure the connection at startup, PgDog will not be able to guarantee that connections have the correct session settings in [transaction mode](../transaction-mode.md).
164
+
Once the parser is disabled, PgDog will rely solely on the `pgdog.role` and `pgdog.shard` parameters to make its routing decisions.
165
+
166
+
### Session state
167
+
168
+
The query parser is used to intercept and interpret `SET` commands, which set session variables at runtime.
169
+
170
+
If the parser is disabled and your application uses `SET` commands to configure the connection at startup, PgDog will not be able to guarantee that all connections have the correct session settings in [transaction mode](../transaction-mode.md).
0 commit comments