You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/docs/core/settings.mdx
+20-18Lines changed: 20 additions & 18 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -14,9 +14,8 @@ Certain settings need to be provided for CocoIndex to work, e.g. database connec
14
14
15
15
Note that in general, you have two ways to launch CocoIndex:
16
16
17
-
* Call CocoIndex APIs from your own Python application or library.
18
-
* Use [Cocoindex CLI](cli). It's handy for most routine indexing building and management tasks.
19
-
17
+
* Call CocoIndex APIs from your own Python application or library.
18
+
* Use [Cocoindex CLI](cli). It's handy for most routine indexing building and management tasks.
20
19
21
20
CocoIndex exposes process-level settings specified by `cocoindex.Settings` dataclass.
22
21
Settings can be configured in three different ways.
@@ -77,10 +76,10 @@ But be careful that if you call `cocoindex.init()` only under the path of main (
77
76
78
77
`cocoindex.init()` is optional:
79
78
80
-
- You can call `cocoindex.init()` with a `cocoindex.Settings` dataclass object as argument, or without any argument.
79
+
* You can call `cocoindex.init()` with a `cocoindex.Settings` dataclass object as argument, or without any argument.
81
80
When without argument, the settings will be loaded from the `@cocoindex.settings` function or environment variables.
82
81
83
-
- You don't have to explicitly call `cocoindex.init()`.
82
+
* You don't have to explicitly call `cocoindex.init()`.
84
83
CocoIndex will be automatically initialized when needed, e.g. when any method of any flow is called the first time.
85
84
But calling `cocoindex.init()` explicitly (usually at startup time, e.g. in the main function of your application) has the benefit of making sure CocoIndex library is initialized and any potential exceptions are raised earlier before proceeding with the application.
86
85
If you need this clarity, you can call it explicitly even if you don't want to provide settings by the `cocoindex.init()` call.
@@ -91,9 +90,9 @@ But be careful that if you call `cocoindex.init()` only under the path of main (
91
90
92
91
`cocoindex.Settings` is a dataclass that contains the following fields:
93
92
94
-
*`app_namespace` (type: `str`, required): The namespace of the application.
95
-
*`database` (type: `DatabaseConnectionSpec`, required): The connection to the Postgres database.
96
-
*`global_execution_options` (type: `GlobalExecutionOptions`, optional): The global execution options shared by all flows.
93
+
*`app_namespace` (type: `str`, required): The namespace of the application.
94
+
*`database` (type: `DatabaseConnectionSpec`, required): The connection to the Postgres database.
95
+
*`global_execution_options` (type: `GlobalExecutionOptions`, optional): The global execution options shared by all flows.
97
96
98
97
### App Namespace
99
98
@@ -110,15 +109,15 @@ If not set, all flows are in a default unnamed namespace.
110
109
111
110
`DatabaseConnectionSpec` configures the connection to a database. Only Postgres is supported for now. It has the following fields:
112
111
113
-
*`url` (type: `str`): The URL of the Postgres database to use as the internal storage, e.g. `postgres://cocoindex:cocoindex@localhost/cocoindex`.
112
+
*`url` (type: `str`): The URL of the Postgres database to use as the internal storage, e.g. `postgres://cocoindex:cocoindex@localhost/cocoindex`.
114
113
115
114
*Environment variable* for `Settings.database.url`: `COCOINDEX_DATABASE_URL`
116
115
117
-
*`user` (type: `Optional[str]`, default: `None`): The username for the Postgres database. If not provided, username will come from `url`.
116
+
*`user` (type: `Optional[str]`, default: `None`): The username for the Postgres database. If not provided, username will come from `url`.
118
117
119
118
*Environment variable* for `Settings.database.user`: `COCOINDEX_DATABASE_USER`
120
119
121
-
*`password` (type: `Optional[str]`, default: `None`): The password for the Postgres database. If not provided, password will come from `url`.
120
+
*`password` (type: `Optional[str]`, default: `None`): The password for the Postgres database. If not provided, password will come from `url`.
122
121
123
122
*Environment variable* for `Settings.database.password`: `COCOINDEX_DATABASE_PASSWORD`
124
123
@@ -129,30 +128,33 @@ If not set, all flows are in a default unnamed namespace.
129
128
130
129
:::
131
130
132
-
*`max_connections` (type: `int`, default: `25`): The maximum number of connections to keep in the pool.
131
+
*`max_connections` (type: `int`, default: `25`): The maximum number of connections to keep in the pool.
133
132
134
133
*Environment variable* for `Settings.database.max_connections`: `COCOINDEX_DATABASE_MAX_CONNECTIONS`
135
134
136
-
*`min_connections` (type: `int`, default: `5`): The minimum number of connections to keep in the pool.
135
+
*`min_connections` (type: `int`, default: `5`): The minimum number of connections to keep in the pool.
137
136
138
137
*Environment variable* for `Settings.database.min_connections`: `COCOINDEX_DATABASE_MIN_CONNECTIONS`
139
138
140
-
141
139
:::info
142
140
143
141
If you use the Postgres database hosted by [Supabase](https://supabase.com/), please click **Connect** on your project dashboard and find the following URL:
144
142
145
-
* If you're on a IPv6 network, use the URL under **Direct connection**. You can visit [IPv6 test](https://test-ipv6.com/) to see if you have IPv6 Internet connection.
146
-
* Otherwise, use the URL under **Session pooler**.
143
+
* If you're on a IPv6 network, use the URL under **Direct connection**. You can visit [IPv6 test](https://test-ipv6.com/) to see if you have IPv6 Internet connection.
144
+
* Otherwise, use the URL under **Session pooler**.
145
+
Note that Supabase has a pool size limit of 15 by default, while CocoIndex's default `max_connections` value is 25.
146
+
You can adjust either value to make sure Supabase's pool size limit is greater than CocoIndex's `max_connections` value.
147
+
Supabase's pool size limit can be adjusted under "Database" -> "Settings".
148
+
* CocoIndex doesn't support *Transaction pooler* now.
147
149
148
150
:::
149
151
150
152
### GlobalExecutionOptions
151
153
152
154
`GlobalExecutionOptions` is used to configure the global execution options shared by all flows. It has the following fields:
153
155
154
-
*`source_max_inflight_rows` (type: `int | None`, default: `1024`): The maximum number of concurrent inflight rows for all source operations.
155
-
*`source_max_inflight_bytes` (type: `int | None`, default: `None`): The maximum number of concurrent inflight bytes for all source operations.
156
+
*`source_max_inflight_rows` (type: `int | None`, default: `1024`): The maximum number of concurrent inflight rows for all source operations.
157
+
*`source_max_inflight_bytes` (type: `int | None`, default: `None`): The maximum number of concurrent inflight bytes for all source operations.
156
158
157
159
See also [flow definition docs](/docs/core/flow_def#control-processing-concurrency) about why it's necessary to control processing concurrency, and how to configure it on per-source basis.
158
160
If both global and per-source limits are specified, both need to be satisfied to admit additional source rows.
0 commit comments