From 9ff28af04b1f7ec583f9cbfdf1c3ba5e4848d63d Mon Sep 17 00:00:00 2001 From: Altair Bow Date: Fri, 29 Sep 2023 14:07:23 +0800 Subject: [PATCH] update readme --- .github/ISSUE_TEMPLATE/bug_report.md | 5 +- README.md | 102 +++++++++++++++++---------- README_CN.md | 84 ++++++++++++++-------- 3 files changed, 120 insertions(+), 71 deletions(-) diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index ce1ea88..2aac433 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -14,8 +14,9 @@ A clear and concise description of what the bug is. Switch to django's backend (`django.db.backends.xxx`), does the problem reproduce? **Environment** -- Python Version: -- Django Version: + +- Python Version: +- Django Version: **Traceback** Post traceback here. diff --git a/README.md b/README.md index fb11912..91b3eba 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,9 @@ # django-db-connection-pool -*:star: If this project is helpful to you, please light up the star, Thank you:smile:* +:star: If this project is helpful to you, please light up the star, Thank you:smile: -MySQL & Oracle & PostgreSQL & JDBC (Oracle, OceanBase) connection pool components for Django, -Be based on [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy). +MySQL & Oracle & PostgreSQL & JDBC (Oracle, OceanBase) connection pool components for Django, +Be based on [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy). Works fine in multiprocessing and multithreading django project. * [中文版](README_CN.md) @@ -13,22 +13,29 @@ Works fine in multiprocessing and multithreading django project. ### Installation Install with `pip` with all engines: + ```bash $ pip install django-db-connection-pool[all] ``` + or select specific engines: + ```bash $ pip install django-db-connection-pool[mysql,oracle,postgresql,jdbc] ``` + or one of mysql,oracle,postgresql,jdbc + ```bash $ pip install django-db-connection-pool[oracle] ``` ### Update settings.DATABASES -#### MySQL +#### MySQL + change `django.db.backends.mysql` to `dj_db_conn_pool.backends.mysql`: + ```python DATABASES = { 'default': { @@ -37,8 +44,10 @@ DATABASES = { } ``` -#### Oracle +#### Oracle + change `django.db.backends.oracle` to `dj_db_conn_pool.backends.oracle`: + ```python DATABASES = { 'default': { @@ -47,8 +56,10 @@ DATABASES = { } ``` -#### PostgreSQL +#### PostgreSQL + change `django.db.backends.postgresql` to `dj_db_conn_pool.backends.postgresql`: + ```python DATABASES = { 'default': { @@ -58,82 +69,91 @@ DATABASES = { ``` #### Pool options(optional) + you can provide additional options to pass to SQLAlchemy's pool creation, key's name is `POOL_OPTIONS`: ```python DATABASES = { 'default': { - 'POOL_OPTIONS' : { + 'POOL_OPTIONS': { 'POOL_SIZE': 10, 'MAX_OVERFLOW': 10, 'RECYCLE': 24 * 60 * 60 } - } - } + } +} ``` -`django-db-connection-pool` has more configuration options here: [PoolContainer.pool_default_params](https://github.com/altairbow/django-db-connection-pool/blob/master/dj_db_conn_pool/core/__init__.py#L13-L20) - +`django-db-connection-pool` has more configuration options +here: [PoolContainer.pool_default_params](https://github.com/altairbow/django-db-connection-pool/blob/master/dj_db_conn_pool/core/__init__.py#L13-L20) + Here's the explanation of these options(from SQLAlchemy's Doc): * **pool_size**: The size of the pool to be maintained, - defaults to 5. This is the largest number of connections that - will be kept persistently in the pool. Note that the pool - begins with no connections; once this number of connections - is requested, that number of connections will remain. - `pool_size` can be set to 0 to indicate no size limit; to - disable pooling, use a :class:`~sqlalchemy.pool.NullPool` - instead. + defaults to 5. This is the largest number of connections that + will be kept persistently in the pool. Note that the pool + begins with no connections; once this number of connections + is requested, that number of connections will remain. + `pool_size` can be set to 0 to indicate no size limit; to + disable pooling, use a :class:`~sqlalchemy.pool.NullPool` + instead. * **max_overflow**: The maximum overflow size of the - pool. When the number of checked-out connections reaches the - size set in pool_size, additional connections will be - returned up to this limit. When those additional connections - are returned to the pool, they are disconnected and - discarded. It follows then that the total number of - simultaneous connections the pool will allow is pool_size + - `max_overflow`, and the total number of "sleeping" - connections the pool will allow is pool_size. `max_overflow` - can be set to -1 to indicate no overflow limit; no limit - will be placed on the total number of concurrent - connections. Defaults to 10. - -* **recycle**: If set to a value other than -1, number of seconds - between connection recycling, which means upon checkout, - if this timeout is surpassed the connection will be closed - and replaced with a newly opened connection. - Defaults to -1. + pool. When the number of checked-out connections reaches the + size set in pool_size, additional connections will be + returned up to this limit. When those additional connections + are returned to the pool, they are disconnected and + discarded. It follows then that the total number of + simultaneous connections the pool will allow is pool_size + + `max_overflow`, and the total number of "sleeping" + connections the pool will allow is pool_size. `max_overflow` + can be set to -1 to indicate no overflow limit; no limit + will be placed on the total number of concurrent + connections. Defaults to 10. + +* **recycle**: If set to a value other than -1, number of seconds + between connection recycling, which means upon checkout, + if this timeout is surpassed the connection will be closed + and replaced with a newly opened connection. + Defaults to -1. Or, you can use dj_db_conn_pool.setup to change default arguments(for each pool's creation), before using database pool: ```python import dj_db_conn_pool + dj_db_conn_pool.setup(pool_size=100, max_overflow=50) ``` #### multiprocessing environment -In a multiprocessing environment, such as uWSGI, each process will have its own `dj_db_conn_pool.core:pool_container` object, -It means that each process has an independent connection pool, for example: + +In a multiprocessing environment, such as uWSGI, each process will have its own `dj_db_conn_pool.core:pool_container` +object, +It means that each process has an independent connection pool, for example: The `POOL_OPTIONS` configuration of database `db1` is`{ 'POOL_SIZE': 10, 'MAX_OVERFLOW': 20 }`, If uWSGI starts 8 worker processes, then the total connection pool size of `db1` is `8 * 10`, The maximum number of connections will not exceed `8 * 10 + 8 * 20` - ## JDBC + Thanks to [JPype](https://github.com/jpype-project/jpype), django-db-connection-pool can connect to database by jdbc ### Usage + #### Set Java runtime environment + ```bash export JAVA_HOME=$PATH_TO_JRE; export CLASSPATH=$PATH_RO_JDBC_DRIVER_JAR ``` #### Update settings.DATABASES + ##### Oracle change `django.db.backends.oracle` to `dj_db_conn_pool.backends.jdbc.oracle`: + ```python DATABASES = { 'default': { @@ -143,7 +163,9 @@ DATABASES = { ``` ##### OceanBase + use `dj_db_conn_pool.backends.jdbc.oceanbase`: + ```python DATABASES = { 'default': { @@ -153,8 +175,10 @@ DATABASES = { ``` ### Performing raw SQL queries -Just like django's built-in backends, all JDBC backends support named parameters in raw SQL queries, + +Just like django's built-in backends, all JDBC backends support named parameters in raw SQL queries, you can execute raw sql queries like this: + ```python from django.db import connections diff --git a/README_CN.md b/README_CN.md index 0be0405..cf85c08 100644 --- a/README_CN.md +++ b/README_CN.md @@ -1,32 +1,41 @@ # django-db-connection-pool -*:star: 如果这个库能对你有所帮助，不妨点个star，谢谢:smile:* +:star: 如果这个库能对你有所帮助，不妨点个star，谢谢:smile: -驱动 Django 访问 MySQL、Oracle、PostgreSQL、JDBC (Oracle, OceanBase) 连接池的轮子, +驱动 Django 访问 MySQL、Oracle、PostgreSQL、JDBC (Oracle, OceanBase) 连接池的轮子, 基于 [SQLAlchemy](https://github.com/sqlalchemy/sqlalchemy) 队列池。 在多进程、多线程 django 项目中，运行良好。 * [English version](README.md) ## 快速开始 + ### 安装 + + 使用 `pip` 安装所有所支持的数据库组件: + ```bash $ pip install django-db-connection-pool[all] ``` + + 或者是选择性安装数据库组件: + ```bash $ pip install django-db-connection-pool[mysql,oracle,postgresql,jdbc] ``` + + 或只选择部分组件，比如 Oracle + ```bash $ pip install django-db-connection-pool[oracle] ``` ### 更新 settings.DATABASES 配置 -#### MySQL +#### MySQL + 将 ENGINE `django.db.backends.mysql` 更改为 `dj_db_conn_pool.backends.mysql`: + ```python DATABASES = { 'default': { @@ -45,7 +54,9 @@ DATABASES = { ``` #### Oracle + 将 ENGINE `django.db.backends.oracle` 更改为 `dj_db_conn_pool.backends.oracle`: + ```python DATABASES = { 'default': { @@ -63,8 +74,10 @@ DATABASES = { } ``` -#### PostgreSQL +#### PostgreSQL + 将 ENGINE `django.db.backends.postgresql` 更改为 `dj_db_conn_pool.backends.postgresql`: + ```python DATABASES = { 'default': { @@ -83,6 +96,7 @@ DATABASES = { ``` #### 连接池配置（可选） + 目前连接池限制用户传入的连接池配置为：POOL_SIZE（连接池容量）、MAX_OVERFLOW（连接池容量向上浮动最大值） 这两个参数包含在 `POOL_OPTIONS` 内，例如下面的配置，default 的连接池常规容量为10个连接，最大浮动10个， 即为：在 default 连接池创建后，随着程序对连接池的请求，连接池内连接将逐步增加到10个，如果在连接池内连接 @@ -108,32 +122,32 @@ DATABASES = { 附这些参数的解释：(摘录于 SQLAlchemy 的文档): * **pool_size**: The size of the pool to be maintained, - defaults to 5. This is the largest number of connections that - will be kept persistently in the pool. Note that the pool - begins with no connections; once this number of connections - is requested, that number of connections will remain. - `pool_size` can be set to 0 to indicate no size limit; to - disable pooling, use a :class:`~sqlalchemy.pool.NullPool` - instead. + defaults to 5. This is the largest number of connections that + will be kept persistently in the pool. Note that the pool + begins with no connections; once this number of connections + is requested, that number of connections will remain. + `pool_size` can be set to 0 to indicate no size limit; to + disable pooling, use a :class:`~sqlalchemy.pool.NullPool` + instead. * **max_overflow**: The maximum overflow size of the - pool. When the number of checked-out connections reaches the - size set in pool_size, additional connections will be - returned up to this limit. When those additional connections - are returned to the pool, they are disconnected and - discarded. It follows then that the total number of - simultaneous connections the pool will allow is pool_size + - `max_overflow`, and the total number of "sleeping" - connections the pool will allow is pool_size. `max_overflow` - can be set to -1 to indicate no overflow limit; no limit - will be placed on the total number of concurrent - connections. Defaults to 10. - -* **recycle**: If set to a value other than -1, number of seconds - between connection recycling, which means upon checkout, - if this timeout is surpassed the connection will be closed - and replaced with a newly opened connection. - Defaults to -1. + pool. When the number of checked-out connections reaches the + size set in pool_size, additional connections will be + returned up to this limit. When those additional connections + are returned to the pool, they are disconnected and + discarded. It follows then that the total number of + simultaneous connections the pool will allow is pool_size + + `max_overflow`, and the total number of "sleeping" + connections the pool will allow is pool_size. `max_overflow` + can be set to -1 to indicate no overflow limit; no limit + will be placed on the total number of concurrent + connections. Defaults to 10. + +* **recycle**: If set to a value other than -1, number of seconds + between connection recycling, which means upon checkout, + if this timeout is surpassed the connection will be closed + and replaced with a newly opened connection. + Defaults to -1. 或者调用 `dj_db_conn_pool.setup` 覆盖默认参数 @@ -143,24 +157,32 @@ dj_db_conn_pool.setup(pool_size=10, max_overflow=20) ``` #### 多进程环境 -在多进程环境内，每个进程都会拥有一个`dj_db_conn_pool.core:pool_container`对象， 意味着每个进程都拥有一个独立的连接池，举例说明： + +在多进程环境内，每个进程都会拥有一个`dj_db_conn_pool.core:pool_container`对象， +意味着每个进程都拥有一个独立的连接池，举例说明： 数据库`db1`的`POOL_OPTIONS`的配置是 -`{ 'POOL_SIZE': 10, 'MAX_OVERFLOW': 20 }`，项目启动了8个进程，则该项目的`db1`连接池总大小是`8 * 10`，最大连接数是`8 * 10 + 8 * 20` +`{ 'POOL_SIZE': 10, 'MAX_OVERFLOW': 20 }`，项目启动了8个进程，则该项目的`db1`连接池总大小是`8 * 10` +，最大连接数是`8 * 10 + 8 * 20` ## JDBC（仍在完善中，不建议用于生产） + 基于 [JPype](https://github.com/jpype-project/jpype)，django-db-connection-pool 现在可以通过 jdbc 连接到数据库并保持连接 ### 使用方法 + #### 设置环境变量 + ```bash export JAVA_HOME=$PATH_TO_JRE; export CLASSPATH=$PATH_RO_JDBC_DRIVER_JAR ``` #### 更新 settings.DATABASES 配置 + ##### Oracle 将 ENGINE `django.db.backends.oracle` 更改为 `dj_db_conn_pool.backends.jdbc.oracle`: + ```python DATABASES = { 'default': { @@ -179,7 +201,9 @@ DATABASES = { ``` ##### OceanBase + 使用 `dj_db_conn_pool.backends.jdbc.oceanbase` 作为 Django 数据库后端: + ```python DATABASES = { 'default': {