feat: Added pool_maxsize for RequestsHttpConnection (#216)

Signed-off-by: Niket Singh <singhnik82@gmail.com>

Signed-off-by: Niket Singh <singhnik82@gmail.com>
Co-authored-by: Niket Singh <niket.singh@dailyrounds.org>
Co-authored-by: Harsha Vamsi Kalluri <harshavamsi096@gmail.com>

CHANGELOG.md

@@ -4,6 +4,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ## [Unreleased]
 ### Added
 - Added Point in time API rest API([#191](https://github.com/opensearch-project/opensearch-py/pull/191))
+- Added pool_maxsize for RequestsHttpConnection ([#216](https://github.com/opensearch-project/opensearch-py/pull/216))
 - Github workflow for changelog verification ([#218](https://github.com/opensearch-project/opensearch-py/pull/218))
 - Added overload decorators to helpers-actions.pyi-"bulk" ([#239](https://github.com/opensearch-project/opensearch-py/pull/239))
 - Document Keberos authenticaion ([214](https://github.com/opensearch-project/opensearch-py/pull/214))

USER_GUIDE.md

@@ -1,6 +1,6 @@
-- [Getting started with the OpenSearch Python client](#getting-started-with-the-opensearch-python-client)
+- [User guide of OpenSearch Python Client](#user-guide-of-opensearch-python-client)
   - [Setup](#setup)
-  - [Sample code](#sample-code)
+  - [Example](#example)
     - [Creating a client](#creating-a-client)
     - [Creating an index](#creating-an-index)
     - [Adding a document to an index](#adding-a-document-to-an-index)
@@ -9,8 +9,8 @@
     - [Searching for a document](#searching-for-a-document)
     - [Deleting a document](#deleting-a-document)
     - [Deleting an index](#deleting-an-index)
-  - [Making API calls](#making-api-calls)
-    - [Point in time API](#point-in-time-api-calls)
+  - [Making API Calls](#making-api-calls)
+    - [Point in Time API](#point-in-time-api)
   - [Using plugins](#using-plugins)
     - [Alerting plugin](#alerting-plugin)
       - [**Searching for monitors**](#searching-for-monitors)
@@ -390,7 +390,9 @@ Refer the AWS documentation regarding usage of IAM credentials to sign requests
 
 Opensearch-py client library also provides an in-house IAM based authentication feature, `AWSV4SignerAuth` that will help users to connect to their opensearch clusters by making use of IAM roles.
 
-##### Pre-requisites to use `AWSV4SignerAuth`
+`AWSV4SignerAuth` uses RequestHttpConnection as transport class for communication with opensearch clusters. Opensearch-py client library provides `pool_maxsize` option to modify default connection-pool size.
+
+#### Pre-requisites to use `AWSV4SignerAuth`
  - Python version 3.6 or above,
  - Install [botocore](https://pypi.org/project/botocore/) using pip
 
@@ -413,7 +415,8 @@ client = OpenSearch(
     http_auth = auth,
     use_ssl = True,
     verify_certs = True,
-    connection_class = RequestsHttpConnection
+    connection_class = RequestsHttpConnection,
+    pool_maxsize = 20
 )
 
 q = 'miller'

opensearchpy/connection/http_requests.py

@@ -66,6 +66,8 @@ class RequestsHttpConnection(Connection):
     :arg http_compress: Use gzip compression
     :arg opaque_id: Send this value in the 'X-Opaque-Id' HTTP header
         For tracing all requests made by this transport.
+    :arg pool_maxsize: Maximum connection pool size used by pool-manager
+        For custom connection-pooling on current session
     """
 
     def __init__(
@@ -82,6 +84,7 @@ def __init__(
         headers=None,
         http_compress=None,
         opaque_id=None,
+        pool_maxsize=None,
         **kwargs
     ):
         if not REQUESTS_AVAILABLE:
@@ -94,6 +97,14 @@ def __init__(
         for key in list(self.session.headers):
             self.session.headers.pop(key)
 
+        # Mount http-adapter with custom connection-pool size. Default=10
+        if pool_maxsize and isinstance(pool_maxsize, int):
+            pool_adapter = requests.adapters.HTTPAdapter(
+                pool_maxsize=pool_maxsize
+            )
+            self.session.mount("http://", pool_adapter)
+            self.session.mount("https://", pool_adapter)
+
         super(RequestsHttpConnection, self).__init__(
             host=host,
             port=port,