FEAT: Adding Ouput Converter APIs (#190)

### Work Item / Issue Reference  
<!-- 
IMPORTANT: Please follow the PR template guidelines below.
For mssql-python maintainers: Insert your ADO Work Item ID below (e.g.
AB#37452)
For external contributors: Insert Github Issue number below (e.g. #149)
Only one reference is required - either GitHub issue OR ADO Work Item.
-->

<!-- mssql-python maintainers: ADO Work Item -->
>
[AB#34913](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/34913)
>
[AB#34914](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/34914)
>
[AB#34915](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/34915)
>
[AB#34916](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/34916)

-------------------------------------------------------------------
### Summary   
This pull request adds a flexible output converter system to the
database connection, allowing custom Python functions to be registered
for converting SQL data types when fetching results. It introduces new
methods for managing these converters and updates the row handling logic
to apply them automatically. Comprehensive tests are added to verify
correct registration, retrieval, removal, and integration of output
converters, including edge cases and chaining behavior.

**Core output converter system:**

* Added methods to `Connection` (`add_output_converter`,
`get_output_converter`, `remove_output_converter`,
`clear_output_converters`) for registering and managing output converter
functions for specific SQL types. These converters are called when
values of the registered SQL type are read from the database.
(`mssql_python/connection.py`)
* Updated the `Row` class to apply registered output converters
automatically to values fetched from the database, with fallback logic
for string types and robust error handling. (`mssql_python/row.py`)

**Testing and validation:**

* Added extensive tests for output converter management, including
adding, retrieving, removing, clearing, chaining, temporary replacement,
and integration during data fetching. Tests also cover edge cases such
as handling `NULL` values and using multiple converters at once.
(`tests/test_003_connection.py`)
* Introduced helper converter functions for specific SQL types (e.g.,
`DATETIMEOFFSET` and custom string handling) to support and validate the
new converter system in tests. (`tests/test_003_connection.py`)

**Test and utility enhancements:**

* Imported additional modules (`struct`, `datetime`, `timezone`, and
constants) to support new test cases and converter logic.
(`tests/test_003_connection.py`)

---------

Co-authored-by: Jahnvi Thakkar <jathakkar@microsoft.com>

Author: jahnvi480

mssql_python/connection.py

@@ -14,6 +14,7 @@
 import re
 import codecs
 from typing import Any
+import threading
 from mssql_python.cursor import Cursor
 from mssql_python.helpers import add_driver_to_connection_str, sanitize_connection_string, sanitize_user_input, log
 from mssql_python import ddbc_bindings
@@ -187,6 +188,10 @@ def __init__(self, connection_str: str = "", autocommit: bool = False, attrs_bef
         # TODO: Think and implement scenarios for multi-threaded access to cursors
         self._cursors = weakref.WeakSet()
 
+        # Initialize output converters dictionary and its lock for thread safety
+        self._output_converters = {}
+        self._converters_lock = threading.Lock()
+
         # Auto-enable pooling if user never called
         if not PoolingManager.is_initialized():
             PoolingManager.enable()
@@ -531,6 +536,92 @@ def cursor(self) -> Cursor:
         cursor = Cursor(self)
         self._cursors.add(cursor)  # Track the cursor
         return cursor
+    
+    def add_output_converter(self, sqltype, func) -> None:
+        """
+        Register an output converter function that will be called whenever a value 
+        with the given SQL type is read from the database.
+        
+        Thread-safe implementation that protects the converters dictionary with a lock.
+        
+        ⚠️ WARNING: Registering an output converter will cause the supplied Python function 
+        to be executed on every matching database value. Do not register converters from 
+        untrusted sources, as this can result in arbitrary code execution and security 
+        vulnerabilities. This API should never be exposed to untrusted or external input.
+        
+        Args:
+            sqltype (int): The integer SQL type value to convert, which can be one of the 
+                          defined standard constants (e.g. SQL_VARCHAR) or a database-specific 
+                          value (e.g. -151 for the SQL Server 2008 geometry data type).
+            func (callable): The converter function which will be called with a single parameter,
+                            the value, and should return the converted value. If the value is NULL
+                            then the parameter passed to the function will be None, otherwise it 
+                            will be a bytes object.
+        
+        Returns:
+            None
+        """
+        with self._converters_lock:
+            self._output_converters[sqltype] = func
+            # Pass to the underlying connection if native implementation supports it
+            if hasattr(self._conn, 'add_output_converter'):
+                self._conn.add_output_converter(sqltype, func)
+        log('info', f"Added output converter for SQL type {sqltype}")
+    
+    def get_output_converter(self, sqltype):
+        """
+        Get the output converter function for the specified SQL type.
+        
+        Thread-safe implementation that protects the converters dictionary with a lock.
+        
+        Args:
+            sqltype (int or type): The SQL type value or Python type to get the converter for
+            
+        Returns:
+            callable or None: The converter function or None if no converter is registered
+    
+        Note:
+            ⚠️ The returned converter function will be executed on database values. Only use
+            converters from trusted sources.
+        """
+        with self._converters_lock:
+            return self._output_converters.get(sqltype)
+
+    def remove_output_converter(self, sqltype):
+        """
+        Remove the output converter function for the specified SQL type.
+        
+        Thread-safe implementation that protects the converters dictionary with a lock.
+        
+        Args:
+            sqltype (int or type): The SQL type value to remove the converter for
+            
+        Returns:
+            None
+        """
+        with self._converters_lock:
+            if sqltype in self._output_converters:
+                del self._output_converters[sqltype]
+                # Pass to the underlying connection if native implementation supports it
+                if hasattr(self._conn, 'remove_output_converter'):
+                    self._conn.remove_output_converter(sqltype)
+        log('info', f"Removed output converter for SQL type {sqltype}")
+    
+    def clear_output_converters(self) -> None:
+        """
+        Remove all output converter functions.
+        
+        Thread-safe implementation that protects the converters dictionary with a lock.
+        
+        Returns:
+            None
+        """
+        with self._converters_lock:
+            self._output_converters.clear()
+            # Pass to the underlying connection if native implementation supports it
+            if hasattr(self._conn, 'clear_output_converters'):
+                self._conn.clear_output_converters()
+        log('info', "Cleared all output converters")
 
     def execute(self, sql: str, *args: Any) -> Cursor:
         """

mssql_python/row.py

@@ -24,7 +24,13 @@ def __init__(self, cursor, description, values, column_map=None):
             column_map: Optional pre-built column map (for optimization)
         """
         self._cursor = cursor
-        self._values = values
+        self._description = description
+        
+        # Apply output converters if available
+        if hasattr(cursor.connection, '_output_converters') and cursor.connection._output_converters:
+            self._values = self._apply_output_converters(values)
+        else:
+            self._values = values
 
         # TODO: ADO task - Optimize memory usage by sharing column map across rows
         # Instead of storing the full cursor_description in each Row object:
@@ -42,6 +48,57 @@ def __init__(self, cursor, description, values, column_map=None):
 
         self._column_map = column_map
 
+    def _apply_output_converters(self, values):
+        """
+        Apply output converters to raw values.
+        
+        Args:
+            values: Raw values from the database
+            
+        Returns:
+            List of converted values
+        """
+        if not self._description:
+            return values
+        
+        converted_values = list(values)
+        
+        for i, (value, desc) in enumerate(zip(values, self._description)):
+            if desc is None or value is None:
+                continue
+            
+            # Get SQL type from description
+            sql_type = desc[1]  # type_code is at index 1 in description tuple
+            
+            # Try to get a converter for this type
+            converter = self._cursor.connection.get_output_converter(sql_type)
+            
+            # If no converter found for the SQL type but the value is a string or bytes,
+            # try the WVARCHAR converter as a fallback
+            if converter is None and isinstance(value, (str, bytes)):
+                from mssql_python.constants import ConstantsDDBC
+                converter = self._cursor.connection.get_output_converter(ConstantsDDBC.SQL_WVARCHAR.value)
+            
+            # If we found a converter, apply it
+            if converter:
+                try:
+                    # If value is already a Python type (str, int, etc.), 
+                    # we need to convert it to bytes for our converters
+                    if isinstance(value, str):
+                        # Encode as UTF-16LE for string values (SQL_WVARCHAR format)
+                        value_bytes = value.encode('utf-16-le')
+                        converted_values[i] = converter(value_bytes)
+                    else:
+                        converted_values[i] = converter(value)
+                except Exception:
+                    # Log the exception for debugging without leaking sensitive data
+                    if hasattr(self._cursor, 'log'):
+                        self._cursor.log('debug', 'Exception occurred in output converter', exc_info=True)
+                    # If conversion fails, keep the original value
+                    pass
+        
+        return converted_values
+
     def __getitem__(self, index):
         """Allow accessing by numeric index: row[0]"""
         return self._values[index]

requirements.txt

@@ -4,3 +4,4 @@ pybind11
 coverage
 unittest-xml-reporting
 setuptools
+psutil