decimal type crash on select, decimal truncation, unable to use decimal with precision of 31 with Ingres pre 9.2.

Change is in two parts:

1) use the correct (string) length along with precision/scale for
decimal data retrieved from the DBMS

2) use the correct (string) length along with precision/scale for
decimal data sent to the DBMS

Most of the change is on removing the need for arithmetic on "fetch"
from database (and send), arithmetic is done once and the same value
is used to get data from the DBMS by puting the correct values into
the IIDBI_DESCRIPTOR. The IIDBI_DESCRIPTOR is set for Bind (send)
and fetch (get) operations. The IIDBI_DESCRIPTOR values is then
used at the time the ODBC call is made, no math is issued at this
point in time.

The new get/send code that was added for the Decimal type does not
have any code that is specific to decimals. The decimal specific
code is in the describe code. This is stage 1 of a multi stage
code re-factor, to use the cType and internal size for all types
and then use it everywhere where that information is needed. All
types will set the size and type and then the get code will be a
single block of code that uses the descriptor information.

Testing Done:

All decimal tests in tets suite now pass. I.e.:

    test_decimalCrash passes (test suite no longer crashes)
    test_decimalLongValues passes
    test_decimalBindLongValues now passes (and does not generate ADF
        errors) with pre Ingres 9.2 versions of Ingres (i.e. versions
        where 31 is the maximum precision for decimal type).



git-svn-id: http://code.ingres.com/ingres/drivers/python/main@1880 45b5d43f-8932-4c86-835a-3654e5842839

Author: clach04

dbi/iidbicurs.c

@@ -1,4 +1,5 @@
 /*
+** vim:filetype=c:ts=4:sw=4:et:nowrap 
 ** Copyright (c) 2008 Ingres Corporation
 **
 ** This program is free software; you can redistribute it and/or modify
@@ -134,8 +135,32 @@
 **          trace files.
 **      05-Aug-2009 (Chris.Clark@ingres.com)
 **          Removed unused dbi_cursorFetchall()
+**      19-Aug-2009 (Chris.Clark@ingres.com) 
+**          Added vim hints. 
+**          Fixed bug with select of decimal values. 
+**          Leave "precision" alone, precision means something and should 
+**          not be altered. Now set internalSize to the size of the buffer
+**          space for decimal on fetch. Ensure the correct (max) string
+**          length is used for fetching decimal values from the DBMS. Added
+**          new macro MAX_STRING_LEN_FOR_DECIMAL_PRECISION for calculating
+**          string buffer length.
 **/
 
+/* 
+** Decimal values are sent/received as strings. 
+** Define macro that calculates the largest string needed for 
+** a decimal with a given precision. 
+** 
+** max string length/size of decimal value is based on: 
+**      precision (not the precision/scale encoded combo) for digits 
+**      +1 for NULL terminator 
+**      +1 for period/dot/comma 
+**      +1 for possible negative sign. 
+**      +1 for possible leading zero where precision=scale,
+**          e.g Ingres literal Decimal(0.1, 1, 1)
+*/ 
+#define MAX_STRING_LEN_FOR_DECIMAL_PRECISION(x) (x+4)
+
 RETCODE BindParameters(IIDBI_STMT *pstmt, unsigned char isProc);
 
 /*{
@@ -1147,6 +1172,38 @@ dbi_cursorFetchone( IIDBI_STMT *pstmt )
                 }
                 break;
 
+            case SQL_DECIMAL: 
+                /* Start of generic get code */
+                rc = SQLGetData(pstmt->hdr.handle, i+1, 
+                                     pstmt->descriptor[i]->cType,
+                                     pstmt->descriptor[i]->data, 
+                                     pstmt->descriptor[i]->internalSize,
+                                     &orind); 
+                if (orind == SQL_NULL_DATA) 
+                    pstmt->descriptor[i]->isNull = 1; 
+                else 
+                    pstmt->descriptor[i]->isNull = 0; 
+                if (rc == SQL_NO_DATA) 
+                    pstmt->fetchDone = TRUE; 
+                if (SQL_SUCCEEDED(rc) || rc == SQL_NO_DATA) 
+                { 
+                    rc = DBI_SQL_SUCCESS; 
+                    break; 
+                } 
+                else 
+                { 
+                    return_code = IIDBI_ERROR( rc, NULL, NULL,  
+                        pstmt, &pstmt->hdr.err ); 
+                    DBPRINTF(DBI_TRC_STAT) 
+                    ( "%d = dbi_cursorFetchone (%d) %s %s %x\n\n", 
+                        rc, __LINE__, pstmt->hdr.err.sqlState,  
+                        pstmt->hdr.err.messageText,  
+                        pstmt->hdr.err.native); 
+                    return return_code; 
+                } 
+                break; 
+ 
+
             default:            
                 rc = SQLGetData(pstmt->hdr.handle, i+1, SQL_C_CHAR, 
                                      pstmt->descriptor[i]->data,
@@ -1605,6 +1662,8 @@ BindParameters(IIDBI_STMT *pstmt, unsigned char isProc)
     void *data;
     SQLUINTEGER precision;
     SQLSMALLINT scale;
+    SQLSMALLINT cType;
+    SQLLEN internalSize;
     unsigned char isNull;
 
     DBPRINTF(DBI_TRC_ENTRY)("%p: Begin BindParameters\n", pstmt);
@@ -1626,6 +1685,8 @@ BindParameters(IIDBI_STMT *pstmt, unsigned char isProc)
         precision = pstmt->parameter[i]->precision;
         scale = pstmt->parameter[i]->scale;
         isNull = pstmt->parameter[i]->isNull;
+        cType = pstmt->parameter[i]->cType;
+        internalSize = pstmt->parameter[i]->internalSize;
         /*
         ** make orind persistent beyond this call by using the field
         ** in the parameter descriptor structure.
@@ -1700,9 +1761,30 @@ BindParameters(IIDBI_STMT *pstmt, unsigned char isProc)
             }
             break; 
 
+        case SQL_DECIMAL:
+            /* Start of generic BindParameters code */
+            if (isNull)
+                *orind = SQL_NULL_DATA;
+            else
+                *orind = SQL_NTS;
+            rc = SQLBindParameter(pstmt->hdr.handle, i+1, SQL_PARAM_INPUT, 
+                cType, type, precision, scale, data, internalSize , orind);
+
+            if (rc != SQL_SUCCESS) 
+            {
+                return_code = IIDBI_ERROR( rc, NULL, NULL, pstmt->hdr.handle, 
+                    &pstmt->hdr.err );
+
+                DBPRINTF(DBI_TRC_STAT)
+                    ( "%d = SQLBindParameter (%d) %s %s %x\n    %s\n",
+                    rc, __LINE__, pstmt->hdr.err.sqlState, 
+                    pstmt->hdr.err.messageText, pstmt->hdr.err.native, 0 );
+                return return_code;
+            }
+            break; 
+
         case SQL_CHAR:
         case SQL_VARCHAR:
-		case SQL_DECIMAL:
             DBPRINTF(DBI_TRC_STAT)("CHAR or VARCHAR\n");
             if (isNull)
                 *orind = SQL_NULL_DATA;
@@ -1921,11 +2003,13 @@ RETCODE dbi_describeColumns (IIDBI_STMT *pstmt)
     SQLINTEGER internalSize;
     SQLINTEGER displaySize; 
     SQLUINTEGER precision;  
+    int cType;
 
     numCols = pstmt->descCount;
 
     for (i = 0; i < numCols; i++)
     {
+        cType=SQL_C_CHAR; /* Default */
         rc = SQLDescribeCol(hstmt, i+1, colName, 256, &cbColName, &type, 
             &precision, &scale, &nullable);
         if (!SQL_SUCCEEDED(rc))
@@ -1949,11 +2033,7 @@ RETCODE dbi_describeColumns (IIDBI_STMT *pstmt)
         case SQL_DATE:
         case SQL_TIME:
             pstmt->descriptor[i]->precision = 26;
-            break;
-
-        case SQL_NUMERIC:
-        case SQL_DECIMAL:
-            pstmt->descriptor[i]->precision = 34;
+            cType=SQL_C_TIMESTAMP;
             break;
 
         case SQL_LONGVARCHAR:
@@ -1977,6 +2057,15 @@ RETCODE dbi_describeColumns (IIDBI_STMT *pstmt)
         }
         if (displaySize == MAX_DISPLAY_SIZE) /* if set at max, default to -1 */
             displaySize = -1;
+        switch (pstmt->descriptor[i]->type)
+        {
+            case SQL_NUMERIC:
+            case SQL_DECIMAL:
+                internalSize = MAX_STRING_LEN_FOR_DECIMAL_PRECISION(precision);
+                cType = SQL_C_CHAR;
+                break;
+
+            default:
         rc = SQLColAttribute(hstmt, i+1, SQL_DESC_OCTET_LENGTH, 0, 0, NULL, 
             &internalSize);
         if (!SQL_SUCCEEDED(rc))
@@ -1989,13 +2078,16 @@ RETCODE dbi_describeColumns (IIDBI_STMT *pstmt)
                         pstmt->hdr.err.native);
             return return_code;
         }
-        DBPRINTF(DBI_TRC_STAT)("For Col %d, type is %d, name is %s, precision is %d, scale is %d display size is %d internal size is %d and nullable is %d\n", i, type, colName, precision, scale, displaySize, internalSize, nullable); 
+                break;
+        }
+        DBPRINTF(DBI_TRC_STAT)("For Col %d, type is %d (cType is %d), name is %s, precision is %d, scale is %d display size is %d internal size is %d and nullable is %d\n", i, type, cType, colName, precision, scale, displaySize, internalSize, nullable); 
          pstmt->descriptor[i]->scale = scale;
          pstmt->descriptor[i]->nullable = nullable;
          pstmt->descriptor[i]->columnName = calloc(1,cbColName + 1);
          strcpy(pstmt->descriptor[i]->columnName, (char *)colName);
          pstmt->descriptor[i]->displaySize = displaySize;
          pstmt->descriptor[i]->internalSize = internalSize;
+         pstmt->descriptor[i]->cType = cType;
      }
      return DBI_SQL_SUCCESS;
 }

dbi/ingresdbi.c

@@ -1,4 +1,5 @@
 /*
+** vim:filetype=c:ts=4:sw=4:et:nowrap 
 ** Copyright (c) 2008 Ingres Corporation
 **
 ** This program is free software; you can redistribute it and/or modify
@@ -214,6 +215,12 @@
 **      Added correct counter increment to Py_None
 **      simplified NULL logic into one place by
 **      removing duplicate code.
+**  19-Aug-2009 (Chris.Clark@ingres.com)
+**      Compile warnings clean up of "unreferenced local variable" (win32
+**      specific).
+**      Set accurate decimal precision/scale/internalSize information
+**      for bind parameters.
+**      Added trace information for bind parameters as they are processed.
 **/
 
 static PyObject *IIDBI_Warning;
@@ -1151,9 +1158,11 @@ static PyObject * IIDBI_connect(PyObject *self, PyObject *args,
     RETCODE rc=DBI_SQL_SUCCESS;
     int i;
     int result = FALSE;
+#ifndef WIN32
     struct stat buf;
     char odbcconfig[MAX_PATH];
     char *ii_system;
+#endif /* WIN32 */
     static char *kwlist[] = 
     {
         "dsn", "database", "vnode", "uid", "pwd", "autocommit", "selectloops",
@@ -6200,17 +6209,35 @@ int IIDBI_sendParameters(IIDBI_CURSOR *self, PyObject *params)
         else if (PyObject_IsInstance(elem, decimalType))
         {
             char *decimal = NULL;
+            char *tmp_str = NULL;
             parameter[i]->data = 
                 strdup(PyString_AsString(PyObject_Str(elem)));
+            if (parameter[i]->data == NULL)
+            {
+                Py_XDECREF(params);
+                exception = IIDBI_InternalError;
+                errMsg = "strdup returned NULL";
+                result = IIDBI_handleError((PyObject *)self, exception, errMsg);
+                goto errorExit;
+            }
             parameter[i]->type = SQL_DECIMAL;
             parameter[i]->cType = SQL_C_CHAR;
-            parameter[i]->precision =
-                (int)strlen((char *)parameter[i]->data);
+            parameter[i]->internalSize = (int)strlen((char *)parameter[i]->data) + 1; /* +1 for NULL terminator */
+            tmp_str = (char *)parameter[i]->data;
+            /* Ignore leading signs and leading zeros */
+            while (*tmp_str!='\0' && (*tmp_str=='-' || *tmp_str=='0')) /* check for '+' too? Not seen one before */
+            {
+                tmp_str++;
+            }
+            parameter[i]->precision = (int)strlen(tmp_str);
             decimal = strstr(parameter[i]->data,".");
             if (!decimal)
                 decimal = strstr(parameter[i]->data, ",");
             if (decimal)
+            {
                 decimal++;
+                parameter[i]->precision--; /* do not count decimal seperator as part of precision */
+            }
             if (decimal)
                 parameter[i]->scale = (int)strlen(decimal);
             else
@@ -6227,6 +6254,7 @@ int IIDBI_sendParameters(IIDBI_CURSOR *self, PyObject *params)
             goto errorExit;
         }
         Py_DECREF(elem);
+        DBPRINTF(DBI_TRC_STAT)("For bind %d, type is %d (cType is %d), precision is %d, scale is %d internal size is %d and nullable is %d\n", i, parameter[i]->type, parameter[i]->cType, parameter[i]->precision, parameter[i]->scale, parameter[i]->internalSize, parameter[i]->isNull);
     } /* for (i = 0; i < parmCount; i++) */
 
     DBPRINTF(DBI_TRC_RET)("%p: IIDBI_sendParameters }}}1\n", self);

hdr/iidbi.h

@@ -1,4 +1,5 @@
 /*
+** vim:filetype=c:ts=4:sw=4:et:nowrap
 ** Copyright (c) 2008 Ingres Corporation
 **
 ** This program is free software; you can redistribute it and/or modify
@@ -267,14 +268,14 @@ typedef struct
 typedef struct _IIDBI_DESCRIPTOR
 {
     char            *columnName;
-    int             type;
-    int             cType;
+    int             type;  /* ODBC DBMS type */
+    int             cType; /* The ODBC C type used for "transport" */
     int 	    precision;
     int 	    scale;
     void            *data;
     int             nullable;
     int             displaySize;
-    int             internalSize;
+    int             internalSize; /* Used for internal memory malloc */
     unsigned char   isNull;
     long            orInd;
 } IIDBI_DESCRIPTOR;