readme.pulloperations

        Using the CIM/XML Pull Operations and FQL query Language

Date: 12 November 2014

STATUS 

The client pull operations and FQL query language are incorporated into
OpenPegasus 2.14. consistent and compatible with the DMTF specifications for
the pull operations (DMTF DSP0200 v 1.4, and DMTF DSP0212 v 1.01.
This readme defines the characteristics, limitations,
etc. for that implementation.

===========================================
OVERVIEW:

The operation extensions for pull operations defined in the DMTF specification
DSP0200 V 1.4 were implemented in OpenPegasus effective version 2.14
including Client and Server.

These operations extend the CIM/XML  individual operations to operation
sequences where the server must maintain state between operations in a
sequence and the client must execute multiple operations to get the full
set of instances or instance paths.

The following new CIM/XML operations as defined in DSP0200 are included;

    -openEnumerateInstances
    -openEnumerateInstancePaths
    -openReferenceInstances
    -openReferenceInstancePaths
    -openAssociatiorInstances
    -openAssociatorInstancePaths
    -openQueryInstances
    -pullInstancesWithPath
    -pullInstancePaths
    -pullInstances
    -closeEnumeration
    -enumerationCount (deprecated by DMTF and incomplete)

Since the concept of sequences of operations linked together (open, pull, close)
is a major extension to the original CIM/XML operation concept of completely
independent operations several new pieces of functionality are implemented
to control interOperation timeouts, counts of objects to be returned, etc.

NOTE: More detailed information on the pull operations is contained in:
1. The OpenPegasus wiki (pull operations work group)
2. The pull operations PEP (preliminary today)
3. Presentations in the OpenPegasus wiki, in particular in the
   pull operations workgroup documentation at:

https://wiki.opengroup.org/pegasus-wiki/doku.php?id=dev:workgroups:pulloperationsupportworkgroup:pull_operation_support_work_group
   

OPENPEGASUS CLIENT

The new operations follow the same pattern as the APIs for existing operations
in that:

1. All errors are handled as CIMException and Exception the same as the
   original client operations.  Note, however, that there are additional
   CIMStatusCodes for the new operations.

2. The means of inputting parameters are the same except that there are
   significantly more input parameters with the open operations and for the
   first time operations return parameters as well as objects in the
   response.  Specifically the open and pull operations return values for
   the enumerationContext argument which is the identity for a pull sequence and
   the endOfSequence argument which is the marker the server sends in open and
   pull responses when it has no more objects to send.

The significant differences between the open... and original enumerate,
associator, and reference operations includes:

1. The new pull client operations typically require multiple client operations
   to retrieve a complete set of data (ex. openEnumerateInstances and
   pullInstancesWithPath) are equivalent to the old enumerateInstances
   client request.  While the whole collection of responses might be returned
   on the open if the maxObjectCount argument is set larger than the the number
   of objects in the total response, depending on the speed of delivery of
   objects from providers, the response might still not deliver everything
   in the initial response.  The client code writer should always assume that
   multiple requests will be required.

2. Processing of parameters on responses (i.e. the endOfSequence and
   enumerationContext arguments are returned for open and pull operations)
   These return arguments are used to control the loop retrieving data
   for an enumeration sequence (ex. the sequence of OpenEnumerateInstances
   and PullInstancesWithPath that represents a complete enumeration
   sequence). The sequence continues until the server responds with
   EndOfSequence is true, and the enumerationContext is the identifier
   for the sequence.

3. Numeric arguments (Uint32 and Uint64) include the option of NULL in some
   cases so they are packaged inside classes Uint32Arg and Uint64Arg in the
   client API.

4. The openAsociatorInstances and openReferenceInstances operations ONLY
   process instances.  They do not include the capability to return classes
   as reference and associator requests do and therefore return CIMInstance
   rather than CIMObject.

5. Paths are returned for the instance operations as part of the returned
   instances (ex. openEnumerateInstances and pullInstancesWithPath) where
   they were not with EnumerateInstances.

6. The client must maintain state between operations in an enumeration
   sequence (using the enumerationContext parameter returned with open
   and pull responses). The client must always return the last
   enumerationContext received as the server may modify the
   enumerationContext for each response in an enumeration sequence.

The client API is defined in the header file:

    pegasus/src/Pegasus/Client/CIMCLient.h


OPENPEGASUS SERVER

The OpenPegasus server attempts to always deliver at least some objects
in a response.  It does not wait to deliver the exact number requested but
waits to deliver at least some. The sever will wait about 15 seconds to
deliver and if there is nothing delivered from the providers it will then
return a response with zero objects in it so that the client does not
timeout.  This would only occur if the providers are extremely slow (i.e.
greater than 15 seconds) in preparing response objects to be delivered.

The OpenPegasus server always closes an enumeration sequence upon receipt of any
error from the providers, repository, etc. Therefore the server will reject
any request that has the continueOnError argument = true;

Expansion to allow the continue on error may be added in a future version.
In any case, the whole purpose of the continue on error is really to allow
input from good providers to be mixed with providers that return errors so
that generally this would mean simply changing the logic in the return mechanism
to not shutdown when an error is received from any given provider.

Generally we do not believe that the providers need to do much more in the
future to support the continueOnError other than possibly allowing the provider
to continue processing after it has received an error.

OPENPEGASUS PROVIDERS

This implementation (OpenPegasus version 2.14) requires NO changes to the
existing providers.  The provider APIs operate just as they do with the
original operations.

Because the server processing is different however, there may be some
behavior differences primarily because the client now controls the speed of
delivery of objects.

In previous versions of Pegasus, the server attempted to deliver objects as
rapidly as then can be put on the network.  In the case of HTTP chunked requests
they are delivered in chunks of about 100 objects. The primary delay for the
providers was the processing of each segment through the server.  The server
is blocked so that no other segment can proceed through the server until that
segment is processed and sent on the network.
In the case of non-chunked responses, they are completely gathered in the serve
and then delivered as one non-chunked response. There were no delays for the
providers, just lots of possible memory use in the server.

The responses from providers (delivered through the deliver(...) interface) are
gathered into segments of about 100 objects and this group of objects is moved
through the server to be delivered to the client.

However with the inclusion of the pull operations, the segments (CIMResponseData
objects containing the instance or path objects) from the providers are cached
in the server response cache until the next pull request and that number
is returned in the response to that pull. Thus, if the client is slow to issue
pull requests, the providers might be delayed at some point to reduce memory
usage in the server (the delay appears as slow response to the deliver operation).

In other words, the time to process large sets of responses from the provider
now depends on the speed of handling the client.

It is important to remember in developing providers that the OpenPegasus server
can most efficiently process responses if they are passed from the provider
to the server individually or in small arrays of objects rather than the
provider gathering very large arrays of objects and sending them to the
server.

FQL (FILTER QUERY LANGUAGE)

The FQL implementation is complete in accord with the DMTF
specification DSP 0212 except for a few issues including:

1. The regex for the LIKE operation is the same as CQL basic. It provides
   only the following special characters   "." and  "*".
2. The implementation does not include the comparison of
   embeddedInstances functionality.
3. The implementation does not include the handling of Uint8[] as
   strings.
4. Since the cimserver provides the filtering, it can only filter properties
   that are returned from the providers.  Therefore all of the properties in
   the filter MUST BE included in any property list provided with the request.

For more details on the FQL implementation see the readme.txt in the
    directory pegasus/src/Pegasus/FQL.

OpenPegasus will remove these limitations in a future version.

The FQL implementation includes a large set of sample queries in the
directory:

    pegasus/src/Pegasus/FQL/tests/Parser

including both good and error generating queries.

==================================================================
LIMITATIONS IN OPENPEGASUS 2.14

1. The openQueryInstances does not allow requesting the class on response
   (i.e. the returnQueryResultClass argument must be false in this version).
   Since OpenPegasus is actively proposing that this argument be deprecated
   we will see what happens in future releases.

2. The openEnumerateInstanceNames, openAssociatorNames, and openReferenceNames
   do not allow use of the query filter.  This is because:
   a. The intention is to deprecate these operations completely and remove
      them in a future versionof both the specificatons and OpenPegasus.
   b. They require that the server call the providers with the corresponding
      enumerate, associators, references to get the full instances to
      filter and then remap this to the corresponding Names operation.
      We propose that if the user wants just the paths, this can be achieved
      with the instances operation with the propertylist set empty which
      requests that the server return no properties.

3. The filterQuery filtering is done by the server, not the providers in this
   version of OpenPegasus. This will be modified in a future version of
   OpenPegasus when the provider API extensions for the pull operations
   have been resolved.  Version 2.1 of the CMPI specification will resolve
   this issue.  However, this has imposed another limitation as mentioned above,
   the properties that are part of the filter MUST BE included in any
   propertyList in the request.  OpenPegasus does not check to be sure that
   all properties in the filter are also in the request and would therefore
   try to filter the response on non-existent properties.

4. The input parameter continueOnError is processed correctly by the client
   but the Pegasus server only provides code for 'false' value, since the server
   does not include logic to continue processing responses after an error is
   encountered.
   This is consistent with the statement in the specification that use of
   this functionality is optional and the fact that the DMTF agrees that all
   of the issues of continuing after errors have not been clarified.

5. The operation enumerationCount is not processed by the server today since
   a) Getting the count would be the same cost as the corresponding
   enumeration, b) the server does not include a history or estimating
   mechanism for this to date.
   NOTE: After a through review as part of the development of the next version
   of CMPI we have concluded that this operation is probably not worth the
   effort.  Since it is optional, Pegasus will only return the unknown status
   at this point. Further it is the intention of the DMTF to deprecate this
   function.

===================================================================
PULL OPERATION CONFIGURATION PARAMETERS

The server includes several configuration parameters to set limits on the
processing of pull operations.

RUNTIME CONFIGURATION PARAMETERS

1. Maximum value of interoperation time (pullOperationsMaxTimeout) -
This parameter defines the maximum time allowed between the return of an open
or pull response and the receipt of the next pull or a close operation before
the server may close the enumeration.
The specification allows the server to set a maximum interoperation time and
refuse open requests that with requested operationTimeout greater than that
time.

2. Maximum number of objects returned in a single open or pull operation
(pullOperationsMaxObjectCount) - The server can set a maximum limit on the
number of objects that can be returned in a single open or pull operation
with the maxObjectCount parameter. This parameter sets the maximum limit.
The absolute maximum allowed without recompiling (and changing a value in
Common/Constants.h) is 10000.

3. Default operationTimeout (pullOperationsdefaultTimeout) - If the client
does not specify an operation timeout in the open request, the server uses
the value defined by this runtime configuration variable.  The default
is 30 seconds.

STATISTICS

The server does maintain some statistics on the pull operations and outputs
these to the console when the server is shutdown.  In this version there
is no way to output these statistics other than to the console and at
shutdown.

COMPILE TIME CONFIGURATION PARAMETERS

1. Whether the server allows 0 as an interoperation timeout value - The value
zero is a special value for the interoperationTimeout in that it tells the
server to not timeout the enumeration sequence. This should never be used
as it allows a client to open enumeration sequences that will not be
cleaned up if the client does not properly terminate them.  There is a
compile time variable in CIMOperationRequestDispatcher that would allow this
to be set so 0 interoperation is allowed but we recommend that it never
be used since it removes a significant component of the server management
of enumerationContext information.

With this value for interoperationTimeout, the only way to close an
enumeration sequence is to complete all of the pulls or issue the close.
If for some reason the sequence is not completed, that enumeration context
would remain open indefinitely.  Since in Pegasus any open enumeration
context uses resources (the context object and any provider responses that
have not yet been issued in a response) it would appear that most
platforms would not want to allow the existence of enumeration contexts
that cannot be closed by the server.

2. Maximum consecutive pull requests with 0 maxObjectCount - The use of the
pull operation with maxObjectCount set to zero could be used to keep an
enumeration context open indefinitely (this tells the server to restart the
interoperationTimeout but not send any objects in the response). Therefore the
specification allows for the server setting maximum limits on this behavior
and returning the error CIM_ERR_SERVER_LIMITS_EXCEEDED if this limit is
exceeded.

Note that this is maximum CONSECUTIVE pulls so that issuing a pull with
a non-zero count resets this counter.

Pegasus sets the value of this limit to 1000 and allows the implementer to
modify it by compiling  with the PEGASUS_MAXIMUM_ZERO_OBJECTCOUNT define  in
CIMOperationRequestDispatcher.cpp modified.

3. Time to wait for next response from providers - In the case where providers
are responding very slowly, the goal is to generate responses with
zero instances at regular intervals to allow the client to keep the
enumeration context alive.  This wait time is set by a compile time define
(PEGASUS_PULL_MAX_OPERATION_WAIT_SEC)in pegasus/src/Pegasus/Common/Constants.h
and is set to 15 seconds in the current release.

4. Time to wait before killing off an enumeration context that is blocked
by missing provider responses. In the rare case where providers do not complete
their responses to the server there is a compile-time counter that trys to
clean up the providers and finally just kill the enumeration sequence after
a defined number of consecutive pulls that return zero objects.
The limit is defined in pegasus/src/Pegasus/Constants.h

NOTE: The development team is trying to consolidate all such constants and
#define definitions that control overall server characteristics but are not
runtime parameters in Constants.h

===================================================================
TESTING PULL OPERATIONS

The pull operations are tested primarily with two client programs in the
directory Pegasus/Client/tests

1. pullop and it corresponding Makefile provide extensive tests of the
   pull operations and comparison of the results with the corresponding non
   pull operations.

2. PullErrors  tests a number of error scenarios with the pull operations.

3. cimcli has been extended to allow execution of the pull operations with new
   operations that parallel the existing operations:
   enumerateinstances(ei) - corresponding is pullenumerateInstances(pei)
   ni -> pni   enumerateInstanceNames
   r -> pr     references
   rn -> prn   referenceNames
   a  -> pa    associators
   ar -> par   associatorNames

   These execute complete pull sequences (open, pull) with cimcli options
   to control parameters like maxObjectCount, interoperation Timeout, etc.


=============================================================
TODO LIST - Post 2.14 release
   1. Binary operation from OOP.  Need to add counter to binary
      protocol to be able to count objects in response. Generates
      warnings in things like messageserializer and does not work with
      OOP right now.  Fixed by converting to XML. Concluded that we do not
      need to do this. The binary response is not really used often
      in the current environment So double mapping it is not a major issue.
      Leave this as FUTURE
   2. Minor TODOs, diagnostics, etc. still in the code. Reduced almost to
      none now.  We are leaving some in as PEGASUS_DEBUG
   3. Extension to avoid double move of objects in CIMResponseData (one
      into enumerationContext queue and second to new CIMResponseData for
      response.  Want to avoid second move by extending open/pull response
      messages to include count and CIMResponse data to count objects out
      of queue when converting (avoids the second move). This would mean
      extending the output writers to allow the count field to be supplied
      so they would only create output for up to the count supplied.(Make
      this future beyond bug 9676). This is fairly extensive because it
      extends beyond CIMResponseData to SCMO and XML writers where the
      XmlWriters used by encodeXmlResponse would have to have counters
      added. Then instead of copying on getCache we would simply pass the
      cache and count on and the writer would take and remove.
   4. Add more static tests (currently only OpenEnumerateInstances and
      OpenEnumerateInstanceNames covered).
   5. Correct issue between operations and HTTP where we are sending
      trailers with exceptions. Modify response mechanisms so that we
      set non-chunked for all responses where we send error responses to
      avoid the trailers. NOTE: There should be now a bug on this in general
      where we would want to send an initial error without the trailer. Should
      have always done that.
   6. It would be more efficient in dispatcher to always use exception for
       rejects and change the _reject functions so that they never return
       when they reject. This is VERY LOW PRIORITY and primarily saves
       a few lines of code in the reject functions and their calls.  Means we
       would code.
       _rejectIfEnumerationToBroad(...);
       _rejectIfThisParameterIncorrect(...);

       instead of
       if (_rejectIfEnum...)
       {
           return true
       }
       It would mean that the method trace for the handlers would not return
       an exit if we rejected. VERY LOW PRIORITY. Possibly FUTURE. No behavior
       change, just more compact source code but it messes with the method
       trace logic.
    7. There are still a couple of template functions around the task of
       distributing requests to the multiple providers.
       In fact there are two similar but different templates for
       the associators(i.e. assoc, ref, etc.) functions and the parallel
       openAssoc functions.  It would be nice to consolidate that logic and
       further to try to create a non-template form for those functions. LOW
       PRIORITY
    8. Trace functions in CIMResponseData,h & cpp should be PEGASUS_DEBUG.
       Same for trace function in EnumerationContext and
       EnumerationContextTable
    9. Question. We added trace in CIMRequestOperationDispatcher.cpp if
       query enum is recevied with no object path.  However, since this is
       a provider problem should we be doing something other than a trace
       (ex. log) since traces are often not really used and they do generate
       lots of data whereby something like this could get lost. Also, need
       to review what level of trace if we keep trace.
   10. Better statistics keeping for open, etc. through cimperf.
   11. Incorporate a simple test of pull operations into the TestClient
   12. Define a simpler iteration interface for the client so that
       the client does not have to directly control the open, pull, etc.
       operations.
   13. Map properties in the FQL Filter to properties in any property list
       provided.
   14. Output statistical information on a regular basis rather than just
       as server shutdown and to the log or somewhere else than just the
       console.