Immediate Result and Global Result usage in Python

[felipemarkson]

Hi!

I have a question about the two ways to use DSS's APIs.

The dss_capi provides two modes of usage: Immediate Result (IR) and Global Result (GR).

These two modes are also imported on dss_python, but by default, only the Global Result is used:

https://github.com/dss-extensions/dss_python/blob/2dbc72ed875108d3f98d21cb0a488bab6b0d7f4c/dss/v7.py#L22-L24

In the same documentation of dss_capi is cited that:

It removes two arguments of the function call -- it may seem an exaggeration, but it does have an impact on Python, for example.

But, what are the differences between these two modes of usage?

When is the IR more appropriate instead of GR?

And, for the last, is there a way to create a local instance of DSS? For example, that I can instantiate inside a function and don't changes others? If not, is there a way to implement it?

Thanks in advance!

[PMeira]

Hi, Felipe,

But, what are the differences between these two modes of usage?

The minimal examples at https://github.com/dss-extensions/dss_capi/tree/master/examples illustrate the basic usage.

Basically, most of the API functions do something like:

• grab some data from the OpenDSS internals
• potentially apply a transformation
• copy the result to an output buffer

In terms of operation, there are no differences. This is an API-only aspect. In fact, all GR functions just call the IR functions with the matching pointers. The only difference is where vector/array results are stored.

• Without GR, besides the extra parameters, the API would always allocate memory for the results, and the user should than call the appropriate DSS_Dispose... to free the memory block.
• With GR, the internal buffer usually grows to a certain size and stays there, so no need to reallocate.

On the Python side, right now we always copy the buffer into a new "native" buffer (e.g. through NumPy), independent of GR or IR usage. Same is valid for the other DSS Extensions.

Since DSS C-API version 0.10.0, the buffers can be reused since we introduced a capacity value besides the current size value (hence the usage of int[2] instead of a plain int). This already makes low-level usage simpler. The GR API just generalizes that to internal buffers, so the user doesn't really need to worry about those memory blocks for "99%" of the cases. The other 1% would be cases of extreme memory constraints where it's better to free the memory buffers as soon as possible -- I don't think this is a common issue, neither in a Raspberry Pi with 4 GB of RAM, or in an enterprise server with 1 TB.

When is the IR more appropriate instead of GR?

That's up to the DSS C-API users.

On DSS Python, I left the GR as a default to check if anybody would complain. A few years later, so far nothing -- thus, either the IR bindings in DSS Python will be removed by version 1.0 or we will track the memory better and avoid the extra copies and migrate the default back to the IR functions, then removing the GR usage. Some decisions will depend on how much better/faster is CPython 3.11.

Nowadays, DSS Sharp and DSS MATLAB are already using or have open PRs to migrate to the GR API. OpenDSSDirect.py probably will be migrated soon, too, getting it closer to DSS Python. For OpenDSSDirect.jl, we should benchmark a couple of things first before deciding how to handle this and the topic of your next question.

And, for the last, is there a way to create a local instance of DSS? For example, that I can instantiate inside a function and don't changes others? If not, is there a way to implement it?

You can with the next version of DSS C-API since most of the global state in the OpenDSS engine was encapsulated in a new DSSContext class. See the latest beta versions at https://pypi.org/project/dss-python/#history
Most of the parallel-machine functions (diakoptics is pending) of OpenDSS v8+ were rewritten using DSSContexts too.

On the official OpenDSS, even the latest versions, you can't create multiple instances, all you have is the actor/PM mechanism, which is limited.

Some more notes at: #42 (comment)

Please also check the pull requests for the source code and other notes:

• v0.12.x mega-merge dss_capi#109
• Use DSS C-API 0.12.x #43

Besides some minor final tweaks, we're finishing the initial documentation for this new release before publishing the new docs repo and the final version.

Implementation-wise, all functions need to pass the DSS context as a pointer. To reduce the overhead in both run-time and programming time, DSS Python just uses functools.partial to pre-attach the context pointer to the functions. Thankfully that was enough to avoid changing the majority of the Python code.

Threading in Python is not always that useful (due to the GIL), but since the CFFI releases the GIL, it can still help. And multiple DSS engines can still be useful in single-threaded scenarios. As always, Python is still great for prototyping.

By the way, I don't recommend instantiating throw-away instances since it would be heavy. I'd say creating a pool of instances and reusing them as necessary is the best way to go.

[felipemarkson]

Ok Paulo!

Thanks!

I'll wait until the release of the next version of DSS C-API.