-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Python Client API support for shared memory #570
Conversation
9a4f7ba
to
9b909f6
Compare
232d66a
to
b46eed2
Compare
src/clients/python/build_wheel.sh
Outdated
else | ||
cp Release/crequest.dll \ | ||
"${WHLDIR}/tensorrtserver/api/." | ||
cp ../c++/Release/request.dll \ | ||
"${WHLDIR}/tensorrtserver/api/." | ||
cp Release/cshmwrap.dll \ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shared memory won't work on Windows (at least not with the api you are using). Need to have functions return error on windows platform
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will do
5a61f1a
to
4073d99
Compare
1105401
to
586aada
Compare
1d7ad0a
to
4fc0e35
Compare
@@ -832,51 +988,61 @@ def _prepare_request(self, inputs, outputs, | |||
c_void_p(_crequest_infer_ctx_input_new(byref(input), self._ctx, input_name))) | |||
|
|||
# Set the input shape | |||
if len(input_values) > 0: | |||
shape_value = np.asarray(input_values[0].shape, dtype=np.int64) | |||
if isinstance(input_values, (list, tuple)): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a little confused about how you've changed this logic... we can discuss.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since the input_values can be a list/tuple of np.arrays or a single c_void_p representing the input data values in shared memory, this check is needed. We should only get the shape from the input data if input_values is a list i.e. does not use shared memory.
8f6cdd2
to
c866d9c
Compare
c866d9c
to
7a41f1a
Compare
- avoid extra mmap while setting input - avoid extra memcpy for reading output
…gion per input with libchm
Adds support for using the shared memory functionality via the Python client API