-
Notifications
You must be signed in to change notification settings - Fork 183
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for online drift detectors #1108
Add support for online drift detectors #1108
Conversation
# If batch is configured, wrap X in a list so that it is not unpacked | ||
X = np.array(input_data) | ||
if self._ad_settings.batch_size: | ||
X = [X] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wrap X
in a list (of length 1) so that we don't unpack into single instances when running in batch mode (i.e. offline drift detectors or outlier detectors). pred
is then returned as a list of length 1.
This slightly circuitous logic is intended to avoid having two duplicate try-except blocks for the two (offline vs online) use cases...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great @ascillitoe! 🚀
I've added a minor comment around the state save logic and check, but everything else should be good to go once the linter and tests pass!
This PR adds support for online drift detectors to Alibi Detect runtime.
Overview
This involves two primary changes:
self.t
updated each time. If a request contains a batch of instances, the payload must be unpacked and passed to the detector one instance at a time.state_save_freq
time-steps, wherestate_save_freq
is a new configurable parameter added toAlibiDetectSettings
.Limitations
Parallel inference is NOT currently supported for online detectors (i.e.
parallel_workers=0
is required). This would require careful thought wrt to how state is managed/shared across parallel workers, in addition to considering the implications of distributing non-iid samples between workers.TODO's