Skip to content

Add serialization test and fix NumUniqueSeparators #122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 7, 2022

Conversation

thehomebrewnerd
Copy link
Contributor

Moves serialization test from TestLSA to PrimitiveT so all primitives are tested for ability to serialize.

Fixes serialization issue with NumUniqueSeparators

Closes #113

@rwedge
Copy link
Contributor

rwedge commented Apr 6, 2022

I think the calculate feature matrix calls are extending the total time spent for the unit tests by a lot

Do you think that confirming saving and loading the feature would be sufficient for the test (and skip calculating the feature matrixes)?

@thehomebrewnerd
Copy link
Contributor Author

Do you think that confirming saving and loading the feature would be sufficient for the test (and skip calculating the feature matrixes)?

I think most of the increased time comes from the Elmo serialization test (at least when I test locally). Testing with CFM might be a little overkill, but if we skip CFM with Elmo I think things will run much faster. Let me test it out.

@thehomebrewnerd
Copy link
Contributor Author

@rwedge Things seem much faster after this change: e27ee38

Should we go with something like this or just drop the CFM stuff completely?

@rwedge
Copy link
Contributor

rwedge commented Apr 6, 2022

@thehomebrewnerd Let's keep CFM (barring ELMO)

I'd like to make a call to the serialized feature's primitive get_function before serialization, since for LSA that's when the sklearn pipeline is initialized

Copy link
Contributor

@dvreed77 dvreed77 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@thehomebrewnerd thehomebrewnerd merged commit ee6dc7b into main Apr 7, 2022
@thehomebrewnerd thehomebrewnerd deleted the fix-num-separators-serialization branch April 7, 2022 16:24
This was referenced Apr 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

NumUniqueSeparators error when serialized to JSON
3 participants