Small change to derive_predictors_and_scores for speed + normalization
#119
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…sResearch/collab-creatures into pp-collab2-pairwise-copying-pred
…ollab-creatures into pp-collab2-pairwise-copying-pred
…sResearch/collab-creatures into pp-collab2-howfarscore
…ollab-creatures into ru-random-hungry-2
|
Added option to scale values based on empirical CDF |
…uff (collab2 ). (#126) * Some work on the random_foragers notebook and fixing stuff. * Linting + completing the random_foragers notebook. * Finished random_foragers * Interactive plots now should be displayed in HTML * make format * small fixes to random foragers * Some more tweaks + zero-index fixes. * Hungry birds simulation updated. * Minor. * Completed the follower NB. * Saves the samples from each one of the R,H,F to disk, for later plotting in a single figure. * Comparative fig. * Minor. * Make lint and format * Typos * Improved explanations of the predictors and the scores. * Updated the model description in the random notebook. * Minor * Added option for initial positions. Updated RHF. * reviewed random * added toc to followers * fixed followers * Small formulas + model updates * small modification * small fixes, re-ran * fixing save and display in follower * format lint, dilling in hungry --------- Co-authored-by: rfl-urbaniak <rfl.urbaniak@gmail.com>
…/collab-creatures into pp-collab2-upgrade-derive
rfl-urbaniak
requested changes
Oct 3, 2024
rfl-urbaniak
added
status:awaiting response
PalkaPuri
changed the title
derive_predictors_and_scores for speedderive_predictors_and_scores for speed + normalization
PalkaPuri
added
status:awaiting review
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I noticed that generating a combined DF of predictors and scores was taking very long for large datasets (my kernel kept dying). This operation was very compute expensive because we were using
df.mergewhich involves searching through column values to find matching rows. However, we can get away with using something simpler likepd.concatsince all predictor/score DataFrames inherit the grid fromlocal_windowsand hence the rows match by design. This should help clear the speed bottleneck.I also updated the function to return eCDF normalized values as an option