Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly calculate the offset of the column_view & apply offsets in copy_meta_ranges #423

Merged
19 commits merged into from
Oct 31, 2022
Merged

Properly calculate the offset of the column_view & apply offsets in copy_meta_ranges #423

Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
815ad6a
Add an AddScoresStage to the sid pipeline, and validate the scores ag…
dagardner-nv Oct 27, 2022
3a8109a
Parametarize running the test for both row and column major
dagardner-nv Oct 28, 2022
e24d4bc
Use get_meta method to ensure slixing is applied
dagardner-nv Oct 28, 2022
3761187
Parametarize the pipeline_batch_size
dagardner-nv Oct 28, 2022
feb8589
Perform cuda copy using cudf::type_dispatcher
dagardner-nv Oct 29, 2022
be4a60c
Parametarize values
dagardner-nv Oct 29, 2022
d35343a
Move compare_class_to_scores to utils.py
dagardner-nv Oct 29, 2022
6cc99ab
Add add-scores test
dagardner-nv Oct 29, 2022
f1c2636
Merge branch 'branch-22.11' into david-add-scores-bug
dagardner-nv Oct 29, 2022
ac3fda4
Concatinate the input rather that repeating it, this forces the
dagardner-nv Oct 31, 2022
4be8737
Merge branch 'david-add-scores-bug' of github.com:dagardner-nv/Morphe…
dagardner-nv Oct 31, 2022
eb1dc97
fix tests
dagardner-nv Oct 31, 2022
e1f65e6
Fix type hint on convmesg class
dagardner-nv Oct 31, 2022
81207a3
Fix offset in ConvMdg
dagardner-nv Oct 31, 2022
0edcf0a
Remove unused imports
dagardner-nv Oct 31, 2022
86317f6
Parametrize column ordering, pipeline size, and data size in filter d…
dagardner-nv Oct 31, 2022
a6dae37
Adding fixes for making sliced copies and for calling set_meta() with…
mdemoret-nv Oct 31, 2022
a3a7861
Remove unused imports
dagardner-nv Oct 31, 2022
04d7cfc
Remove unused DispatchedCopy
dagardner-nv Oct 31, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add add-scores test
  • Loading branch information
@dagardner-nv
dagardner-nv committed Oct 29, 2022
commit 6cc99ab0b798b4bc841fa9d2277ba803fe806777
6 changes: 6 additions & 0 deletions tests/test_abp.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,14 @@
from morpheus.stages.input.file_source_stage import FileSourceStage
from morpheus.stages.output.write_to_file_stage import WriteToFileStage
from morpheus.stages.postprocess.add_classifications_stage import AddClassificationsStage
from morpheus.stages.postprocess.add_scores_stage import AddScoresStage
from morpheus.stages.postprocess.serialize_stage import SerializeStage
from morpheus.stages.postprocess.validation_stage import ValidationStage
from morpheus.stages.preprocess.deserialize_stage import DeserializeStage
from morpheus.stages.preprocess.preprocess_fil_stage import PreprocessFILStage
from utils import TEST_DIRS
from utils import calc_error_val
from utils import compare_class_to_scores

# End-to-end test intended to imitate the ABP validation test
FEATURE_LENGTH = 29
Expand Down Expand Up @@ -102,12 +104,14 @@ def async_infer(callback=None, **k):
TritonInferenceStage(config, model_name='abp-nvsmi-xgb', server_url='test:0000', force_convert_inputs=True))
pipe.add_stage(MonitorStage(config, description="Inference Rate", smoothing=0.001, unit="inf"))
pipe.add_stage(AddClassificationsStage(config))
pipe.add_stage(AddScoresStage(config, prefix="score_"))
pipe.add_stage(
ValidationStage(config, val_file_name=val_file_name, results_file_name=results_file_name, rel_tol=0.05))
pipe.add_stage(SerializeStage(config))
pipe.add_stage(WriteToFileStage(config, filename=out_file, overwrite=False))

pipe.run()
compare_class_to_scores(out_file, config.class_labels, '', 'score_', threshold=0.5)
results = calc_error_val(results_file_name)
assert results.diff_rows == 0

Expand Down Expand Up @@ -145,12 +149,14 @@ def test_abp_cpp(config, tmp_path):
force_convert_inputs=True))
pipe.add_stage(MonitorStage(config, description="Inference Rate", smoothing=0.001, unit="inf"))
pipe.add_stage(AddClassificationsStage(config))
pipe.add_stage(AddScoresStage(config, prefix="score_"))
pipe.add_stage(
ValidationStage(config, val_file_name=val_file_name, results_file_name=results_file_name, rel_tol=0.05))
pipe.add_stage(SerializeStage(config))
pipe.add_stage(WriteToFileStage(config, filename=out_file, overwrite=False))

pipe.run()
compare_class_to_scores(out_file, config.class_labels, '', 'score_', threshold=0.5)
results = calc_error_val(results_file_name)
assert results.diff_rows == 0

Expand Down