Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated lib kernel unit tests #1599

Open
wants to merge 46 commits into
base: master
Choose a base branch
from

Conversation

oOTigger
Copy link
Contributor

@oOTigger oOTigger commented Feb 25, 2025

Description of changes:

Updated kernel tests from #992:

  • accessor
  • array_shape
  • copy_tensor_accessor
  • datatype_dispatch
  • legion_dim
  • local_cpu_allocator
  • local_cuda_allocator
  • perf_metrics

This change is Reviewable

oOTigger and others added 30 commits January 21, 2025 20:48
…lexflow#1429)

* initial commit for machine view adjacent modules

* Formatting

* Tests for new machine_view.cc functions

* formatting

* Minor Test correction

* formatting

* PR fixes

* PR Fixes

---------

Co-authored-by: Pietro Max Marsella <marsella@stanford.edu>
* Debug

* Change to base DL AMI

* Print disk usage

* Run nvidia-smi

* Remove excess cuda installs in base ami

* Re-enable freeing space in GPU CI

* Try updating nix-develop version

* Check what happens if you just enter the non-nixGL environment

* Try switching AMIs

* Try to remove the module stuff

* Move to lockshaw/develop-action

* Try pointing at a fixed commit

* Update nix-develop action

* Update nix-develop action to use BASH_FUNC filtering

* Remove all the /usr/local/cuda entries

* Switch back to gpu-ci env

* Update the cuda arch

* Try out the new runs-on gpu image

* Move over to pinned runs-on image

* Remove a bunch more unnecessary stuff in image to get back disk space

* Try using an emphemeral store

* Try mounting

* Fix bug

* Try sudo

* Move nix into _work

* Rollback all unnecessary changes

* Re-enable waiting on cpu-ci
* Start on pcg builder

* Add tests and some implementation for pcg builder

* Add pcg tests, make dtgen constructors explicit to fix bug

* Add remainder of PCG tests

* Fix build issues in local-execution

* Format

* Address Reyna comments, add topological_order function for PCG

* Pre multidigraph refactor

* Removing visitable from sp code

* Add open dataflow graph, start to replace pcg dataflow graph

* Start refactoring substitutions

* Add utility functions to support pattern matching

* Pre-refactor inputs

* Fix proj url

* Get back to substitutions, now with unordered graph inputs

* Get substitutions building

* substitutions-tests now builds

* Fix bug in filter, pass some initial substitution tests

* Add tests for fmt::to_string, fix some substitutions bugs

* Pass initial unit tests for find_pattern_matches

* Start on unit tests for pcg pattern

* Pass initial test for find_pattern_matches

* Fix small build issue in tests

* Format

* Sync tests in CI with tests in proj

* Fix minor build errors in kernels and local-execution

* Format

* Remove outdated code

* More outdated code removal

* More cleanup, add test for sp decomposition

* Pull apart containers.h

* More sp testing and fixes

* Break up graph algorithms.h

* Pre- full SP algo commit

* Add initial implementation and tests for cbc decomposition and inverse line graph

* Pass test for get_inverse_line_graph

* Add new multidigraph

* Fix get_inverse_line_graph to return a MultiDiGraph instead of a DiGraph

* Add tests for parallel and series reduction finding

* Add really rough implementation of valdez sp decomposition

* Fix local-execution build

* Add implementations and tests for applying series/parallel reductions

* Format

* Clean up sp decomposition interface and tests

* Format

* Add comments for top-level substitutions functions, add proj doxygen support

* Start sketching out substitutions code

* Fix build errors

* Add ability to permute node ids

* Cleanup and start to test new substitutions code

* Add test case for evaluate_substitution_output

* Add naive isomorphism detection code

* Add graph inputs to open dataflow graph isomorphism

* Add input permutation to evaluate_substitution_output

* Fix permute_node_ids

* Add test for permute_input_ids

* Migrate over to mutable implementation of apply_substitution

* Add fast isomorphism checking and an initial implementation of full substitution logic

* Pass initial full substitutions test

* Cleanup old isomorphism checking code

* Fix post-merge bugs

* Fix broken pcg builder test

* Format

* Reorganize code and remove some outdated code pre-code-review

* Format

* Restarting work on this after working on export-model-arch

* Adding in some a simple function to get the currently available substritutions

* nonnegative_int additions, code cleanup, etc.

* A bunch more moving over to nonnegative_int

* Even more nonnegative_int updating

* Fix build

* Fix failing tests

* Format

* Format

---------

Co-authored-by: Colin Unger <lockshaw@lockshaw.net>
Co-authored-by: Victor Li <vli42@sapling2.stanford.edu>
@oOTigger oOTigger changed the title Lib kernel unit tests Updated lib kernel unit tests Feb 25, 2025
@oOTigger oOTigger requested a review from lockshaw February 25, 2025 04:00
@oOTigger oOTigger self-assigned this Feb 25, 2025
Copy link

codecov bot commented Feb 25, 2025

Codecov Report

Attention: Patch coverage is 16.15385% with 327 lines in your changes missing coverage. Please review.

Project coverage is 62.18%. Comparing base (10ef31b) to head (2914494).

Files with missing lines Patch % Lines
lib/kernels/src/copy_tensor_accessor.cc 10.86% 41 Missing ⚠️
lib/kernels/include/kernels/accessor.h 7.14% 39 Missing ⚠️
lib/kernels/src/array_shape.cc 3.12% 31 Missing ⚠️
lib/pcg/src/pcg/metric_attrs.cc 0.00% 31 Missing ⚠️
lib/kernels/src/cpu/cast_kernels.cc 0.00% 25 Missing ⚠️
lib/kernels/src/cpu/replicate_kernels.cc 0.00% 20 Missing ⚠️
lib/kernels/src/managed_per_device_ff_handle.cc 0.00% 19 Missing ⚠️
lib/kernels/src/cpu/combine_kernels.cc 0.00% 18 Missing ⚠️
lib/kernels/src/local_cpu_allocator.cc 0.00% 14 Missing ⚠️
lib/kernels/src/cpu/reverse_kernels.cc 0.00% 13 Missing ⚠️
... and 21 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1599      +/-   ##
==========================================
- Coverage   62.90%   62.18%   -0.72%     
==========================================
  Files         635      645      +10     
  Lines       15201    15490     +289     
==========================================
+ Hits         9562     9633      +71     
- Misses       5639     5857     +218     
Flag Coverage Δ
unittests 62.18% <16.15%> (-0.72%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
lib/kernels/include/kernels/allocation.h 100.00% <ø> (ø)
lib/kernels/include/kernels/local_cuda_allocator.h 0.00% <ø> (ø)
lib/kernels/src/allocation.cc 72.72% <100.00%> (+10.22%) ⬆️
...tion/include/local-execution/local_cpu_allocator.h 100.00% <ø> (ø)
...n/include/local-execution/task_argument_accessor.h 0.00% <ø> (ø)
...cution/include/local-execution/tracked_allocator.h 0.00% <ø> (ø)
lib/local-execution/src/local_cpu_allocator.cc 57.14% <100.00%> (+7.14%) ⬆️
...ocal-execution/src/local_task_argument_accessor.cc 70.37% <100.00%> (ø)
lib/local-execution/src/ops/replicate.cc 0.00% <ø> (ø)
lib/local-execution/src/per_device_op_state.cc 0.00% <ø> (ø)
... and 34 more

... and 3 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants