Batched matrix multiplication. #1261

FOsterfeld · 2023-11-08T13:36:03Z

split dimension is a batch dimension

Due Diligence

General:
- base branch must be main for new features, latest release branch (e.g. release/1.3.x) for bug fixes
- title of the PR is suitable to appear in the Release Notes
Implementation:
- unit tests: all split configurations tested
- unit tests: multiple dtypes tested
- documentation updated where needed

Description

Issue/s resolved: #890

Changes proposed:

Type of change

Memory requirements

Performance

Does this change modify the behaviour of other functions? If so, which?

yes / no

split dimension is a batch dimension

ghost · 2023-11-08T13:37:51Z

👇 Click on the image for a new way to code review

Legend

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2023-11-08T14:23:07Z

Thank you for the PR!

codecov · 2023-11-08T15:04:00Z

Codecov Report

Attention: Patch coverage is 97.36842% with 8 lines in your changes missing coverage. Please review.

Project coverage is 92.13%. Comparing base (bac864d) to head (fc97280).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
heat/core/linalg/basics.py	97.36%	8 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1261      +/-   ##
==========================================
+ Coverage   92.07%   92.13%   +0.05%     
==========================================
  Files          83       83              
  Lines       12196    12163      -33     
==========================================
- Hits        11230    11206      -24     
+ Misses        966      957       -9

Flag	Coverage Δ
unit	`92.13% <97.36%> (+0.05%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

mrfh92 · 2023-11-10T16:33:22Z

heat/core/linalg/basics.py

@@ -487,9 +487,12 @@ def matmul(a: DNDarray, b: DNDarray, allow_resplit: bool = False) -> DNDarray:
    sanitation.sanitize_in(a)
    sanitation.sanitize_in(b)

-    if a.gshape[-1] != b.gshape[0]:
+    batch_dim = max(a.ndim, b.ndim) - 2
+    batched = batch_dim > 2


maybe > 0 instead of >2 or did I understand sth wrong?

Thanks, you're right, it should be 0.

mrfh92 · 2023-11-13T07:59:24Z

Sieht gut aus soweit 👍

Vorschläge für die weitere Arbeit:

Wenn du oben neben "codecov" (rotes X) auf Details und dann auf "View this Pull Request on Codecov" gehst, kannst du sehen, welche Zeilen deiner Änderungen nicht getestet werden bisher. Entsprechende Tests kannst du in heat/core/linalg/tests/test_basics.py unterbringen.
Mit diesem Fall wäre dann gebatchte Matrizenmultiplikation für den Fall abgedeckt, dass die LA-Dimensionen nicht gesplitted sind. Dies entspricht quasi dem Hinzufügen von Batchdimensionen zum Fall "2xsplit=None" im ursprünglichen Code.
Als nächstes könnten wir den Fall angehen, dass zu den anderen 8 (?) bestehenden Fällen Batch-dimensionen hinzugefügt werden; nun wäre allerdings die Batchdimension nicht gesplitted und dafür evtl. die LA-Dimension. Mit etwas Glück lässt sich das recht einfach aus dem bestehenden Code erzeugen, da ja PyTorch bereits gut mit Batches arbeiten kann.
Den Fall, dass einmal eine LA-Dimension gesplitted ist und einmal eine Batchdimension gesplitted ist, würde ich ausschließen und eine enstprechende Fehlermeldung ausgeben.

github-actions · 2023-11-21T09:47:53Z

Thank you for the PR!

github-actions · 2023-11-21T10:23:35Z

Thank you for the PR!

…nsion

github-actions · 2023-11-21T11:20:02Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2023-11-21T12:15:34Z

Thank you for the PR!

github-actions · 2023-11-21T14:09:39Z

Thank you for the PR!

github-actions · 2023-11-21T14:46:27Z

Thank you for the PR!

github-actions · 2023-11-21T15:08:15Z

Thank you for the PR!

github-actions · 2023-11-22T13:59:23Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-08-17T22:54:08Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-08-20T13:30:34Z

Thank you for the PR!

mrfh92 · 2024-08-20T13:53:23Z

@FOsterfeld Something seems to have changed that lets the tests in QR fail. Can you reproduce this error on the workstation?

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-08-28T17:51:01Z

Thank you for the PR!

github-actions · 2024-08-30T15:47:32Z

Thank you for the PR!

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-09-02T12:24:34Z

Thank you for the PR!

JuanPedroGHM · 2024-09-02T12:38:57Z

Benchmarks results - Sponsored by perun

function	mpi_ranks	device	metric	value	ref_value	std	% change	type	alert	lower_quantile	upper_quantile
matmul_split_0	4	CPU	RUNTIME	0.142147	0.186948	0.0350541	-23.9647	jump-detection	True	nan	nan
matmul_split_1	4	CPU	RUNTIME	0.111954	0.134253	0.0147728	-16.6099	jump-detection	True	nan	nan
concatenate	4	CPU	RUNTIME	0.193592	0.161892	0.0454197	19.581	jump-detection	True	nan	nan
matmul_split_0	4	GPU	RUNTIME	0.0429973	0.058403	0.034678	-26.3782	jump-detection	True	nan	nan
kmedoids	4	CPU	RUNTIME	0.688199	1.23864	0.00419769	-44.4392	trend-deviation	True	0.697841	1.99332
apply_inplace_max_abs_scaler_and_inverse	4	CPU	RUNTIME	0.000499201	0.00151836	1.56914e-05	-67.1223	trend-deviation	True	0.000501199	0.00351122
apply_inplace_robust_scaler_and_inverse	4	CPU	RUNTIME	2.38697	5.38711	0.0112881	-55.6911	trend-deviation	True	2.40194	9.88492

Grafana Dashboard
Last updated: 2024-09-02T12:38:56Z

JuanPedroGHM

👍

mrfh92 · 2024-09-04T14:55:53Z

@mtar have your suggestions been addressed by the changes?

…gebra_for_arrays_with_dimension_2_in_particular_matmul

github-actions · 2024-09-06T06:06:03Z

Thank you for the PR!

JuanPedroGHM · 2024-09-06T06:12:06Z

Benchmarks results - Sponsored by perun

function	mpi_ranks	device	metric	value	ref_value	std	% change	type	alert	lower_quantile	upper_quantile
matmul_split_1	4	CPU	RUNTIME	0.122906	0.109381	0.0313848	12.3656	jump-detection	True	nan	nan
concatenate	4	CPU	RUNTIME	0.167615	0.196439	0.0260386	-14.6728	jump-detection	True	nan	nan
apply_inplace_normalizer	4	CPU	RUNTIME	0.00100724	0.00947982	8.56803e-05	-89.375	jump-detection	True	nan	nan
matmul_split_0	4	GPU	RUNTIME	0.0483279	0.0545347	0.0310459	-11.3814	jump-detection	True	nan	nan
matmul_split_1	4	GPU	RUNTIME	0.0367425	0.0272262	0.0292762	34.9526	jump-detection	True	nan	nan
qr_split_1	4	CPU	RUNTIME	0.179386	0.451767	0.00646491	-60.2924	trend-deviation	True	0.179536	0.796189
kmeans	4	CPU	RUNTIME	0.309642	0.909418	0.00239133	-65.9517	trend-deviation	True	0.312577	1.79242
reshape	4	CPU	RUNTIME	0.155737	0.365847	0.00110721	-57.431	trend-deviation	True	0.156546	0.731662
lanczos	4	GPU	RUNTIME	0.600791	0.586755	0.00316215	2.39205	trend-deviation	True	0.579288	0.597313
kmeans	4	GPU	RUNTIME	0.657029	0.632586	0.004188	3.86392	trend-deviation	True	0.617689	0.64691
kmedians	4	GPU	RUNTIME	1.11586	1.01658	0.0125118	9.76642	trend-deviation	True	0.994589	1.04602
kmedoids	4	GPU	RUNTIME	1.2525	1.14701	0.00847031	9.19655	trend-deviation	True	1.12248	1.18111
apply_inplace_robust_scaler_and_inverse	4	GPU	RUNTIME	5.99905	5.60869	0.0446267	6.95995	trend-deviation	True	5.47179	5.75939
apply_inplace_normalizer	4	GPU	RUNTIME	0.00179942	0.00169872	4.94232e-05	5.92757	trend-deviation	True	0.00166147	0.00175474

Grafana Dashboard
Last updated: 2024-09-06T06:12:05Z

first implementation of the minimal solution

67943f9

split dimension is a batch dimension

mrfh92 marked this pull request as draft November 8, 2023 13:37

FOsterfeld and others added 2 commits November 8, 2023 14:41

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

1c60823

…gebra_for_arrays_with_dimension_2_in_particular_matmul

access b.gshape[-2] only if input is not batched

0ddba6c

mrfh92 reviewed Nov 10, 2023

View reviewed changes

LScheib assigned LScheib and FOsterfeld and unassigned LScheib Nov 13, 2023

fixed batched condition

d60c0ca

throw a NotImplementedError for wrong split dimension on batched matmul

a60d3ac

FOsterfeld added 2 commits November 21, 2023 12:13

fixed dimension condition

e16366b

added test for batched matmul with split dimension being a batch dime…

7644dd4

…nsion

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

5d34282

…gebra_for_arrays_with_dimension_2_in_particular_matmul

FOsterfeld added 2 commits November 21, 2023 15:04

fixed condition for different batch dimensions

095ccc5

added some tests for correctly thrown errors

e55f8b8

fixed test for batched matmul on gpu

a9ae2bf

test for batched matmul on gpu

06913c7

remove unnecessary test with device=gpu

ba60c82

FOsterfeld requested review from mtar and mrfh92 August 17, 2024 22:39

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

3886942

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

9f9462c

…gebra_for_arrays_with_dimension_2_in_particular_matmul

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

3f15690

…gebra_for_arrays_with_dimension_2_in_particular_matmul

FOsterfeld added 2 commits August 30, 2024 17:39

initialize random generator

ca066b2

refactored code for None splits

398b27e

mrfh92 added the PR talk label Sep 1, 2024

ClaudiaComito added this to the 1.5.0 milestone Sep 2, 2024

JuanPedroGHM added the benchmark PR label Sep 2, 2024

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

6f92537

…gebra_for_arrays_with_dimension_2_in_particular_matmul

mrfh92 removed the PR talk label Sep 2, 2024

JuanPedroGHM approved these changes Sep 4, 2024

View reviewed changes

ClaudiaComito added the enhancement New feature or request label Sep 5, 2024

ClaudiaComito mentioned this pull request Sep 5, 2024

matmul on multidimensional arrays #890

Closed

Merge branch 'main' into features/1104-Implement_consistent_linear_al…

fc97280

…gebra_for_arrays_with_dimension_2_in_particular_matmul

mtar approved these changes Sep 6, 2024

View reviewed changes

ClaudiaComito merged commit 914e5c3 into main Sep 9, 2024
44 checks passed

FOsterfeld mentioned this pull request Sep 11, 2024

[Bug]: Batched Matrix-Vector Multiplication does not work correctly #1645

Closed

mtar deleted the features/1104-Implement_consistent_linear_algebra_for_arrays_with_dimension_2_in_particular_matmul branch October 21, 2024 13:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batched matrix multiplication. #1261

Batched matrix multiplication. #1261

FOsterfeld commented Nov 8, 2023 •

edited by ClaudiaComito

Loading

ghost commented Nov 8, 2023 •

edited by ghost

Loading

Legend

github-actions bot commented Nov 8, 2023

codecov bot commented Nov 8, 2023 •

edited

Loading

mrfh92 Nov 10, 2023 •

edited

Loading

FOsterfeld Nov 21, 2023

mrfh92 commented Nov 13, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 22, 2023

github-actions bot commented Aug 17, 2024

github-actions bot commented Aug 20, 2024

mrfh92 commented Aug 20, 2024

github-actions bot commented Aug 28, 2024

github-actions bot commented Aug 30, 2024

github-actions bot commented Sep 2, 2024

JuanPedroGHM commented Sep 2, 2024

JuanPedroGHM left a comment

mrfh92 commented Sep 4, 2024

github-actions bot commented Sep 6, 2024

JuanPedroGHM commented Sep 6, 2024

Batched matrix multiplication. #1261

Batched matrix multiplication. #1261

Conversation

FOsterfeld commented Nov 8, 2023 • edited by ClaudiaComito Loading

Due Diligence

Description

Changes proposed:

Type of change

Memory requirements

Performance

Does this change modify the behaviour of other functions? If so, which?

ghost commented Nov 8, 2023 • edited by ghost Loading

Legend

github-actions bot commented Nov 8, 2023

codecov bot commented Nov 8, 2023 • edited Loading

Codecov Report

mrfh92 Nov 10, 2023 • edited Loading

Choose a reason for hiding this comment

FOsterfeld Nov 21, 2023

Choose a reason for hiding this comment

mrfh92 commented Nov 13, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 21, 2023

github-actions bot commented Nov 22, 2023

github-actions bot commented Aug 17, 2024

github-actions bot commented Aug 20, 2024

mrfh92 commented Aug 20, 2024

github-actions bot commented Aug 28, 2024

github-actions bot commented Aug 30, 2024

github-actions bot commented Sep 2, 2024

JuanPedroGHM commented Sep 2, 2024

Benchmarks results - Sponsored by perun

JuanPedroGHM left a comment

Choose a reason for hiding this comment

mrfh92 commented Sep 4, 2024

github-actions bot commented Sep 6, 2024

JuanPedroGHM commented Sep 6, 2024

Benchmarks results - Sponsored by perun

FOsterfeld commented Nov 8, 2023 •

edited by ClaudiaComito

Loading

ghost commented Nov 8, 2023 •

edited by ghost

Loading

codecov bot commented Nov 8, 2023 •

edited

Loading

mrfh92 Nov 10, 2023 •

edited

Loading