Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Object versioning for DataCarrier object #579

Merged
merged 29 commits into from
Jul 21, 2022
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0ff6b73
Object versioning for DataCarrier object (finality: firedrake.Constant)
nbouziani May 4, 2020
c945975
Update object versioning + add test
nbouziani May 6, 2020
f44f7eb
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani May 9, 2020
43aca1e
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani Jul 22, 2020
6e0178b
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani Sep 7, 2020
fe63bf7
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani Dec 20, 2020
e5af975
Update AUTHORS
nbouziani Dec 20, 2020
ff738b2
Merge branch 'master' into DataCarrier-object-versionning
nbouziani Apr 26, 2021
516454d
Update data versioning for Dat and Mat (base and petsc_base classes)
nbouziani Apr 29, 2021
07a47b3
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani May 24, 2021
29973b1
Add some missing DataCarrier inits
nbouziani May 26, 2021
7d3d4be
Merge master
nbouziani Sep 24, 2021
6d39e69
Merge remote-tracking branch 'origin/master' into DataCarrier-object-…
nbouziani Nov 14, 2021
520078b
Add mat.py
nbouziani Nov 14, 2021
d89e93c
Refactor object versioning using PETSc state counter
nbouziani Nov 16, 2021
ca5e50d
Fix typo
nbouziani Nov 16, 2021
54dd188
Add tests for Dat
nbouziani Nov 16, 2021
18f9fa4
Remove spurious incrementations from VecAccessMixin
nbouziani Nov 16, 2021
2836c7a
Cleanup
nbouziani Nov 21, 2021
7bcde5a
Add dat_version to MixedDat + test
nbouziani Nov 21, 2021
0e879bd
Specify petsc branch in ci.yml
nbouziani Nov 24, 2021
55ad119
Update few things
nbouziani Nov 25, 2021
82f9ac5
Update test + remove @abstractmethod from increment_dat_version
nbouziani Nov 25, 2021
3edaf9a
Merge branch 'DataCarrier-object-versionning' of github.com:OP2/PyOP2…
nbouziani Nov 25, 2021
ba8d12c
Merge master
nbouziani Jul 13, 2022
0989f1d
Fix lint
nbouziani Jul 13, 2022
a1d9d9f
Update ci.yml
nbouziani Jul 14, 2022
9eabf60
Add tests for Dat context managers
nbouziani Jul 21, 2022
ff58332
Merge branch 'DataCarrier-object-versionning' of github.com:OP2/PyOP2…
nbouziani Jul 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion pyop2/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -1250,6 +1250,9 @@ class DataCarrier(object):
(:class:`Global`), rank 1 (:class:`Dat`), or rank 2
(:class:`Mat`)"""

def __init__(self):
self.dat_version = 0

@cached_property
def dtype(self):
"""The Python type of the data."""
Expand Down Expand Up @@ -1366,6 +1369,7 @@ def __init__(self, dataset, data=None, dtype=None, name=None, uid=None):
# a dataset dimension of 1.
dataset = dataset ** 1
self._shape = (dataset.total_size,) + (() if dataset.cdim == 1 else dataset.dim)
DataCarrier.__init__(self)
_EmptyDataMixin.__init__(self, data, dtype, self._shape)

self._dataset = dataset
Expand Down Expand Up @@ -2267,6 +2271,7 @@ def __init__(self, dim, data=None, dtype=None, name=None, comm=None):
return
self._dim = as_tuple(dim, int)
self._cdim = np.prod(self._dim).item()
DataCarrier.__init__(self)
_EmptyDataMixin.__init__(self, data, dtype, self._dim)
self._buf = np.empty(self.shape, dtype=self.dtype)
self._name = name or "global_%d" % Global._globalcount
Expand Down Expand Up @@ -2327,6 +2332,7 @@ def shape(self):
@property
def data(self):
"""Data array."""
self.dat_version += 1
if len(self._data) == 0:
raise RuntimeError("Illegal access: No data associated with this Global!")
return self._data
Expand All @@ -2338,12 +2344,13 @@ def dtype(self):
@property
def data_ro(self):
"""Data array."""
view = self.data.view()
view = self._data.view()
view.setflags(write=False)
return view

@data.setter
def data(self, value):
self.dat_version += 1
self._data[:] = verify_reshape(value, self.dtype, self.dim)

@property
Expand Down Expand Up @@ -2375,6 +2382,7 @@ def copy(self, other, subset=None):

@collective
def zero(self):
self.dat_version += 1
self._data[...] = 0

@collective
Expand Down Expand Up @@ -3112,6 +3120,7 @@ def pack(self):
@validate_type(('sparsity', Sparsity, SparsityTypeError),
('name', str, NameTypeError))
def __init__(self, sparsity, dtype=None, name=None):
DataCarrier.__init__(self)
self._sparsity = sparsity
self.lcomm = sparsity.lcomm
self.rcomm = sparsity.rcomm
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good start, but not all the places in which the data in a Dat or matrix can be modified. In petsc_base.py the .vec, and vec_wo context managers also induce modification of the data. (see VecAccessMixin). For matrices you should probably update the version in assemble (but also other functions that sound like they modify data, like zero_rows/set_values...).

Additionally, the halo exchange functions (global_to_local_end/local_to_global_end) might change data state as well (the user is allowed to call these).

Finally, the big one, a par_loop also updates argument state of all arguments other than READ arguments. Probably adapt ParLoop.update_arg_data_state.

Copy link
Contributor Author

@nbouziani nbouziani Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much for you advice! I do apologize for my late reply, I have completely forgotten about this pending task! Yes, I initially pushed changes to handle op2.Global without considering op2.Dat/op2.Mat because object versioning was at that time needed for firedrake.Constant only.

I have just pushed some changes following your comment.

Regarding assemble: did you mean firedrake.assemble ? My first guess to catch the various assembly calls performed on base.Mat was to update the data version in petsc_base.Mat.assemble but I am not sure that all assembly calls go through this route. In particular, unlike firedrake.Matrix, firedrake.ImplicitMatrix does not seem to rely on base.Mat. Instead it uses the PETSc matrix object (PETSc.Mat()), and I am not sure how these two play with each other.

What do you think ?

Copy link
Contributor Author

@nbouziani nbouziani Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naive question for ParLoop: Why would we need to adapt update_arg_data_state (which is called in ParLoop.compute) whereas local_to_global_end, which is called after, will take into account the access mode to update the data version ?

Expand Down
32 changes: 32 additions & 0 deletions test/unit/test_globals.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,3 +44,35 @@ def test_global_operations():
assert (g1 * g2).data == 10.
g1 *= g2
assert g1.data == 10.


def test_global_dat_version():
g1 = op2.Global(1, data=1.)
g2 = op2.Global(1, data=2.)

assert g1.dat_version == 0
assert g2.dat_version == 0

# Access data property
d1 = g1.data

assert g1.dat_version == 1
assert g2.dat_version == 0

# Access data property
g2.data[:] += 1

assert g1.dat_version == 1
assert g2.dat_version == 1

# Access zero property
g1.zero()

assert g1.dat_version == 2
assert g2.dat_version == 1

# Access data setter
g2.data = d1

assert g1.dat_version == 2
assert g2.dat_version == 2