Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Hackathon 5th No.3】为 Paddle 新增 masked_fill API #57355

Merged
merged 34 commits into from
Nov 2, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4925ae4
add masked_fill for paddle
AndSonder Sep 15, 2023
fbec78a
update doc
AndSonder Sep 15, 2023
3cfde41
update some test case
AndSonder Sep 15, 2023
37dd891
remove full_like
AndSonder Sep 15, 2023
793fcdc
update test codes
AndSonder Sep 15, 2023
0c1e8ce
update test cases
AndSonder Sep 16, 2023
faa459a
recover codes
AndSonder Sep 16, 2023
0a4d8d4
update test codes
AndSonder Sep 19, 2023
746a818
fix gradients error
AndSonder Sep 20, 2023
d48d73d
update test codes
AndSonder Sep 20, 2023
7a0ef26
fix
AndSonder Sep 21, 2023
de8cd75
add bf16 test cases
AndSonder Sep 21, 2023
fb1a2ea
update code-block
AndSonder Sep 25, 2023
810575f
Merge branch 'masked_fill' of https://github.com/AndSonder/Paddle int…
AndSonder Sep 25, 2023
1b213bf
Merge branch 'develop' into masked_fill
AndSonder Sep 25, 2023
df6013e
update code-block
AndSonder Sep 25, 2023
82bb307
Merge branch 'masked_fill' of https://github.com/AndSonder/Paddle int…
AndSonder Sep 25, 2023
1349206
update test codes
AndSonder Sep 26, 2023
9ef08e4
Merge branch 'develop' into masked_fill
AndSonder Oct 3, 2023
5f463ad
Update __init__.py
AndSonder Oct 3, 2023
fe74eff
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
AndSonder Oct 3, 2023
4d394a0
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
AndSonder Oct 3, 2023
211d655
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
AndSonder Oct 8, 2023
0703cd0
fix
AndSonder Oct 8, 2023
1c0abb8
Merge branch 'develop' of https://github.com/PaddlePaddle/Paddle into…
AndSonder Oct 11, 2023
f9bdfa0
fix code style and recover third_party
AndSonder Oct 11, 2023
f5db8f6
add v grad check
AndSonder Oct 11, 2023
0f48c13
add scalar value case
AndSonder Oct 11, 2023
c8040ce
fix test case
AndSonder Oct 12, 2023
ccb535c
use logical_not
AndSonder Oct 18, 2023
2317773
Merge branch 'develop' into masked_fill
AndSonder Oct 18, 2023
f678e39
fix doc style
AndSonder Oct 20, 2023
d6b7bd7
Update manipulation.py
AndSonder Oct 23, 2023
bccd1f6
Merge branch 'develop' into masked_fill
AndSonder Oct 24, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
add masked_fill for paddle
  • Loading branch information
AndSonder committed Sep 15, 2023
commit 4925ae4bc934523b15f3d1f9a5eaa58764da60ca
4 changes: 4 additions & 0 deletions python/paddle/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,6 +223,8 @@
from .tensor.manipulation import view # noqa: F401
from .tensor.manipulation import view_as # noqa: F401
from .tensor.manipulation import unfold # noqa: F401
from .tensor.manipulation import masked_fill # noqa: F401
from .tensor.manipulation import masked_fill_ # noqa: F401
from .tensor.math import abs # noqa: F401
from .tensor.math import abs_ # noqa: F401
from .tensor.math import acos # noqa: F401
Expand Down Expand Up @@ -843,4 +845,6 @@
'i1e',
'polygamma',
'polygamma_',
'masked_fill',
'masked_fill_',
]
4 changes: 4 additions & 0 deletions python/paddle/tensor/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,8 @@
from .manipulation import view # noqa: F401
from .manipulation import view_as # noqa: F401
from .manipulation import unfold # noqa: F401
from .manipulation import masked_fill # noqa: F401
from .manipulation import masked_fill_ # noqa: F401
from .math import abs # noqa: F401
from .math import abs_ # noqa: F401
from .math import acos # noqa: F401
Expand Down Expand Up @@ -673,6 +675,8 @@
'i1e',
'polygamma',
'polygamma_',
'masked_fill',
'masked_fill_',
]

# this list used in math_op_patch.py for magic_method bind
Expand Down
154 changes: 154 additions & 0 deletions python/paddle/tensor/manipulation.py
Original file line number Diff line number Diff line change
Expand Up @@ -4543,6 +4543,160 @@ def moveaxis(x, source, destination, name=None):
return out


def masked_fill(x, mask, value, name=None):
"""
Fills elements of self tensor with value where mask is True. The shape of mask must be broadcastable with the shape of the underlying tensor.

Args:
x (Tensor) : The Destination Tensor. Supported data types are int32,
int64, float32, float64.
mask (Tensor): The boolean tensor indicate the position to be filled.
The data type of mask must be bool.
value (Scaler or 0-D Tensor): The value used to fill the target tensor.
name(str, optional): The default value is None. Normally there is no
need for user to set this property. For more information, please
refer to :ref:`api_guide_Name`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

参数说明中写明支持的dtype,且要和设计文档中的一致; Scaler -> Scalar

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


Returns:
Tensor, same dimention and dtype with x.
Examples:
.. code-block:: python
>>> import paddle
>>> x = paddle.ones((3, 3), dtype="float32")
>>> # doctest: +SKIP
>>> mask = paddle.randint(0, 2, [3, 3]).astype('bool')
>>> print(mask)
Tensor(shape=[3, 3], dtype=bool, place=Place(gpu:0), stop_gradient=True,
[[False, True , False],
[False, False, False],
[True , True , False]])
>>> out = paddle.masked_fill(x, mask, 2)
>>> print(out)
Tensor(shape=[3, 3], dtype=float32, place=Place(gpu:0), stop_gradient=True,
[[1., 2., 1.],
[1., 1., 1.],
[2., 2., 1.]])
"""
check_variable_and_dtype(
x,
'x',
[
'float32',
'float64',
'int32',
'int64',
],
'paddle.tensor.manipulation.masked_fill',
)
check_variable_and_dtype(
mask,
'mask',
['bool'],
'paddle.tensor.manipulation.masked_fill',
)

if in_dynamic_mode():
value_tensor = paddle.full_like(x, value)
out = paddle.where(mask, value_tensor, x)
return out
else:
helper = LayerHelper("masked_fill", **locals())

dtype = x.dtype
check_dtype(
dtype,
'dtype',
[
'bool',
'float16',
'float32',
'float64',
'int16',
'int32',
'int64',
'uint16',
],
'full_like',
)

value_tensor = helper.create_variable_for_type_inference(x.dtype)
helper.append_op(
type='fill_any_like',
inputs={'X': [x]},
attrs={'value': value, "dtype": x.dtype},
outputs={'Out': [value_tensor]},
)

out = helper.create_variable_for_type_inference(x.dtype)

mask_shape = list(mask.shape)
value_shape = list(value_tensor.shape)
x_shape = list(x.shape)

if value_shape == x_shape and mask_shape == value_shape:
broadcast_mask = mask
broadcast_value = value_tensor
broadcast_x = x
else:
zeros_like_value = paddle.zeros_like(value_tensor)
zeros_like_x = paddle.zeros_like(x)
zeros_like_mask = paddle.zeros_like(mask)
zeros_like_mask = paddle.cast(zeros_like_mask, value_tensor.dtype)
cast_cond = paddle.cast(mask, x.dtype)

broadcast_zeros = paddle.add(zeros_like_value, zeros_like_x)
broadcast_zeros = paddle.add(broadcast_zeros, zeros_like_mask)
broadcast_value = paddle.add(value_tensor, broadcast_zeros)
broadcast_x = paddle.add(x, broadcast_zeros)
broadcast_mask = paddle.add(cast_cond, broadcast_zeros)
broadcast_mask = paddle.cast(broadcast_mask, 'bool')

helper.append_op(
type='where',
inputs={
'Condition': broadcast_mask,
'X': broadcast_value,
'Y': broadcast_x,
},
outputs={'Out': [out]},
)

return out


@inplace_apis_in_dygraph_only
def masked_fill_(x, mask, value, name=None):
"""
Inplace version of ``masked_fill`` API, the output Tensor will be inplaced with input ``x``.
Please refer to :ref:`api_paddle_masked_fill`.
Examples:
.. code-block:: python
>>> import paddle
>>> input_tensor = paddle.ones((3, 3), dtype="float32")
>>> mask_tensor = paddle.to_tensor([[True, False, True],
... [False, True, False],
... [True, False, True]])
>>> inplace_res = paddle.masked_fill_(input_tensor, mask_tensor, 0)
>>> print(inplace_res)
"""
if in_dynamic_mode():
check_variable_and_dtype(
x,
'x',
['float32', 'float64', 'int32', 'int64'],
'paddle.tensor.manipulation.masked_fill',
)
check_variable_and_dtype(
mask,
'mask',
['bool'],
'paddle.tensor.manipulation.masked_fill',
)
value_tensor = paddle.full_like(x, value)
out = paddle.where(mask, value_tensor, x)
return out


def non_negative_axis(arr, axis):
ndim = len(arr.shape)
if axis >= 0:
Expand Down