[CodeCamp #18] add ops bbox_overlaps

[enemy1205]

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Add CPU implementation of ops bbox_overlaps

cpu version

pure python version

Modification

Please briefly describe what modification is made in this PR.

BC-breaking (Optional)

No

Use cases (Optional)

passed tests/test_ops/test_bbox.py

Checklist

Before PR:

• I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
• Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
• Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
• New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects, like MMDet or MMCls.
• CLA has been signed and all committers have signed the CLA in this PR.

[CLAassistant]

All committers have signed the CLA.

[grimoire]

Since we already have a pure PyTorch implementation, could you provide a benchmark for the speed-up of the new op?

[enemy1205]

Since we already have a pure PyTorch implementation, could you provide a benchmark for the speed-up of the new op?

I tested it using pytest-benchmark as follows :
Firstly ,in file tests/test_ops/test_bbox.py , since the two calls are the same function, only the parameters are different, I commented out the semi-precision call in order to prevent it from averaging

    @pytest.mark.parametrize('device', [
        'cpu',
        pytest.param(
            'cuda',
            marks=pytest.mark.skipif(
                not IS_CUDA_AVAILABLE, reason='requires CUDA support')),
        pytest.param(
            'mlu',
            marks=pytest.mark.skipif(
                not IS_MLU_AVAILABLE, reason='requires MLU support')),
        pytest.param(
            'mps',
            marks=pytest.mark.skipif(
                not IS_MPS_AVAILABLE, reason='requires MPS support'))
    ])
    def test_bbox_overlaps_float(self, device,benchmark):
        benchmark(self._test_bbox_overlaps,device, dtype=torch.float)

    # @pytest.mark.parametrize('device', [
    #     pytest.param(
    #         'cuda',
    #         marks=pytest.mark.skipif(
    #             not IS_CUDA_AVAILABLE, reason='requires CUDA support')),
    #     pytest.param(
    #         'mlu',
    #         marks=pytest.mark.skipif(
    #             not IS_MLU_AVAILABLE, reason='requires MLU support'))
    # ])
    # def test_bbox_overlaps_half(self, device,benchmark):
    #     benchmark(self._test_bbox_overlaps,device, dtype=torch.half)

Then ,in file mmcv/ops/bbox.py
In the end of the function, I implemented calls to the C++ and pure pytorch versions by commenting out the other one in turn

    mode_dict = {'iou': 0, 'iof': 1}
    assert mode in mode_dict.keys()
    mode_flag = mode_dict[mode]
    # Either the boxes are empty or the length of boxes' last dimension is 4
    assert (bboxes1.size(-1) == 4 or bboxes1.size(0) == 0)
    assert (bboxes2.size(-1) == 4 or bboxes2.size(0) == 0)
    assert offset == 1 or offset == 0

    rows = bboxes1.size(0)
    cols = bboxes2.size(0)
    if aligned:
        assert rows == cols
        ious = bboxes1.new_zeros(rows)
    else:
        ious = bboxes1.new_zeros((rows, cols))

    if rows * cols == 0:
        return ious
    
    # Pure Pytorch
    # return _bbox_overlaps_cpu(
    #         bboxes1, bboxes2, mode=mode, aligned=aligned, offset=offset)
    
    # C++ version
    ext_module.bbox_overlaps(
        bboxes1, bboxes2, ious, mode=mode_flag, aligned=aligned, offset=offset)
    return ious

By executing pytest tests/test_ops/test_bbox.py
The test results are as follows:

The top is the C++ version and the bottom is the pure Pytorch version, so there is some speed optimization

[grimoire]

Assert the input and output datatype here.

[grimoire]

The data size in the unit test is relatively small. Have you benchmark it on large datas (1000+ boxes)?

[enemy1205]

benchmark(self._test_bbox_overlaps,device, dtype=torch.float)

As you say，Pytorch optimizes large amounts of data far more efficiently than C++ loops
test code:

        b1 = torch.tensor([[1.0+i, 1.0+i, 3.0+i, 3.0+i] for i in range(100000)]).to(device).type(dtype)
        b2 = torch.tensor([[2.0+i, 2.0+i, 4.0+i, 4.0+i] for i in range(100000)]).to(device).type(dtype)
        should_output = np.array([1/7]*100000)
        out = bbox_overlaps(b1, b2, aligned=True)
        assert np.allclose(out.cpu().numpy(), should_output, 1e-2)

elapsed time

// Pure C++ loop
------------------------------------------------------ benchmark: 1 tests ------------------------------------------------------
Name (time in ms)                      Min       Max      Mean   StdDev    Median      IQR  Outliers     OPS  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------
test_bbox_overlaps_float[cpu]     367.5079  456.5294  405.8569  36.7101  396.9816  60.0164       2;0  2.4639       5           1
--------------------------------------------------------------------------------------------------------------------------------
// C++  with openmp (#pragma omp parallel for)
------------------------------------------------------ benchmark: 1 tests ------------------------------------------------------
Name (time in ms)                      Min       Max      Mean   StdDev    Median      IQR  Outliers     OPS  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------
test_bbox_overlaps_float[cpu]     368.4876  410.3432  381.1380  16.7261  375.3134  14.5139       1;1  2.6237       5           1
--------------------------------------------------------------------------------------------------------------------------------
// Pure Pytorch
------------------------------------------------------ benchmark: 1 tests ------------------------------------------------------
Name (time in ms)                      Min       Max      Mean   StdDev    Median      IQR  Outliers     OPS  Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------
test_bbox_overlaps_float[cpu]     191.3801  229.2963  204.5748  14.8945  200.2914  17.5259       1;0  4.8882       5           1
--------------------------------------------------------------------------------------------------------------------------------

So in practice, the pytorch version may indeed be more efficient

[grimoire]

LGTM