Skip to content

Conversation

@glitch
Copy link
Collaborator

@glitch glitch commented Apr 29, 2021

Closes #762

  • Fixes off by one error in Segemented array peel method.
  • Updates Strings.peel unit test to check that bytes pdarrays are appropriately structured with respect to null bytes.

…l method.

Updates Strings.peel unit test to check that bytes pdarrays are
appropriately structured with respect to null bytes.
@glitch glitch requested review from mhmerrill and reuster986 April 29, 2021 15:45
# This mimics what should be stored server-side in the strings.bytes pdarray
expected_series_dec = convert_to_ord(series.to_list())
actual_dec = pda.bytes.to_ndarray().tolist()
self.assertEqual(expected_series_dec, actual_dec)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar enough with pytest to know this, but does self.assertEqual(list1, list2) actually do the element-wise comparison and check that they are all True? To me, I would be wary that it is just calling assert list1 == list2 under the hood, which would produce the wrong result (or an error).

Elsewhere, I have been using self.assertTrue((arr1 == arr2).all()) with either numpy arrays or arkouda arrays to ensure all the element-wise comparisons are happening, but if assertEqual does the right thing with python lists, I would be interested to know.

Copy link
Collaborator Author

@glitch glitch Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question, I'm fairly certain python's unittest.TestCase (where the assertion methods are coming from) really default down to the operator methods, so assertEquals(a, b) delegates to a == b which is an equivalence test vs a is b which is same ref.

I ran the following in a python console to double check
Gist here

x = ["a", "b", "c"]
y = ["a", "b", "c"]
z = ["c", "b", "a"]
x == y
> True
x is y
> False
x == z
> False

import unittest
t = unittest.TestCase()
t.assertEqual(x, y)  # passes
t.assertTrue(x is y) # fails
t.assertEqual(x, z)  # fails

# There is also a assertListEqual(a, b) which may be a bit more explicit
t.assertListEqual(x, y)  # passes
t.assertListEqual(x, z)  # fails

So while it's testing the correct thing, I think I'll push an update to improve clarity with assertListEquals

Copy link
Collaborator Author

@glitch glitch Apr 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I don't have a link to the python src handy, but for 3.7.9 unittest / case.py _baseAssertEqual the code is this

    def _baseAssertEqual(self, first, second, msg=None):
        """The default assertEqual implementation, not type specific."""
        if not first == second:
            standardMsg = '%s != %s' % _common_shorten_repr(first, second)
            msg = self._formatMessage(msg, standardMsg)
            raise self.failureException(msg)

so it is basically python's == function, we should be good with either assertEqual or assertListEqual

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok. I guess I had confused the behavior of python lists with numpy arrays and had forgotten that a == b will work with the former.

Side note: if a and b are numpy arrays (or arkouda arrays), a == b will return a bool array the same length as a and b with the result of each element-wise comparison, instead of a single bool value. So calling assertEqual(a, b) will raise a ValueError error saying "the truth value of an array is ambiguous" or something like that. Not a problem for python lists, but something to watch out for when writing unit tests on numpy/arkouda arrays.

@mhmerrill mhmerrill merged commit 2829f7d into Bears-R-Us:master Apr 29, 2021
@glitch glitch deleted the 762-Peel-OffByOne-work branch April 29, 2021 17:50
stress-tess pushed a commit that referenced this pull request Jun 15, 2021
Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
-PR #666 string test ends_with failure
-PR #781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 24, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Minimum changes to pass current tests (2/2):

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
-PR Bears-R-Us#666 string test ends_with failure
-PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 25, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
-PR Bears-R-Us#666 string test ends_with failure
-PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 28, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 28, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 29, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jun 30, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 1, 2021
- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 1, 2021
…pass tests from PR#627:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 1, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 7, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 9, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 14, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Jul 20, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Aug 5, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
stress-tess pushed a commit to stress-tess/arkouda that referenced this pull request Nov 22, 2021
…e with master and pass tests:

- Updates logging statements to use logLevel instead of v (PR Bears-R-Us#760)
- Updates `check` to `check_table` (PR Bears-R-Us#792)
- Removes outdated `Strings.attach` function which used `_` for subcomponents (new convention is `.` per PR Bears-R-Us#774)
- Makes `unregister_strings_by_name` a staticmethod

Manually applied recent changes to `SegmentedArray.chpl` because automatic merge was thrown off by addition of `class SegSArray` including:
- PR Bears-R-Us#666 string test ends_with failure
- PR Bears-R-Us#781 off by one in SegmentedArray peel method
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Off-by-one error in Strings.peel

3 participants