Add registration functionality to Categorical

[glitch]

Part of Issue #729 involves adding register support to GroupBy. GroupBy internal components may be of type Categorical which currently does not have a register implementation.

UPDATED 2021-04-22: The scope of this ticket has changed after discussion on registration, in-place changes, and scopes. The goal of this ticket is now:

• Use dot character to delineate sub-component names/extensions for Categorical & Strings subcomponents (i.e. for Strings object registered as foo, it will register components foo.offsets and foo.bytes)
• Static unregister functions will be added / remain in place for Categorical, Strings, and pdarray with the following changes
  • consistent naming following pattern unregister_<class>_by_name
  • Only support string arg case. Object case should use the instance unregister() methods
• Add static set of component names for Categorical so we can iterate through it where it makes sense. (NOTE: We were going to do the same for Strings, but there are only two components so it may not make sense, may need to revisit. Similarly type sensitive methods will have to do manual / hard-coded removal i.e. attach, unregister_<>_by_name)
• is_registered() will be updated to handle cases where not all of the required parts are registered.
• Regarding Issue Registration should update pdarray.name in place #729-GroupBy sub-section: We are going to skip GroupBy registration functions in favor of a Complex object implementation approach on the server.

[glitch]

I ran across some unexpected (to me) behavior today. Given a static method for unregistering by name implemented using attach like so:

def unregister_categorical_by_name(user_defined_name):
  Categorical.attach(user_defined_name).unregister()

When I run a unit test that looks like the following:

 c = ak.Categorical(ak.array([f"my_cat {i}" for i in range(1, 11)]))
self.assertTrue(c.register("test_me").is_registered(), "test_me categorical should be registered")
categorical.Categorical.unregister_categorical_by_name("test_me")
self.assertFalse(c.is_registered())

an error gets thrown from is_registered method by the pdarrayclass.list_registry() because the SymbolTable is empty.

While there is a bug in the list_registry() implementation, the real surprise here was the triggering of Arkouda server garbage collection. because the underlying object goes out of scope.

If I change the static unregister_categorical to return the object, it still gets garbage collected unless I save it to a new variable.
i.e.

c = ak.Categorical(ak.array([f"my_cat {i}" for i in range(1, 11)]))
c.register("test_me")

# This still triggers deletion
c = Categorical.unregister_categorical_by_name("test_me") 

If we change it slightly

...
c.register("test_me")
d = Categorical.unregister_categorical_by_name("test_me") 
c.is_registered() # this still works
d = c # This triggers garbage collection on Arkouda server
d.is_registered() # throws exception because symbol table is empty.

@mhmerrill What is the expected behavior on a static unregister? Should we expect it to trigger garbage collection? pdarrayclass.unregister_pdarray("name") doesn't look like it garbage collects and I'm somewhat confused by what python is doing.

[glitch]

I should note I kind of lean on the side of thinking that it's ok (the auto-garbage collection). If you are unregistering by name chances are you already lost your reference to the variable and are doing some kind of manual clean-up.

I also want to stress that doing a regular categorical_obj.unregister() does NOT trigger garbage collection, only the static by string version.

@mhmerrill If you're ok with it happening I can just add/stress it in the documentation of that method.

[mhmerrill]

@glitch unregister should not trigger garbage collection and delete the object. You should be able to register and unregister an object/array without effecting the allocation, if the variable is still live. We rely on Python to garbage collect the objects and trigger the delete messages to the server. Just to be clear delete and unregister are separate operations.

[glitch]

Mike & I had a gitter discussion and I did some more experimentation, here's the rough description of what I found
I added the following code to tests/registration_test.py

    def reg_unreg(self, name):
        b = pdarrayclass.attach_pdarray(name)
        b.unregister()
        b = None
        return None

    def test_scope_issue(self):
        a = ak.ones(3, dtype=ak.int64)
        a.register("foo")
        print("String rep of a, 1st time: ", str(a))
        self.reg_unreg("foo")
        print("String rep of a, 2nd time: ", str(a))
        print("1: ", pdarrayclass.list_registry())

test_scope_issue gets kicked off, it creates and registers pdarray a with the name "foo" and then we print out a string representation of it. Next we call a function which binds b to the pdarray on the arkouda server via attach_pdarray("foo"); b then goes out of scope. This triggers the __del__() of the attached pdarray object which in turn issues a delete to the server for "foo".

When control returns back to a and we try to print out a string representation of it, we get a RuntimError because "foo" no longer exists on the arkouda server.

EDIT: I want to note, when you're working through the instance methods such as foo_array.unregister(), everything is fine because you have the objects on hand and they aren't going out of scope.

It's really the semantics around the static method of unregistering a thing by it's name on the server. We already spoke a bit about getting rid that functionality altogether but kept it around in the case of emergency manual intervention (presumably your client died and you wanted to re-attach to some objects and clean out others) It's kind of like the warning Firefox gives you when you bring up about:config.

EDIT 2: Here's a much simpler case

        # Test generic pdarrayclass
        a = ak.ones(3, dtype=ak.int64).register("a_reg")
        print(str(a))  # Passes
        b = pdarrayclass.attach_pdarray("a_reg")
        b.unregister()
        b = None
        print(str(a))  # Fails

[glitch]

@mhmerrill and I chatted about a number of items related to PR#768, registration, and this ticket. The recap:

• We are going to use a dot character to delineate sub-component names/extensions. i.e. Strings has two pieces now they'll look like my_name.bytes and my_name.offsets. Make this change for Categorical & Strings classes
• We are going to keep the static unregister_by_name methods for Pdarray, Strings, & Categorical with the following changes
  • Static function name is going to be made consistent as unregister_*_by_name
  • They will only take a String as an argument
  • They will use the explicitly defined sub-components (i.e. 2 subs for Strings, 4 subs for Categorical), we won't have to dynamically determine them.
• For Categorical & String we are going to add a static list of component names so we can build out the sub-component names for unregister. This will make it easier to loop through the parts in both the instance level unregister and static unregister_by_name functions. EDIT due to multi-types in categorical the static unreg function needs to be a bit more explicit as does the attach method. We can use the list to iterate through register, unregister, and is_registered. String is relatively straightforward since it only has 2 items so I'm going to leave that part alone.
• We are going to skip GroupBy registration functions in favor of a Complex object implementation approach on the server.