Datastore entry method #771
Conversation
hepengfe
added
topic: data
Codecov Report
@@ Coverage Diff @@
## master #771 +/- ##
==========================================
- Coverage 81.13% 80.97% -0.16%
==========================================
Files 249 249
Lines 18824 18693 -131
==========================================
- Hits 15272 15136 -136
- Misses 3552 3557 +5
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
This might not preserve the previous behavior in DataPack since it does not retrieve the subclasses. For example, DataPack.all_annotations is expected to return an iterator of all entries that satisfy issubclass(Annotation) (e.g., Token, Document, Sentence, etc.). Hence DataStore.all_entries("forte.data.ontology.top.Annotation") should also be able to get all subtypes of Annotation in its outputs.
tests/forte/data/data_store_test.py
Outdated
| for entry in list(self.data_store._DataStore__entry_dict.values()) | ||
| if entry[3] == "ft.onto.base_ontology.Document" | ||
| ] | ||
| ann_list = list(self.data_store._DataStore__entry_dict.values()) |
There was a problem hiding this comment.
This might not be correct if there are Links or Groups in DataStore. For example, the format of test entry with tid=10123 looks like a Group but will also be included in ann_list
There was a problem hiding this comment.
Is there any way to check Link, I found Link type entry in self.__elements is similar to a regular entry.
There was a problem hiding this comment.
According to the docstring in DataStore.__init__, a Link entry has the following format: [<parent_tid>, <child_tid>, <tid>, <type_name>, <attr_1>, <attr_2>, ..., <attr_n>, index_id]. So I guess you can still use <type_name> to check if it's a link.
There was a problem hiding this comment.
I think the rest of the PR is fine. What's the current status of this comment?
@mylibrar feel free to resolve the comments if you are fine with them, and you can approve it after your review too.
…inus the deletion count for non-annotation-like entries
…w DataStore implementation
There was a problem hiding this comment.
Looks good to me, but will leave it up to @mylibrar to resolve all comments and approve it.
This PR fixes #767.
Description of changes
all_entries(entry_type_name)num_entries(entry_type_name)Possible influences of this PR.
Describe what are the possible side-effects of the code change.
Test Conducted
all_entries(entry_type_name)returns an iterator of raw entry data in theListformat ofentry_type_namenum_entries(entry_type_name)returns a correct number of entries ofentry_type_name