Clean up backend indexing some more #10376
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dcherian
commented
May 30, 2025
| for k, v in self._variables.items(): | ||
| v = v.copy(deep=True) | ||
| res[k] = v | ||
| v._data = indexing.LazilyIndexedArray(v._data) |
There was a problem hiding this comment.
Harmonizes this test backend with all the other backends
dcherian
commented
May 30, 2025
| @@ -190,7 +190,7 @@ def ds(self): | |||
| def open_store_variable(self, name, var): | |||
| return Variable( | |||
| var.dimensions, | |||
| ScipyArrayWrapper(name, self), | |||
| indexing.LazilyIndexedArray(ScipyArrayWrapper(name, self)), | |||
There was a problem hiding this comment.
harmonized with the other backends
for more information, see https://pre-commit.ci
dcherian
commented
May 30, 2025
| @@ -717,7 +717,7 @@ def from_variables( | |||
|
|
|||
| # preserve wrapped pd.Index (if any) | |||
| # accessing `.data` can load data from disk, so we only access if needed | |||
| data = var._data.array if hasattr(var._data, "array") else var.data | |||
| data = var._data if isinstance(var._data, PandasIndexingAdapter) else var.data | |||
There was a problem hiding this comment.
@benbovy is this right?
There was a problem hiding this comment.
It looks right to me. data = var._data.array if isinstance(... would also work but the pandas Index object should be further extracted from the indexing adapter class via safe_cast_to_index.
dcherian
commented
May 30, 2025
| array = self.array[self.key] | ||
|
|
||
| # self.array[self.key] is now a numpy array when |
There was a problem hiding this comment.
This is a nice win and means we have increased internal consistency.
xref #6874 (comment) and #6874 (comment)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Harmonizes some internal code paths to make the async work less painful (#10327).
I'll add comments for the remaining changes