You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When bringing up the arrow logging for image_files I noticed that objectron demo breaks down on bucket-splits.
The symptom is that images appear to stop updating, but the reality seems to be that all of the images in the bucket start receiving a single TensorData payload which is the concatenation of all the buckets.
Interestingly, see that the tensor-id values continue to change even while the binary payload stays fixed. so the implementation is at least partially working for some of the struct fields.
This feels like something is breaking in a nested slice -- very likely a bug in either arrow or arrow2-convert.
The text was updated successfully, but these errors were encountered:
After much digging I concluded this was an arrow2-convert problem. I should have searched the issues in the first place. Turns out this is a long-standing ticket upstream. DataEngineeringLabs/arrow2-convert#53
jleibs
changed the title
Datastore corruption on image_files during bucket-split
Datastore slice-returns on compacted buckets breaks union components.
Jan 13, 2023
When bringing up the arrow logging for image_files I noticed that objectron demo breaks down on bucket-splits.
The symptom is that images appear to stop updating, but the reality seems to be that all of the images in the bucket start receiving a single TensorData payload which is the concatenation of all the buckets.
Interestingly, see that the tensor-id values continue to change even while the binary payload stays fixed. so the implementation is at least partially working for some of the struct fields.
This feels like something is breaking in a nested slice -- very likely a bug in either arrow or arrow2-convert.
The text was updated successfully, but these errors were encountered: