GH-615: Produce Avro core data types out of Arrow VSR #638

martin-traverse · 2025-02-26T02:52:28Z

What's Changed

Per discussion in #615 , here is a first take on the core producers to generate Avro data from Arrow vectors. There are a few points I'd like to clarify before going further:

Nullability. Avro only understands nullable types as unions but that is not normally how they will be if the data comes from other sources. I have added a special NullableProducer to handle nullable vectors which are not unions. We will need something equivalent in the consumers and probably a setting in the AvroToArrowConfig to control it on read, defaulting to current behaviour. I have also added special handling for nullable unions, because unions can't be nested (i.e. you can't nest "type | null" as a type inside a union). I can add consumers to handle both (unions and regular types) for review, if that sounds right? At the moment the schema for nullable fields gets quite mangled on a round trip!
Arrow has a lot more types than Avro, at the level of minor / vector types. Going Avro -> Arrow we just pick the direct equivalent. Going Arrow -> Avro, we could cast silently if there is no loss of precision. E.g. TinyInt and SmallInt -> Int and so on. For types like e.g. Decimal256 and LargeVarChar we could write out safely but would need support in the consumers to read back the wider types. I could start by adding the safe conversions now and we could come back to the wide types in a later PR maybe?
I have made Producer inherit AutoClosable the same as Consumer does. Not sure if that is always what you want though - it will free the buffers in the VSR, but you might want to keep the VSR after you are finished with IO. Do we need something like detachValueVector() to go with resetValueVector()? Calling close() after detach would not affect the vector.
Type information is inferred from the list of vectors, using minor types. We'll also need to generate the Avro schema, the input for that would be a list of fields. I haven't done it yet but will do if that sounds right.
Dictionary encoding for enums not implemented yet, I'll add it if the rest looks good. Caveat is that dictionaries must be fixed before the encoding starts if we are writing out the whole file in one go (i.e. if the Avro schema is at the start of the container file). If the schema is saved separately that limitation need not apply, we could provide the schema once encoding is finished.

Please do let me know if this is going in the right direction + any comments. If it is I will add the missing pieces and start the exhaustive test coverage to mirror the consumers. Once it's done this PR should get us to the point where we can round trip the contents of an individual block for most data types, but it does not address the container format.

Closes #615.

lidavidm

This looks reasonable to me overall, thanks.

We should document exactly how we handle type conversions, especially around nulls
I'm OK punting on more complicated types for now
I'm wondering if we shouldn't just make the producer not own the vector at all (i.e. it's still the caller's responsibility to close the vector). I think this makes the most sense, e.g. if you have a file reader, the reader already owns the vector, so using it with the avro producer would mean a double-close (or you would have to explicitly unload the data and load it somewhere else)
I'm OK punting on dictionaries for the time being too if you want

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/Producer.java

lidavidm · 2025-02-26T06:19:13Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/Producer.java

+  void skipNull();
+
+  /** Set the position to read value from vector. */
+  void setPosition(int index);


I think we've been trying to be future-proof and use long for indices now (since eventually once we support the new memory APIs we'll be able to address more than 2GiB of memory at once)

Ok, that makes sense. We should really update the consumers to match. If you like, I could make both int and long available as overloads, and keep a long internally. That way we can update the consumer interface to match, without breaking the API. Would you like me to do that?

That sounds good to me.

Per my comment below, this proved more tricky than I first thought. In light of that, are you happy to stick with int for now?

Ok, let's leave it then. (We really need to fix this more generally...)

lidavidm · 2025-02-26T06:20:17Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/BaseAvroProducer.java

+  @Override
+  public void close() throws Exception {
+    vector.close();
+  }
+
+  @Override
+  public boolean resetValueVector(T vector) {
+    this.vector = vector;
+    this.currentIndex = 0;
+    return true;
+  }


Hmm. Should resetting close the prior vector? If not, then maybe we also shouldn't be closing the vector here (i.e. it's the upper level code's responsibility to manage the vector lifetime and the producer is just temporarily holding the vector, not owning it)

I think I agree with you that producers should not own / close the vectors. In which case, this becomes a non-issue! Are you happy for me to update on those lines? The producers will still own a reference, but we could get rid of all the close() methods and remove AutoClosable from the base interface. Would you like me to do that?

Yeah, let's drop close() and AutoClosable!

Done, close() and AutoClosable are gone :-)

lidavidm · 2025-02-26T06:23:08Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/CompositeAvroProducer.java

+    try {
+      AutoCloseables.close(producers);
+    } catch (Exception e) {
+      throw new RuntimeException("Error occurs in close.", e);


Normally close is declared to throw Exception anyways - do we need to rewrap this in RuntimeException?

Otherwise use AutoCloseables.closeNoChecked

Per above, if producers are not owning / closing the vectors this issue goes away. Otherwise yes this would make sense. Let me know, which you'd like me to do.

Per above, close and AutoCloable are gone

martin-traverse · 2025-02-26T15:31:23Z

Thanks so much for all the comments! I've put questions in line, will wait for your answers on how to address those points.

In the meantime I will add the schema generator and producers for the narrow primitive types, as well as some top level doc comments describing the translation. I'll make a start on the test coverage as well....

wgtmac · 2025-02-28T02:36:52Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/BaseAvroProducer.java

+
+  @Override
+  public void setPosition(int index) {
+    currentIndex = index;


Is there any restriction to the new index? Do we expect index < currentIndex?

I have added check, index < 0 or index > value count will throw an illegal argument exception.

I've allowed index == value count, because that is the final state after production is complete. Since the producer can get into that state naturally, and it has a valid meaning (production complete), it seemed wrong to disallow it. Thoughts welcome on that point!

Worth noting, there are several ways value count can not be set on the vector, either because client code doesn't set it, or a writer doesn't (e.g. UnionMapWriter does not update the entries data vector count) in which case the cause of the error may not be obvious (at least it wasn't for me)! Probably still the check is the right thing to do, but something to be aware of.

wgtmac · 2025-02-28T02:42:03Z

.../avro/src/main/java/org/apache/arrow/adapter/avro/producers/logical/AvroDecimalProducer.java

+      super(vector);
+      Preconditions.checkArgument(
+          vector.getTypeWidth() <= 16, "Decimal bytes length should <= 16.");
+      reuseBytes = new byte[vector.getTypeWidth()];


Is this correct? I don't think the type width is constant for each value. It might be similar to what you did in AvroBytesProducer.

Decimal is fixed width byte (either 16 or 32)

Well, I think I have mixed up the requirement from Iceberg which says Stored as fixed using the minimum number of bytes for the given precision.

Apologies, have removed this, I was mirroring consumers which need to handle both bytes and fixed inputs, but for producers we can always used fixed and the type width is always known.

wgtmac · 2025-02-28T02:54:32Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/AvroNullableProducer.java

+  @Override
+  public void produce(Encoder encoder) throws IOException {
+    if (vector.isNull(currentIndex)) {
+      encoder.writeInt(1);


Does it mean that you placed [type, null] as the union type? Is it a good convention to put null before type so the default value is null when reader type does not match the writer type?

I tried doing that, it broke the consumers and the data would not do round trip. In Avro this is considered as the difference between "nullable" and "optional", with "optional" being the semantics you are suggesting. I have kept it as nullable for now. I do think we will need to follow up with a PR on the consumers, particularly to address null handling. Nullable types get mangled in the schema round trip at the moment!

wgtmac · 2025-02-28T02:56:14Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/ArrowToAvroUtils.java

+      case MAP:
+        MapVector mapVector = (MapVector) vector;
+        StructVector entryVector = (StructVector) mapVector.getDataVector();
+        VarCharVector keyVector = (VarCharVector) entryVector.getChildrenFromFields().get(0);


What if the key is not a string type?

Good point! I put a check in the schema generation but not here. In Avro keys have to be string so if that is not the case I have thrown an error. Please leave this open and I will fix - out of time for today!

I added a check for this, now it will throw illegal argument if the key type is not string (which is required by Avro)

martin-traverse · 2025-03-02T16:59:02Z

Hello. I have pushed a second iteration for review. Major updates are:

Added schema generation, to convert Arrow List of Fields to Avro Schema (record schema). I added doc comments with the type mapping. Schema is built from ArrowType while physical producers use MinorType, that bugged me at first but on reflection probably seems correct?
Updated the producers with a more complete set of types, including all widths of float / int types and logical types for all the date / time types (but not duration as of yet). Would appreciate a set of eyes on the timezone mapping for TZ aware timestamps! Per my reading of the Avro spec this is correct but worth double checking.

I have tried to make it so most Arrow types will map to an Avro type if there is a suitable target. The basic approach is:

If a direct equivalent exists, use that
Next, choose an equivalent with no loss of precision (e.g. small int -> int)
Next, choose an equivalent with loss of precision but no loss of range (e.g. time nano -> time micros)
Finally, choose an equivalent with the best match and error on overflow (e.g. unsigned long -> long)

I'm not sure about the last one, would it be better just to say we can't convert those types and throw an error up front?

Large types (for text, binary, list) are not supported yet, there will be some extra thought needed on getting very large buffers between the two APIs. Interval / duration also not done yet, Arrow Interval resembles Avro Duration but is not an exact match and Arrow duration doesn't really have an equivalent that I could see. View types also not included as of now.

I tried to move over to using long as the current row index, however a lot of the high-level APIs on the vector types use ints. For working directly with primitive types I was able to go to the underlying buffer (there were reasons to do that anyway in a few cases), but for the complex types that gets trickier because the type of the child vectors is not known. Is it ok to keep the interface using an int for current index for now? Most of the primitive producers are updated anyway, so the work to move over later would not be too much.

I've run out of time for today. Next I plan to do the tests - like the consumers, will be exhaustive across types, combinations of nullability etc with some edge cases. Then the last thing I'd like to come back to on this PR is enums / dict encoding, just in case that has any impact on the structure of things would be good to know while it's still easy to change things.

I have addressed some of the comments but not all, I will work through the rest but wanted to share this to check I'm still going in the right direction!

lidavidm · 2025-03-05T03:39:43Z

Sorry, I've gotten caught up in my main job - I'll try to circle back here soon

lidavidm · 2025-03-05T06:11:19Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/Producer.java

+  void skipNull();
+
+  /** Set the position to read value from vector. */
+  void setPosition(int index);


Ok, let's leave it then. (We really need to fix this more generally...)

lidavidm · 2025-03-05T06:12:18Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/Producer.java

+   *
+   * @return true if reset is successful, false if reset is not needed.
+   */
+  boolean resetValueVector(T vector);


Hmm, do any of the implementations ever return false?

Nothing ever returns false and the value is never used. I have changed this to return void.

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/AvroStructProducer.java

lidavidm · 2025-03-05T06:45:45Z

...avro/src/main/java/org/apache/arrow/adapter/avro/producers/logical/AvroTimeNanoProducer.java

+  @Override
+  public void produce(Encoder encoder) throws IOException {
+    long nanos = vector.getDataBuffer().getLong(currentIndex * (long) TimeNanoVector.TYPE_WIDTH);
+    long micros = nanos / NANOS_PER_MICRO;


Maybe we should throw on truncation to be safe?

I don't think this will truncate since we are going nanos -> micros so there is more range available. There is loss of precision - I think that is acceptable, since Avro does not have a time-nanos type (at least not yet)!

lidavidm · 2025-03-05T06:46:23Z

...vro/src/main/java/org/apache/arrow/adapter/avro/producers/logical/AvroTimeMilliProducer.java

+ */
+public class AvroTimeMilliProducer extends AvroIntProducer {
+
+  // Time in milliseconds stored as integer, matches Avro time-millis type


Hmm, Arrow timestamp type is always 8 bytes, so won't this suffer a range issue?

I think time for seconds / millis are four byte width? TimeMilliVector. So shouldn't be a range issue for this one.

lidavidm · 2025-03-05T07:01:19Z

...o/src/main/java/org/apache/arrow/adapter/avro/producers/logical/BaseTimestampTzProducer.java

+abstract class BaseTimestampTzProducer<T extends TimeStampVector>
+    extends BaseAvroProducer<TimeStampVector> {
+
+  // Convert TZ values to UTC to encode Avro timestamp types


Arrow stores zoned timestamps in UTC already. The time zone associated with the type is purely used for display. So there should be no need for conversion.

https://github.com/apache/arrow/blob/601fde9033b5def7e667ffec70c0acacd1b9ad50/format/Schema.fbs#L284-L302

This is great! I took out all the timezone conversion logic, the TZ aware producers are much simpler now :-)

lidavidm · 2025-03-05T07:04:19Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/AvroUint4Producer.java

+  @Override
+  public void produce(Encoder encoder) throws IOException {
+    int unsigned = vector.getDataBuffer().getInt(currentIndex * (long) UInt4Vector.TYPE_WIDTH);
+    long signed = unsigned & 0xffffffffL;


Is this https://docs.oracle.com/javase/8/docs/api/java/lang/Integer.html#toUnsignedLong-int- ?

Thanks, updated the code to use that method.

lidavidm · 2025-03-05T07:04:56Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/AvroUint2Producer.java

+  @Override
+  public void produce(Encoder encoder) throws IOException {
+    short unsigned = vector.getDataBuffer().getShort(currentIndex * (long) UInt2Vector.TYPE_WIDTH);
+    int signed = unsigned & 0xffff;


Same here, is Short#toUnsignedInt applicable?

Thanks, updated the code to use that method.

lidavidm · 2025-03-05T07:06:12Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/ArrowToAvroUtils.java

+   * Arrow data types.
+   *
+   * <ul>
+   *   <li>ArrowType.Null --> NULL


not to nitpick the formatting but a table might be better? (also does Javadoc allow unclosed tags like this?)

I've converted the type mapping info into and HTML table - shows up nicely in the IDE auto complete :-)

lidavidm · 2025-03-05T07:07:01Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/ArrowToAvroUtils.java

+   * </ul>
+   *
+   * <p>Nullable fields are represented as a union of [null | base-type]. Special treatment is given
+   * to nullability of unions - a union is considered nullable if the union field is nullable or any


FWIW, I think nullable unions aren't semantically valid - unions have no validity bitmap in Arrow. The only way to represent this is with a nullable child field

I checked the spec and yes you are correct! I have updated the logic accordingly

Can we update the docstring too?

I have updated the comment on unions. Also updated the comment for nullable types, which are encoded as [ base-type | null ], per the discussion above.

martin-traverse · 2025-03-05T21:21:42Z

Thanks @lidavidm for all the comments - really helpful :-) I'll push an update in a few days to address these and also add the tests, will probably be after the weekend.

martin-traverse · 2025-03-24T18:27:04Z

Hi @lidavidm , @wgtmac - Have pushed another update, here is a quick summary:

I have been through all the comments above and responded. Please do come back on any you think need further attention!
I have added tests for both schema and data production. Tests cover primitive types, logical types and containers of primitive and logical types. Some edges cases (max int values, UTF 8 text) are included.
Tests do not include nested containers (e.g. list of lists), these may work but I've not tried it. Negative test cases are also not included (i.e. invalid / illegal / badly initialised data structures).
Dictionary encoding not supported yet (we could push this to a second PR)
Nullable types. I have [type, null] for nullable types instead of [null, type]. There was a comment on this. The current reader framework won't allow the latter, so we can't round-trip if we do it that way, but if you think it is preferable we could do it anyway now we have full coverage on the producers. Anyway I think a separate PR is needed to address issues in the consumers. Lmk what you'd like to do on this.
Union data. Union schema production is working fine, but union data I'm really struggled with. Looking at the format spec, it seems to suggest the type buffer holds an index into the child vectors which is simple enough. But the Java implementation has a bunch of extra logic to do with type IDs that need to line up on their ordinal numbers, then the writers have different logic again for allocating fields. The dense writer seems to have a bug where writers are not initialized. I tried ignoring the writers and just setting up the buffers directly as per the spec, but the vector classes need the type ID mapping to operate. Can you shed any light on how the Java union types are supposed to work? Is there a canonical example somewhere I can work from?

Apologies for the delay, lots else going on as usual but I think this is getting closer now. If you let me know what else you'd like to see for this to be considered ready, I'll try to turn it around as quickly as I can.

lidavidm · 2025-03-25T06:05:58Z

I'm fine splitting things into other PRs if it makes your life easier.

It is known that the Java Union implementation is wrong, unfortunately. See #108

If you want to punt on Union for now that's fine by me.

I'm a bit backlogged and will try to re-review when I get a chance but that may not be for a week or two, sorry.

…tor classes)

martin-traverse · 2025-03-25T09:39:56Z

Thanks @lidavidm - good to know it is not just me going crazy! I have removed support of data production for unions for now. Also rebased on master.

Totally understand the time pressure. I might work on some of the other pieces in the meantime as separate PRs if that's OK, to have a couple of bits ready for you to look at :-)

Also read the thread on Unions. My company uses Arrow Java heavily in our product - time is a big issue for us as well but if I can find some I'll try to raise a few fixes that we're interested in.

martin-traverse · 2025-03-25T11:42:23Z

Hi @lidavidm - There were two test failures in CI relating to nano precision time values that didn't show up in my local machine. I have pushed a fix.

martin-traverse · 2025-03-25T14:32:29Z

CI looks good now - just the link missing on the PR to issue #615, sorry I don't have permission to add that!

martin-traverse · 2025-03-26T08:27:49Z

I think the PR check hit this issue - could you add the milestone to the issue manually for now?

lidavidm · 2025-03-26T08:47:38Z

Please ignore the check 😬

martin-traverse · 2025-03-26T19:17:47Z

Please ignore the check 😬

Ok great - in that case I think the CI is done :)

lidavidm · 2025-03-27T04:57:09Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/producers/AvroListProducer.java

+
+  @Override
+  public void setPosition(int index) {
+    if (index < 0 || index > vector.getValueCount()) {


Should these checks be index >= getValueCount()?

Ah, I see your comment below now. It might be worth noting this somewhere (perhaps the docstring of the base class?)

I have added a doc comment on the base class as you suggested

lidavidm · 2025-03-27T05:16:23Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/ArrowToAvroUtils.java

+   * </ul>
+   *
+   * <p>Nullable fields are represented as a union of [null | base-type]. Special treatment is given
+   * to nullability of unions - a union is considered nullable if the union field is nullable or any


Can we update the docstring too?

lidavidm · 2025-03-27T05:18:14Z

adapter/avro/src/main/java/org/apache/arrow/adapter/avro/ArrowToAvroUtils.java

+        if ((timeType.getUnit() == TimeUnit.SECOND || timeType.getUnit() == TimeUnit.MILLISECOND)) {
+          return builder.intBuilder().prop("logicalType", "time-millis").endInt().noDefault();
+        } else {
+          // All other time types (sec, micro, nano) are encoded as time-micros (LONG)


nit: comment is out of sync with code

Comments updated

martin-traverse · 2025-03-27T19:08:30Z

Hi - I think I have addressed all the new comments. Also found an issue in the fixed list producer - I updated the test cases to show this up and included a fix. Fairly sure all the complex type producers are correct now but would be worth a set of eyes!! The four I have are list, fixed size list, map and struct.

Please let me know if there is anything else needs addressing.

martin-traverse · 2025-04-01T19:38:02Z

Hello - Thought I would make a start on step 2 in the Avro series. I have raised #698 to describe what I'm thinking.

Still very happy to do another round of updates on this one if there are more comments as / when people have time to review - lmk :-) But since there are a few more PRs in the series to get the full capability I thought it was worth starting on the second one!

…his seems to be what the Avro framework intends)

martin-traverse · 2025-04-01T21:44:52Z

This last update uses Avro's logical type classes instead of setting the raw schema props. The output to disk is the same, but if you want to use the generated Avro Schema directly these structures should be available. Needed for round-trip testing.

lidavidm · 2025-04-02T00:36:48Z

The proposal looks good to me. Sorry, I will try to give a hopefully final review here this week - as you have seen I tend to be scattered about so I appreciate your patience

lidavidm · 2025-04-02T05:12:25Z

Ok well these CI failures aren't related to the PR, merging...

martin-traverse · 2025-04-02T07:37:27Z

Hi @lidavidm - Thanks so much for merging, I'm really happy to get this first piece in :-) Have started work on the second bit, once the main pieces are ready I will raise a new PR.

I'm snowed under too so may have to put it down and come back a week later etc., very happy to work in async mode!

martin-traverse mentioned this pull request Feb 26, 2025

Avro write support - will you accept a patch? #615

Closed

martin-traverse changed the title ~~Feature / Produce Avro core data types out of Arrow VSR~~ GH-615 : Produce Avro core data types out of Arrow VSR Feb 26, 2025

martin-traverse changed the title ~~GH-615 : Produce Avro core data types out of Arrow VSR~~ GH-615: Produce Avro core data types out of Arrow VSR Feb 26, 2025

This comment has been minimized.

Sign in to view

lidavidm reviewed Feb 26, 2025

View reviewed changes

wgtmac reviewed Feb 28, 2025

View reviewed changes

lidavidm reviewed Mar 5, 2025

View reviewed changes

lidavidm added the enhancement PRs that add or improve features. label Mar 6, 2025

martin-traverse marked this pull request as ready for review March 24, 2025 19:44

martin-traverse requested review from laurentgo and jbonofre as code owners March 24, 2025 19:44

Martin Traverse and others added 13 commits March 25, 2025 08:58

Avro producers - interface and base class

42187d7

Avro producers - basic types

c97c968

Avro producers - logical types

2b4d6ea

Add the composite AVRO producer

59374c1

Add producers packages to the module exports

d79a420

Protected constructor on abstract class

e0ad9ca

Use byte[] for fixed producer, matching fixed consumer

01d7c52

Add the AVRO enum producer

560f49f

Add skip and set methods to the interface and base class

26c9ba4

Add a specialized nullable producer

9d3bbe9

Fix build warnings

9aeabf4

Fix comment warnings

6a6804b

Utils to create producers for primitive and logical types

ef60f39

Remove support for production of union data (pending fixes in the vec…

81065bf

…tor classes)

martin-traverse force-pushed the feature/avro-production branch from b2b81ee to 81065bf Compare March 25, 2025 09:06

Fix nano precision test in CI for time types

e36682a

github-actions bot added this to the 18.3.0 milestone Mar 25, 2025

lidavidm approved these changes Mar 27, 2025

View reviewed changes

martin-traverse added 6 commits March 27, 2025 18:37

Improve tests for nullable lists and maps

0b59da8

Set value counts for all complex vectors in the tests

bdb5a92

Fix handling of child index updates for fixed size lists

ccf9c3a

Fixes for PR comments

7f0e3f0

Apply spotless

98a99a5

Correct comment on schemas for nullable fields

e1ca9b2

martin-traverse mentioned this pull request Apr 1, 2025

Avro support - Improve existing read capabilities #698

Closed

martin-traverse added 3 commits April 1, 2025 22:03

Eliminate duplication in schema building methods

e39de26

Test logical types using built-in field (not a regular schema prop)

956e867

Use structured types instead of raw props for logical type schemas (t…

00bbef0

…his seems to be what the Avro framework intends)

lidavidm approved these changes Apr 2, 2025

View reviewed changes

lidavidm merged commit c57184a into apache:main Apr 2, 2025
23 of 25 checks passed

martin-traverse deleted the feature/avro-production branch April 2, 2025 07:36

GH-615: Produce Avro core data types out of Arrow VSR #638

GH-615: Produce Avro core data types out of Arrow VSR #638

Conversation

martin-traverse commented Feb 26, 2025 • edited by lidavidm Loading

What's Changed

This comment has been minimized.

lidavidm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martin-traverse commented Feb 26, 2025

Choose a reason for hiding this comment

martin-traverse Mar 24, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martin-traverse commented Mar 2, 2025 • edited Loading

lidavidm commented Mar 5, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martin-traverse commented Mar 5, 2025

martin-traverse commented Mar 24, 2025 • edited Loading

lidavidm commented Mar 25, 2025

martin-traverse commented Mar 25, 2025

martin-traverse commented Mar 25, 2025

martin-traverse commented Mar 25, 2025

martin-traverse commented Mar 26, 2025

lidavidm commented Mar 26, 2025

martin-traverse commented Mar 26, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martin-traverse commented Mar 27, 2025

martin-traverse commented Apr 1, 2025

martin-traverse commented Apr 1, 2025

lidavidm commented Apr 2, 2025

lidavidm commented Apr 2, 2025

martin-traverse commented Apr 2, 2025

martin-traverse commented Feb 26, 2025 •

edited by lidavidm

Loading

martin-traverse Mar 24, 2025 •

edited

Loading

martin-traverse commented Mar 2, 2025 •

edited

Loading

martin-traverse commented Mar 24, 2025 •

edited

Loading