Skip to content

Add fast csp.Struct to Arrow record batch conversion #286

Open
@timkpaine

Description

@timkpaine

Now that @arhamchopra has taken care of fast json conversion, we should move on to fast Apache Arrow conversion. We should have 2 mechanisms for conversion:

  • converting an individual struct, necessary but probably less efficient overall
  • collecting lists of structs and converting to record batches on either a timer or after a certain window size

Native arrow conversion will make certain use cases substantially faster, including in our csp-based webserver framework, but should also have benefits for other adapters like Kafka and websockets.

Metadata

Metadata

Assignees

No one assigned

    Labels

    type: featureIssues and PRs related to new features

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions