Skip to content

Conversation

@kitagry
Copy link
Member

@kitagry kitagry commented Dec 14, 2025

Summary

This PR adds support for polars DataFrames alongside pandas DataFrames in gokart's file processors. Ta
sks can now specify their DataFrame type via the TaskOnKart[T] type parameter, and the appropriate pro
cessor will be automatically selected.

Features

  • Auto-detection: Automatically detects DataFrame type from TaskOnKart[pd.DataFrame] or TaskOnK art[pl.DataFrame] type parameters
  • Backward compatible: Defaults to pandas when no type parameter is specified

Usage Example

import polars as pl
from gokart import TaskOnKart

class MyPolarsTask(TaskOnKart[pl.DataFrame]):
    def output(self):
        return self.make_target('path/to/target.feather')

    def run(self):
        df = pl.DataFrame({'a': [1, 2, 3]})
        self.dump(df)  # Automatically uses polars-compatible processor

class MyPandasTask(TaskOnKart[pd.DataFrame]):
    def output(self):
        return self.make_target('path/to/target.feather')

    def run(self):
        df = pd.DataFrame({'a': [1, 2, 3]})
        self.dump(df)  # Uses pandas processor (default behavior)

@kitagry kitagry changed the base branch from master to add-polars-file-processor December 15, 2025 15:06
@kitagry kitagry force-pushed the support-polars branch 2 times, most recently from 3eb0e3c to 8d6b2b5 Compare December 15, 2025 15:11
@kitagry kitagry changed the title feat: support polars with auto detection from TaskOnKart's Generic type feat: add automatic processor selection based on DataFrame type parameter Dec 15, 2025
@kitagry kitagry force-pushed the add-polars-file-processor branch from 8826a52 to c39e67a Compare December 15, 2025 15:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants