-
Notifications
You must be signed in to change notification settings - Fork 167
Description
Is your feature request related to a problem? Please describe.
We have two separate codepaths for extract_fields and extract_columns.
Furthermore, extract_fields requires one to write the type in the extract, but we should have that information in the fn output.
I think we can simplify the code quite a bit.
Describe the solution you'd like
A single extract (name TBD) decorator that does the following:
@extract("foo", "bar")
def foo_bar(foo: int, bar: int) -> Dict[str, int]:
return {"foo": foo, "bar": bar}
@extract("foo", "bar")
def foo_bar(foo: pd.Series, bar: pd.Series) -> pd.DataFrame:
return pd.DataFrame({"foo": foo, "bar": bar})Then it can register "extensions" to allow for us to processing different extractions. E.G. different DF types, etc...
If its a Dict[str, foo], we know it'll be a bunch of foo type outputs. If its a pd.DataFrame we know its going to be pd.Series. Etc...
Describe alternatives you've considered
Not doing this.
Additional context
Just an idea that popped up when writing some new features.