Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ All notable changes to this project are documented here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project
adheres to [Semantic Versioning](https://semver.org/).

## [Unreleased]

### Added
- Added a runnable PyJanitor interoperability example that demonstrates both
tool orderings while keeping PyJanitor optional.

## [2.0.0] - 2026-07-20

Remediation of the July 2026 v1.2.0 production-readiness audit: the unsafe
Expand Down
25 changes: 25 additions & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ directory has narrated Jupyter walkthroughs.
| `06_large_dataset.py` | Cleaning a large synthetic dataset, with timing |
| `07_pandas_integration.py` | Dropping freshdata into an existing pandas workflow |
| `08_csv_automation.py` | Batch CSV cleaning automation with audit logs |
| `09_pandera_recipe.py` | Validating with pandera before and after freshdata cleaning |
| `10_pyjanitor_interop.py` | Combining PyJanitor transforms with FreshData quality repair |

## Missing-value cleaning

Expand Down Expand Up @@ -72,3 +74,26 @@ for path in Path("inbox").glob("*.csv"):
out.to_csv(Path("clean") / path.name, index=False)
print(path.name, "→", cleaner.report_.summary().splitlines()[0])
```

## PyJanitor interoperability

FreshData and [PyJanitor](https://pyjanitor-devs.github.io/pyjanitor/) solve
different parts of a pandas workflow. Use PyJanitor for explicit reshaping and
method-style transformations; use FreshData for evidence-based quality
detection, conservative repair, and an auditable report.

The runnable [`10_pyjanitor_interop.py`](https://github.com/FreshCode-Org/freshdata/blob/main/examples/10_pyjanitor_interop.py)
example demonstrates both useful orderings on one small inline DataFrame:

- **PyJanitor then FreshData:** normalize the input shape first, then detect and
repair quality issues in the resulting columns.
- **FreshData then PyJanitor:** clean and record the quality decisions first,
then add an explicit presentation transform to the cleaned result.

PyJanitor remains an optional dependency. FreshData 2.0 supports pandas 1.5–2.x;
install the compatible PyJanitor 0.31 line to run the example:

```bash
pip install "pyjanitor<0.32"
python examples/10_pyjanitor_interop.py
```
74 changes: 74 additions & 0 deletions examples/10_pyjanitor_interop.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
"""Use FreshData and PyJanitor in the same pandas workflow.

PyJanitor is optional; install it alongside FreshData to run this example:

pip install freshdata-cleaner "pyjanitor<0.32"

FreshData 2.0 supports pandas 1.5–2.x; PyJanitor 0.31 is the compatible line
verified for this example.

Use PyJanitor for explicit DataFrame reshaping and method-style transforms.
Use FreshData for evidence-based quality detection, conservative repair, and
an audit report. Either tool can go first, depending on which step needs the
other tool's output.
"""

import pandas as pd
from janitor import clean_names, transform_column # type: ignore[import-untyped]

import freshdata as fd


def build_frame() -> pd.DataFrame:
"""Return one small frame used by both ordering examples."""
return pd.DataFrame(
{
" Customer ID ": ["C-01", "C-02", "C-03", "C-03"],
"Order Amount": ["12.50", "n/a", "18.00", "18.00"],
"Region Name": [" North ", "south", "NORTH", "NORTH"],
}
)


def pyjanitor_then_freshdata(raw: pd.DataFrame) -> tuple[pd.DataFrame, fd.CleanReport]:
"""Shape labels explicitly, then detect and repair quality issues."""
shaped = clean_names(raw, remove_special=True, strip_underscores="both")
return fd.clean(
shaped,
id_columns=("customer_id",),
return_report=True,
)


def freshdata_then_pyjanitor(raw: pd.DataFrame) -> tuple[pd.DataFrame, fd.CleanReport]:
"""Repair with an audit trail, then add an explicit presentation column."""
cleaned, report = fd.clean(
raw,
id_columns=("customer_id",),
return_report=True,
Comment thread
coderabbitai[bot] marked this conversation as resolved.
)
enriched = transform_column(
cleaned,
column_name="region_name",
function=lambda value: value.strip().title(),
dest_column_name="region_label",
)
return enriched, report


def main() -> None:
raw = build_frame()

cleaned, clean_report = pyjanitor_then_freshdata(raw)
print("=== PyJanitor then FreshData ===")
print(cleaned)
print(clean_report.summary())

enriched, enriched_report = freshdata_then_pyjanitor(raw)
print("\n=== FreshData then PyJanitor ===")
print(enriched)
print(enriched_report.summary())


if __name__ == "__main__":
main()
9 changes: 9 additions & 0 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,14 @@ python examples/01_missing_values.py
| [`07_pandas_integration.py`](07_pandas_integration.py) | Dropping freshdata into an existing pandas workflow |
| [`08_csv_automation.py`](08_csv_automation.py) | Batch CSV cleaning automation with audit logs |
| [`09_pandera_recipe.py`](09_pandera_recipe.py) | Validating with pandera before and after freshdata cleaning |
| [`10_pyjanitor_interop.py`](10_pyjanitor_interop.py) | Combining explicit PyJanitor transforms with FreshData quality repair |

The PyJanitor example has one optional dependency. FreshData 2.0 supports
pandas 1.5–2.x, so install the compatible PyJanitor 0.31 line before running it:

```bash
pip install "pyjanitor<0.32"
python examples/10_pyjanitor_interop.py
```

See the [documentation](https://freshcode-org.github.io/freshdata/) for full guides.
Loading