Skip to content
This repository was archived by the owner on Sep 27, 2024. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
## Description

<!-- Please describe what you changed and why. -->

Fixes [issue link]

## Checklist

Before submitting a pull request, please be sure to do the following:
- [ ] Read the [How to Contribute guide](https://github.com/tensorflow/model-card-toolkit/blob/master/CONTRIBUTING.md)
if this is your first contribution.
- [ ] Open an issue to discuss this change.
- [ ] Update documentation if applicable.
- [ ] Write new tests if applicable.
24 changes: 24 additions & 0 deletions .github/workflows/cherrypick.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: Cherry pick
on:
issue_comment:
types: [created]
jobs:
cherry-pick:
name: Cherry Pick
# Only cherry pick if user is a release manager
# NB(gcasassaez): We unfortunately have to use fromJSON as GitHub doesn't have a way to specify constant arrays
# See: https://github.community/t/passing-an-array-literal-to-contains-function-causes-syntax-error/17213/3
if: github.event.issue.pull_request != '' && contains(github.event.comment.body, '/cherry-pick') && contains(fromJson('["casassg", "hanneshapke", "codesue"]'), github.event.sender.login)
runs-on: ubuntu-latest
steps:

- name: Checkout the latest code
uses: actions/checkout@v2
with:
token: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 # otherwise, you will fail to push refs to the dest repo

- name: Automatic cherry pick
uses: vendoo/gha-cherry-pick@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
53 changes: 53 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Release Model Card Toolkit package to PyPI and TestPyPI

on:
push:
paths:
- 'model_card_toolkit/**'
- 'setup.py'
branches:
- master
- r*
release:
types: [published]
tags:
- v*

jobs:
build-and-publish:
name: Build Model Card Toolkit package and release to PyPI and Test PyPI
runs-on: ubuntu-latest
container:
image: gcr.io/tfx-oss-public/tfx_base:py37-20200729
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codesue This is very nit, so apologies. Do we need to the tfx-base image here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This image was convenient because it has the right versions of the tools need to build the wheel, e.g. bazel. It's used to build tfx wheels, which have similar build steps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, perfect! Thank you for the explanation @codesue!

steps:
- uses: actions/checkout@v2

- name: Build wheel
run: |
bash -x package_build/initialize.sh; \
CFLAGS=$(/usr/bin/python-config --cflags) \
python package_build/model-card-toolkit/setup.py bdist_wheel; \
MCT_WHEEL=$(find dist -name 'model_card_toolkit-*.whl'); \
- name: Publish Model Card Toolkit distribution package to Test PyPI
uses: pypa/gh-action-pypi-publish@v1.5.0
with:
password: ${{ secrets.TEST_PYPI_API_TOKEN }}
repository_url: https://test.pypi.org/legacy/
skip_existing: true

- name: Publish Model Card Toolkit distribution package to PyPI
if: github.event_name == 'release'
uses: pypa/gh-action-pypi-publish@v1.5.0
with:
password: ${{ secrets.PYPI_API_TOKEN }}

- name: Upload files to a GitHub release
uses: svenstaro/upload-release-action@2.2.1
if: github.event_name == 'release'
with:
repo_token: ${{ secrets.GITHUB_TOKEN }}
file: dist/*
tag: ${{ github.ref }}
overwrite: true
file_glob: true
126 changes: 80 additions & 46 deletions RELEASE.md
Original file line number Diff line number Diff line change
@@ -1,47 +1,81 @@
<!-- mdlint off(HEADERS_TOO_MANY_H1) -->
# Model Card Toolkit Releases

# Release 1.3.2
TFX Addons follows [Semantic Versioning 2.0](https://semver.org/) strategy.

## Major Features and Improvements
* See the [Release Notes](https://github.com/tensorflow/model-card-toolkit/releases) for versions `>1.3.2`.
* See [Past Releases](#past-releases) for release notes for versions `<=1.3.2`.

## Bug fixes and other changes
## Making Major and Minor Releases

1. Create new `rX.Y` branch on https://github.com/tensorflow/model-card-toolkit from `master`.
2. Create new PR with updates to `version.py` against `rX.Y` branch.
* Set the correct version and suffix in [version.py](https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/version.py).
* Ensure proper supported Python libraries are set in [setup.py](https://github.com/tensorflow/model-card-toolkit/blob/master/setup.py).
3. Create a [new release](https://github.com/tensorflow/model-card-toolkit/releases) from `rX.Y` branch. Create a tag with `vX.Y.Z` name.
* Add updates for new features, enhancements, bug fixes
* Add contributors using `git shortlog <last-version>..HEAD -s`
4. Create a new PR and merge an increase of `_MINOR_VERSION` in [version.py](https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/version.py) on `master` to get ready for next release.

## Making Patch Releases
1. Cherry-pick commits to `rX.Y` branch. Release team can just port PR by commenting "/cherry-pick rX.Y" in a merged PR.
2. Create new PR with increasing `_PATCH_VERSION` in `version.py` against `rX.Y` branch.
* Set the correct version and suffix in [version.py](https://github.com/tensorflow/model-card-toolkit/blob/master/model_card_toolkit/version.py).
* Ensure proper supported Python libraries are set in [setup.py](https://github.com/tensorflow/model-card-toolkit/blob/master/setup.py).
3. Create a [new release](https://github.com/tensorflow/model-card-toolkit/releases) from `rX.Y` branch. Create a tag with `vX.Y.Z` name.
* Add updates for new features, enhancements, bug fixes
* Add contributors using `git shortlog <last-version>..HEAD -s`

## Model Card Toolkit Release Team

Current Release Team:

- Hannes Hapke - @hanneshapke
- Gerard Casas Saez - @casassg
- Suzen Fylke - @codesue

## Past Releases
### Release 1.3.2

#### Major Features and Improvements

#### Bug fixes and other changes

* Remove `semantic-version` dependency.
* Update `jinja2` dependency.

## Breaking changes and Deprecations
#### Breaking changes and Deprecations

# Release 1.3.1
### Release 1.3.1

## Major Features and Improvements
#### Major Features and Improvements

## Bug fixes and other changes
#### Bug fixes and other changes

* Fix bug where `ModelCardGenerator` fails if `template_io` is unspecified.

## Breaking changes and Deprecations
#### Breaking changes and Deprecations

# Release 1.3.0
### Release 1.3.0

## Major Features and Improvements
#### Major Features and Improvements

* `ModelCardArtifact`, a TFX/MLMD artifact which references the assets generated by MCT. This can be created with `model_card_toolkit.tfx.create_and_save_artifact()`.
* `tfx.executor.Executor`, which can be used as the core MCT logic for a [TFX custom component](https://www.tensorflow.org/tfx/guide/custom_component).
* `tfx.component.ModelCardGenerator`, a [component](https://www.tensorflow.org/tfx/guide/custom_component) that can be used to generate Model Cards directly in a TFX pipeline.

## Bug fixes and other changes
#### Bug fixes and other changes

* `ModelCard.merge_from_json()`, allowing `ModelCard`s to be populated with JSON without overwriting all pre-existing fields.
* Minor styling change to table CSS in default HTML template
* Minor update to CSS linting in default HTML template

## Breaking changes and Deprecations
#### Breaking changes and Deprecations

* `source` classes now uses `tfx.types.Artifact` instead of `standard_artifacts`.

# Release 1.2.0
### Release 1.2.0

## Major Features and Improvements
#### Major Features and Improvements

* `ModelCard` updates
* Fields
Expand All @@ -56,7 +90,7 @@
* `MlmdSource`: Args to extract data from TFX artifacts in MLMD. Contains the `mlmd_store` and `model_uri` args. Previously, these were args to `ModelCardToolkit`.
* `Source`: Args to extract data from TFX artifacts outside MLMD (by passing in a path to the artifact, or by passing in the artifact directly). Contains `tfma`, `tfdv`, and `model` args.

## Bug fixes and other changes
#### Bug fixes and other changes

* `ModelCardToolkit`
* `model_card.quantitative_analysis.performance_metrics` is now populated when a `tfma.EvalResult` is found in MLMD store.
Expand All @@ -69,66 +103,66 @@
* `PerformanceMetrics`
* Add `confidence_interval` field.

## Breaking changes and Deprecations
#### Breaking changes and Deprecations

* Replace `ModelCardToolkit(output_dir, mlmd_store, model_uri)` with `ModelCardToolkit(output_dir, mlmd_source, source)`. See "Major Features and Improvements" above for details.
* Complete deprecation of `ModelCardToolkit.update_model_card_json()`. Users should migrate to `ModelCardToolkit.update_model_card()`, which uses a proto representation. Alternatively, users can use `ModelCard.to_json()` and `ModelCard.from_json()` to interact with JSON representations.

# Release 1.1.0
### Release 1.1.0

## Major Features and Improvements
#### Major Features and Improvements

## Bug fixes and other changes
#### Bug fixes and other changes

* Update TFX compatibility to TFX 1.2.
* Fix bug where all datasets from MLMD were being compressed into one model_card.Dataset object.

## Breaking changes and Deprecations
#### Breaking changes and Deprecations

# Release 1.0.0
### Release 1.0.0

## Major Features and Improvements
#### Major Features and Improvements

* Introduce `model_card.proto`. See https://developers.google.com/protocol-buffers for more info.
* All classes in `model_card_toolkit.model_card` submodule now have `to_proto()`, `merge_from_proto()`, `copy_from_proto()`, and `clear()` functions.
* `ModelCardToolkit.export_format()` now accepts `model_card` arg.
* `json_util.update()`, which updates a v0.0.1 JSON object to a v0.0.2 JSON object.

## Bug fixes and other changes
#### Bug fixes and other changes

* Update default template layout so charts can wrap to multiple rows
* Installing from source now requires [Bazel](https://docs.bazel.build/versions/master/install.html)>=2.0.0.
* Update model card templates to use new schema.
* `model_card_toolkit.utils.validation.validate_json_schema()` can now validate both schema v0.0.1 and v0.0.2.
* Add `_jinja_loader` attribute to `model_card_toolkit` to allow custom Jinja loaders.

## Breaking changes
#### Breaking changes

* JSON schema v0.0.2 replaces JSON schema v0.0.1.
* `ModelCardToolkit.update_model_card_json()` deprecated and replaced with `ModelCardToolkit.update_model_card()`. Writes to `data/model_card.proto` instead of `data/model_card.json`.
* `graphics.annotate_dataset_feature_statistics_plots` accepts a list of stats files instead of two individual train and eval stats params.

## Deprecations
#### Deprecations

# Release 0.1.3
### Release 0.1.3

## Major Features and Improvements
#### Major Features and Improvements

## Bug fixes and other changes
#### Bug fixes and other changes

* Update model_card.py docstrings. Now hosted on [Responsible AI](https://www.tensorflow.org/responsible_ai/model_card_toolkit/api_docs/python/model_card_toolkit).

## Breaking changes
#### Breaking changes

## Deprecations
#### Deprecations

# Release 0.1.2
### Release 0.1.2

## Major Features and Improvements
#### Major Features and Improvements

* In default_template.md.jinja and default_template.html.jinja, generate metrics table from `quantitative_analysis.performance_metrics`.

## Bug fixes and other changes
#### Bug fixes and other changes

* Reference URLs in default HTML and Markdown template are now hyperlinks
* Fix bug where Considerations div is displayed in HTML model cards, even if Considerations div is empty.
Expand All @@ -140,37 +174,37 @@
* Add model_card_toolkit.validation.validate_json_schema(), a function to validate a Python dictionary against the Model Card JSON schema.
* Fix the bug that some slices may have extra metrics that other slices does not have. e.g. __ERROR__ metric.

## Breaking changes
#### Breaking changes

## Deprecations
#### Deprecations

# Release 0.1.1
### Release 0.1.1

## Major Features and Improvements
#### Major Features and Improvements

* add Markdown template

## Bug fixes and other changes
#### Bug fixes and other changes

* remove `quantitative_analysis` from required fields
* add `input_format` and `export_format` fields
* add `model_architecture`, `input_format`, and `export_format` to HTML template
* add Cats vs Dogs util for `Standalone_Model_Card_toolkit_Demo.ipynb`

## Breaking changes
#### Breaking changes

* Rename `_figure_to_base64str` to `figure_to_base64str`

## Deprecations
#### Deprecations

# Release 0.1.0
### Release 0.1.0

## Major Features and Improvements
#### Major Features and Improvements

Initial release of Model Card Toolkit.

## Bug fixes and other changes
#### Bug fixes and other changes

## Breaking changes
#### Breaking changes

## Deprecations
#### Deprecations
8 changes: 8 additions & 0 deletions WORKSPACE
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,14 @@ http_archive(
],
)

http_archive(
name = "zlib",
build_file = "@com_google_protobuf//:third_party/zlib.BUILD",
sha256 = "629380c90a77b964d896ed37163f5c3a34f6e6d897311f1df2a7016355c45eff",
strip_prefix = "zlib-1.2.11",
urls = ["https://github.com/madler/zlib/archive/v1.2.11.tar.gz"],
)

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")

rules_proto_dependencies()
Expand Down
Loading