Merge pull request Azure#1493 from Azure/release_update/Release-98

update samples from Release-98 as a part of  SDK release

Author: harneetvirk

contrib/fairness/fairlearn-azureml-mitigation.ipynb

@@ -46,10 +46,9 @@
         "Please see the [configuration notebook](../../configuration.ipynb) for information about creating one, if required.\n",
         "This notebook also requires the following packages:\n",
         "* `azureml-contrib-fairness`\n",
-        "* `fairlearn>=0.6.2` (pre-v0.5.0 will work with minor modifications)\n",
+        "* `fairlearn==0.4.6` (v0.5.0 will work with minor modifications)\n",
         "* `joblib`\n",
         "* `liac-arff`\n",
-        "* `raiwidgets==0.4.0`\n",
         "\n",
         "Fairlearn relies on features introduced in v0.22.1 of `scikit-learn`. If you have an older version already installed, please uncomment and run the following cell:"
       ]
@@ -86,7 +85,7 @@
       "outputs": [],
       "source": [
         "from fairlearn.reductions import GridSearch, DemographicParity, ErrorRate\n",
-        "from raiwidgets import FairnessDashboard\n",
+        "from fairlearn.widget import FairlearnDashboard\n",
         "\n",
         "from sklearn.compose import ColumnTransformer\n",
         "from sklearn.impute import SimpleImputer\n",
@@ -257,9 +256,9 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "FairnessDashboard(sensitive_features=A_test,\n",
-        "                  y_true=y_test,\n",
-        "                  y_pred={\"unmitigated\": unmitigated_predictor.predict(X_test)})"
+        "FairlearnDashboard(sensitive_features=A_test, sensitive_feature_names=['Sex', 'Race'],\n",
+        "                   y_true=y_test,\n",
+        "                   y_pred={\"unmitigated\": unmitigated_predictor.predict(X_test)})"
       ]
     },
     {
@@ -312,8 +311,8 @@
         "sweep.fit(X_train, y_train,\n",
         "          sensitive_features=A_train.sex)\n",
         "\n",
-        "# For Fairlearn pre-v0.5.0, need sweep._predictors\n",
-        "predictors = sweep.predictors_"
+        "# For Fairlearn v0.5.0, need sweep.predictors_\n",
+        "predictors = sweep._predictors"
       ]
     },
     {
@@ -330,14 +329,16 @@
       "outputs": [],
       "source": [
         "errors, disparities = [], []\n",
-        "for predictor in predictors:\n",
+        "for m in predictors:\n",
+        "    classifier = lambda X: m.predict(X)\n",
+        "    \n",
         "    error = ErrorRate()\n",
         "    error.load_data(X_train, pd.Series(y_train), sensitive_features=A_train.sex)\n",
         "    disparity = DemographicParity()\n",
         "    disparity.load_data(X_train, pd.Series(y_train), sensitive_features=A_train.sex)\n",
         "    \n",
-        "    errors.append(error.gamma(predictor.predict)[0])\n",
-        "    disparities.append(disparity.gamma(predictor.predict).max())\n",
+        "    errors.append(error.gamma(classifier)[0])\n",
+        "    disparities.append(disparity.gamma(classifier).max())\n",
         "    \n",
         "all_results = pd.DataFrame( {\"predictor\": predictors, \"error\": errors, \"disparity\": disparities})\n",
         "\n",
@@ -386,9 +387,10 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "FairnessDashboard(sensitive_features=A_test, \n",
-        "                  y_true=y_test.tolist(),\n",
-        "                  y_pred=predictions_dominant)"
+        "FairlearnDashboard(sensitive_features=A_test, \n",
+        "                   sensitive_feature_names=['Sex', 'Race'],\n",
+        "                   y_true=y_test.tolist(),\n",
+        "                   y_pred=predictions_dominant)"
       ]
     },
     {
@@ -407,7 +409,7 @@
         "<a id=\"AzureUpload\"></a>\n",
         "## Uploading a Fairness Dashboard to Azure\n",
         "\n",
-        "Uploading a fairness dashboard to Azure is a two stage process. The `FairnessDashboard` invoked in the previous section relies on the underlying Python kernel to compute metrics on demand. This is obviously not available when the fairness dashboard is rendered in AzureML Studio. By default, the dashboard in Azure Machine Learning Studio also requires the models to be registered. The required stages are therefore:\n",
+        "Uploading a fairness dashboard to Azure is a two stage process. The `FairlearnDashboard` invoked in the previous section relies on the underlying Python kernel to compute metrics on demand. This is obviously not available when the fairness dashboard is rendered in AzureML Studio. By default, the dashboard in Azure Machine Learning Studio also requires the models to be registered. The required stages are therefore:\n",
         "1. Register the dominant models\n",
         "1. Precompute all the required metrics\n",
         "1. Upload to Azure\n",

contrib/fairness/fairlearn-azureml-mitigation.yml

@@ -3,7 +3,6 @@ dependencies:
 - pip:
   - azureml-sdk
   - azureml-contrib-fairness
-  - fairlearn>=0.6.2
+  - fairlearn==0.4.6
   - joblib
   - liac-arff
-  - raiwidgets==0.4.0

contrib/fairness/upload-fairness-dashboard.ipynb

@@ -30,7 +30,7 @@
         "1. [Training Models](#TrainingModels)\n",
         "1. [Logging in to AzureML](#LoginAzureML)\n",
         "1. [Registering the Models](#RegisterModels)\n",
-        "1. [Using the Fairness Dashboard](#LocalDashboard)\n",
+        "1. [Using the Fairlearn Dashboard](#LocalDashboard)\n",
         "1. [Uploading a Fairness Dashboard to Azure](#AzureUpload)\n",
         "    1. Computing Fairness Metrics\n",
         "    1. Uploading to Azure\n",
@@ -48,10 +48,9 @@
         "Please see the [configuration notebook](../../configuration.ipynb) for information about creating one, if required.\n",
         "This notebook also requires the following packages:\n",
         "* `azureml-contrib-fairness`\n",
-        "* `fairlearn>=0.6.2` (also works for pre-v0.5.0 with slight modifications)\n",
+        "* `fairlearn==0.4.6` (should also work with v0.5.0)\n",
         "* `joblib`\n",
         "* `liac-arff`\n",
-        "* `raiwidgets==0.4.0`\n",
         "\n",
         "Fairlearn relies on features introduced in v0.22.1 of `scikit-learn`. If you have an older version already installed, please uncomment and run the following cell:"
       ]
@@ -389,11 +388,12 @@
       "metadata": {},
       "outputs": [],
       "source": [
-        "from raiwidgets import FairnessDashboard\n",
+        "from fairlearn.widget import FairlearnDashboard\n",
         "\n",
-        "FairnessDashboard(sensitive_features=A_test, \n",
-        "                  y_true=y_test.tolist(),\n",
-        "                  y_pred=ys_pred)"
+        "FairlearnDashboard(sensitive_features=A_test, \n",
+        "                   sensitive_feature_names=['Sex', 'Race'],\n",
+        "                   y_true=y_test.tolist(),\n",
+        "                   y_pred=ys_pred)"
       ]
     },
     {
@@ -403,7 +403,7 @@
         "<a id=\"AzureUpload\"></a>\n",
         "## Uploading a Fairness Dashboard to Azure\n",
         "\n",
-        "Uploading a fairness dashboard to Azure is a two stage process. The `FairnessDashboard` invoked in the previous section relies on the underlying Python kernel to compute metrics on demand. This is obviously not available when the fairness dashboard is rendered in AzureML Studio. The required stages are therefore:\n",
+        "Uploading a fairness dashboard to Azure is a two stage process. The `FairlearnDashboard` invoked in the previous section relies on the underlying Python kernel to compute metrics on demand. This is obviously not available when the fairness dashboard is rendered in AzureML Studio. The required stages are therefore:\n",
         "1. Precompute all the required metrics\n",
         "1. Upload to Azure\n",
         "\n",

contrib/fairness/upload-fairness-dashboard.yml

@@ -3,7 +3,6 @@ dependencies:
 - pip:
   - azureml-sdk
   - azureml-contrib-fairness
-  - fairlearn>=0.6.2
+  - fairlearn==0.4.6
   - joblib
   - liac-arff
-  - raiwidgets==0.4.0