Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Robust Scaler now added to the Normalizer catalog #5166

Merged
merged 7 commits into from
May 29, 2020

Conversation

michaelgsharp
Copy link
Member

One of the featurizers that was added in the past was the RobustScalerFeaturizer. After talking with the ML.Net team, it was decided that since this is just a normalizer it makes more sense to have it be part of the existing collection of normalizers that ML.Net already has. This PR adds that functionality. It is the equivalent of the RobustScaler in pandas. Once this PR goes in, we can remove the original transformer that was added.

@michaelgsharp michaelgsharp requested review from harishsk and a team May 26, 2020 21:50
@michaelgsharp michaelgsharp self-assigned this May 26, 2020
@harishsk
Copy link
Contributor

/// Check the above links for usage examples.

Please add an entry here for [NormalizeRobustScaling]


Refers to: src/Microsoft.ML.Data/Transforms/Normalizer.cs:75 in 00d4ce7. [](commit_id = 00d4ce7, deletion_comment = False)

@codecov
Copy link

codecov bot commented May 28, 2020

Codecov Report

Merging #5166 into master will increase coverage by 13.17%.
The diff coverage is 97.17%.

@@             Coverage Diff             @@
##           master    #5166       +/-   ##
===========================================
+ Coverage   75.81%   88.98%   +13.17%     
===========================================
  Files         993      237      -756     
  Lines      181224    43385   -137839     
  Branches    19510     2276    -17234     
===========================================
- Hits       137387    38606    -98781     
+ Misses      38538     4460    -34078     
+ Partials     5299      319     -4980     
Flag Coverage Δ
#Debug 88.98% <97.17%> (+13.17%) ⬆️
#production ?
#test 88.98% <97.17%> (+0.05%) ⬆️
Impacted Files Coverage Δ
test/Microsoft.ML.Tests/OnnxConversionTest.cs 99.20% <94.52%> (-0.22%) ⬇️
...Microsoft.ML.Tests/Transformers/NormalizerTests.cs 99.56% <98.09%> (-0.44%) ⬇️
src/Microsoft.ML.Core/Data/ModelSaveContext.cs
src/Microsoft.ML.Core/Data/IEstimator.cs
...soft.ML.Data/Transforms/LabelIndicatorTransform.cs
...ML.Data/Transforms/ConversionsExtensionsCatalog.cs
...icrosoft.ML.AutoML/Experiment/SuggestedPipeline.cs
src/Microsoft.ML.ImageAnalytics/ImageResizer.cs
src/Microsoft.ML.Core/Data/ModelSaving.cs
src/Microsoft.ML.Data/EntryPoints/InputBase.cs
... and 742 more

[CombinatorialData]
public void NormalizingEstimatorConversionTests(
bool fixZero)
{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice refactoring overall! :)

Copy link
Contributor

@harishsk harishsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@michaelgsharp michaelgsharp merged commit 413cf82 into dotnet:master May 29, 2020
@michaelgsharp michaelgsharp deleted the robust-scaler branch November 4, 2020 20:36
@ghost ghost locked as resolved and limited conversation to collaborators Mar 18, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants